feat(vibevoice-asr-seat): switch to Q8_0 (Prime); WER 2/69 vs 3/69 on the bundled clips, +1.1 GB VRAM
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# vibevoice-asr-seat
|
||||
|
||||
**Microsoft VibeVoice-ASR-Streaming-1.5B** (Q4_K) on **nh3-ml1**, served by
|
||||
**Microsoft VibeVoice-ASR-Streaming-1.5B** (**Q8_0** since 2026-09-26; was Q4_K) on **nh3-ml1**, served by
|
||||
**audio.cpp** (`0xShug0/audio.cpp` v0.8.2-audio8-perf-hotfix, Linux CUDA 12.8
|
||||
build) on `:8031`, with direct access only. A utility seat for brokkr's dataset
|
||||
foundry (speech → text with inline speaker labels). Operator-approved 2026-09-26,
|
||||
@@ -25,6 +25,10 @@ is 3/69 = 4.35%, identical across 3 reps. Two of the three errors are "I'm" vs
|
||||
"I am" normalization. RTF is 0.07–0.14. ⚠ **The first request after a start
|
||||
takes ~31 s** (CUDA graph warmup); later ones take 0.3–1 s. VRAM ~2.0 GB.
|
||||
|
||||
The model card recommends Q8_0 (3.3 GB; its CUDA WER is 5.80% vs Q4_K's 7.25%,
|
||||
on 69 words). To switch, change `path` in `conf/server.json`, download the file,
|
||||
and recreate the container.
|
||||
**Switched to Q8_0 (Prime, 2026-09-26 1616).** It is the card's recommended
|
||||
package: 3.3 GB on disk, ~1.1 GB more VRAM. On the same 4 clips, WER went from
|
||||
3/69 to **2/69 = 2.90%**, identical over 3 reps. The remaining 2 are the
|
||||
"I'm"/"I am" normalization artifact, so no real errors are left; Q4_K's
|
||||
"forwarded→folded" is gone. RTF is 0.09–0.17 (Q4_K was 0.07–0.14). This is 69
|
||||
words, so it separates working from broken, not a fine ranking. The Q4_K file was
|
||||
removed. To roll back, download it again and point `conf/server.json` at it.
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
{
|
||||
"id": "vibevoice-asr-streaming-1.5b",
|
||||
"family": "vibevoice_asr_streaming",
|
||||
"path": "/models/vibevoice-asr-streaming-1.5b-q4_k.gguf",
|
||||
"path": "/models/vibevoice-asr-streaming-1.5b-q8_0.gguf",
|
||||
"task": "asr",
|
||||
"mode": "streaming"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user