feat(vibevoice-asr-seat): switch to Q8_0 (Prime); WER 2/69 vs 3/69 on the bundled clips, +1.1 GB VRAM

This commit is contained in:
vh
2026-09-26 16:16:32 -07:00
parent 8e7ae0675d
commit 2fdbac63d5
4 changed files with 11 additions and 6 deletions
+8 -4
View File
@@ -1,6 +1,6 @@
# vibevoice-asr-seat
**Microsoft VibeVoice-ASR-Streaming-1.5B** (Q4_K) on **nh3-ml1**, served by
**Microsoft VibeVoice-ASR-Streaming-1.5B** (**Q8_0** since 2026-09-26; was Q4_K) on **nh3-ml1**, served by
**audio.cpp** (`0xShug0/audio.cpp` v0.8.2-audio8-perf-hotfix, Linux CUDA 12.8
build) on `:8031`, with direct access only. A utility seat for brokkr's dataset
foundry (speech → text with inline speaker labels). Operator-approved 2026-09-26,
@@ -25,6 +25,10 @@ is 3/69 = 4.35%, identical across 3 reps. Two of the three errors are "I'm" vs
"I am" normalization. RTF is 0.07–0.14. ⚠ **The first request after a start
takes ~31 s** (CUDA graph warmup); later ones take 0.3–1 s. VRAM ~2.0 GB.
The model card recommends Q8_0 (3.3 GB; its CUDA WER is 5.80% vs Q4_K's 7.25%,
on 69 words). To switch, change `path` in `conf/server.json`, download the file,
and recreate the container.
**Switched to Q8_0 (Prime, 2026-09-26 1616).** It is the card's recommended
package: 3.3 GB on disk, ~1.1 GB more VRAM. On the same 4 clips, WER went from
3/69 to **2/69 = 2.90%**, identical over 3 reps. The remaining 2 are the
"I'm"/"I am" normalization artifact, so no real errors are left; Q4_K's
"forwarded→folded" is gone. RTF is 0.09–0.17 (Q4_K was 0.07–0.14). This is 69
words, so it separates working from broken, not a fine ranking. The Q4_K file was
removed. To roll back, download it again and point `conf/server.json` at it.
+1 -1
View File
@@ -7,7 +7,7 @@
{
"id": "vibevoice-asr-streaming-1.5b",
"family": "vibevoice_asr_streaming",
"path": "/models/vibevoice-asr-streaming-1.5b-q4_k.gguf",
"path": "/models/vibevoice-asr-streaming-1.5b-q8_0.gguf",
"task": "asr",
"mode": "streaming"
}