# vibevoice-asr-seat

**Microsoft VibeVoice-ASR-Streaming-1.5B** (**Q8_0** since 2026-09-26; was Q4_K) on **nh3-ml1**, served by
**audio.cpp** (`0xShug0/audio.cpp` v0.8.2-audio8-perf-hotfix, Linux CUDA 12.8
build) on `:8031`, with direct access only. A utility seat for brokkr's dataset
foundry (speech → text with inline speaker labels). Operator-approved 2026-09-26,
relayed by brokkr-smithy-dev.

- `POST /v1/audio/transcriptions`: multipart `file=@x.wav`, `model=vibevoice-asr-streaming-1.5b`.
  Returns `{"text": " \n Speaker 0:...", "timing": {"rtf": ...}}`.
- `POST /v1/audio/transcriptions/live?model=…&sample_rate=16000&channels=1&sample_format=s16le`
  takes chunked raw PCM.
- `GET /v1/models`, `GET /health`.

**Why audio.cpp and not vibevoice.cpp or llama.cpp.** The
`christopherthompson81/VibeVoice-ASR-Streaming-1.5B-GGUF` files are audio.cpp
packages with their sidecars embedded, per the model card, so no separate
tokenizer is needed. The release binary expects the CUDA 12 runtime libraries,
so `Dockerfile` puts it on `nvidia/cuda:12.8.1-runtime` with a SHA-256-pinned
download, plus `libgomp1` and **`libsoxr0`**. Without libsoxr it falls back to
linear resampling (16k→24k), and on one clip that turned "cutter" into "country".

**Checks, 2026-09-26.** On the 4 LibriSpeech clips shipped with audio.cpp, WER
is 3/69 = 4.35%, identical across 3 reps. Two of the three errors are "I'm" vs
"I am" normalization. RTF is 0.07–0.14. ⚠ **The first request after a start
takes ~31 s** (CUDA graph warmup); later ones take 0.3–1 s. VRAM ~2.0 GB.

**Switched to Q8_0 (Prime, 2026-09-26 1616).** It is the card's recommended
package: 3.3 GB on disk, ~1.1 GB more VRAM. On the same 4 clips, WER went from
3/69 to **2/69 = 2.90%**, identical over 3 reps. The remaining 2 are the
"I'm"/"I am" normalization artifact, so no real errors are left; Q4_K's
"forwarded→folded" is gone. RTF is 0.09–0.17 (Q4_K was 0.07–0.14). This is 69
words, so it separates working from broken, not a fine ranking. The Q4_K file was
removed. To roll back, download it again and point `conf/server.json` at it.