2.1 KiB
vibevoice-asr-seat
Microsoft VibeVoice-ASR-Streaming-1.5B (Q8_0 since 2026-09-26; was Q4_K) on nh3-ml1, served by
audio.cpp (0xShug0/audio.cpp v0.8.2-audio8-perf-hotfix, Linux CUDA 12.8
build) on :8031, with direct access only. A utility seat for brokkr's dataset
foundry (speech → text with inline speaker labels). Operator-approved 2026-09-26,
relayed by brokkr-smithy-dev.
POST /v1/audio/transcriptions: multipartfile=@x.wav,model=vibevoice-asr-streaming-1.5b. Returns{"text": " \n Speaker 0:...", "timing": {"rtf": ...}}.POST /v1/audio/transcriptions/live?model=…&sample_rate=16000&channels=1&sample_format=s16letakes chunked raw PCM.GET /v1/models,GET /health.
Why audio.cpp and not vibevoice.cpp or llama.cpp. The
christopherthompson81/VibeVoice-ASR-Streaming-1.5B-GGUF files are audio.cpp
packages with their sidecars embedded, per the model card, so no separate
tokenizer is needed. The release binary expects the CUDA 12 runtime libraries,
so Dockerfile puts it on nvidia/cuda:12.8.1-runtime with a SHA-256-pinned
download, plus libgomp1 and libsoxr0. Without libsoxr it falls back to
linear resampling (16k→24k), and on one clip that turned "cutter" into "country".
Checks, 2026-09-26. On the 4 LibriSpeech clips shipped with audio.cpp, WER is 3/69 = 4.35%, identical across 3 reps. Two of the three errors are "I'm" vs "I am" normalization. RTF is 0.07–0.14. ⚠ The first request after a start takes ~31 s (CUDA graph warmup); later ones take 0.3–1 s. VRAM ~2.0 GB.
Switched to Q8_0 (Prime, 2026-09-26 1616). It is the card's recommended
package: 3.3 GB on disk, ~1.1 GB more VRAM. On the same 4 clips, WER went from
3/69 to 2/69 = 2.90%, identical over 3 reps. The remaining 2 are the
"I'm"/"I am" normalization artifact, so no real errors are left; Q4_K's
"forwarded→folded" is gone. RTF is 0.09–0.17 (Q4_K was 0.07–0.14). This is 69
words, so it separates working from broken, not a fine ranking. The Q4_K file was
removed. To roll back, download it again and point conf/server.json at it.