From 2fdbac63d51e56772c644fbcdf189e0f3bb109dd Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Sat, 26 Sep 2026 16:16:32 -0700 Subject: [PATCH] feat(vibevoice-asr-seat): switch to Q8_0 (Prime); WER 2/69 vs 3/69 on the bundled clips, +1.1 GB VRAM --- persistent-memory.md | 1 + servers/nh3-ml1/README.md | 2 +- stacks/vibevoice-asr-seat/README.md | 12 ++++++++---- stacks/vibevoice-asr-seat/conf/server.json | 2 +- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/persistent-memory.md b/persistent-memory.md index 36721ff..8254bd2 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -218,6 +218,7 @@ in `servers/pfi-gx10/README.md`. → `persistent-memory.d/2026-09-24-gx10-ac-res ## Recent decisions +- `[2026-09-26]` **VibeVoice ASR → Q8_0 (Prime).** WER on the 4 bundled LibriSpeech clips 3/69 → 2/69 (only the I'm/I am artifact left), ×3 identical; RTF 0.09–0.17; +1.1 GB VRAM (nh3-ml1 ~11.3/16 GB). Q4_K file removed. - `[2026-09-26]` **esh-matter LIVE: a Matter server (matter.js 1.4.0) on CT 111 @ 10.0.90.20, VLAN 90 only**, for ha-dev (operator-approved, relayed). It is on the IoT link because Matter/Thread IPv6 (Echo ULA + RIO routes) is link-only. Kernel RA with rt_info_max_plen=64, forwarding off; Docker ip-forward/iptables off. The in-CT nftables admits :5580 from HA 10.0.50.46 only and SSH from mgmt ranges only, and accepts everything else so Matter UDP never meets conntrack. The CT is in esh-pve vzdump (fabric creds). Acceptance all green: fdad:: SLAAC; ping6 thermostat; 2 Thread routes learned (Aqara + Echo); ws OK from inside the HA container; refused from 10.0.50.45, nh3-dev and a temp VLAN 90 netns. → `servers/esh-matter/README.md` - `[2026-09-26]` **Embed/rerank LOAD-SHARED across esh-ml1 + nh3-ml1 (Prime).** Second deployments were added for qwen3-embedding and reranker (config) and for reranker-a3-bge-v2-m3 (DB, `/model/new`). Without failover, a dead backend failed 7/40 embeds; `router_settings.enable_weighted_failover: true` (affects only multi-deployment groups) brought it to rerank 80/80 and embed 38/40 at onset, 60/60 sustained. In LiteLLM 1.97 the embedding path does not stamp the failed deployment, so it relies on cooldown. LiteLLM restarted 3× tonight, ~36 s each. - `[2026-09-26]` **Two dataset-foundry utility seats LIVE on nh3-ml1 (brokkr; operator approval relayed):** LFM2.5-VL-3B on llama.cpp `:8030` (gateway `lfm25-vl-3b`, LiteLLM restarted 36 s at 0039) and VibeVoice-ASR-Streaming-1.5B on **audio.cpp** `:8031`, not vibevoice.cpp as specced; the GGUF card names audio.cpp. Controls: VL read a synthetic image exactly, but hallucinates when no image is sent; ASR WER 3/69 on the bundled LibriSpeech clips. libsoxr added (linear resampling misheard a word). First ASR request ~31 s cold. Answered brokkr: the old 27B image seat (10.250.50.54:8015) was retired 09-14; baseline = gateway `image-judge` (Flash-Next). diff --git a/servers/nh3-ml1/README.md b/servers/nh3-ml1/README.md index 2dbe6be..90720c5 100644 --- a/servers/nh3-ml1/README.md +++ b/servers/nh3-ml1/README.md @@ -56,7 +56,7 @@ VRAM ~2.7 GB for both, so ~13 GB is free. - `stacks/lfm-vl-uncensored-seat`: an abliterated LFM2.5-VL-3B running in parallel for brokkr's NSFW-caption A/B, `:8032`, direct only. -GPU total with TEI is ~10.2 of 16 GB. All of these are batch workloads, which +GPU total with TEI is ~11.3 of 16 GB (the ASR seat moved to Q8_0 on 2026-09-26). All of these are batch workloads, which suits this card. **Tried and removed (2026-09-25):** a copy of the code-completion seat diff --git a/stacks/vibevoice-asr-seat/README.md b/stacks/vibevoice-asr-seat/README.md index 07782f1..4018a2b 100644 --- a/stacks/vibevoice-asr-seat/README.md +++ b/stacks/vibevoice-asr-seat/README.md @@ -1,6 +1,6 @@ # vibevoice-asr-seat -**Microsoft VibeVoice-ASR-Streaming-1.5B** (Q4_K) on **nh3-ml1**, served by +**Microsoft VibeVoice-ASR-Streaming-1.5B** (**Q8_0** since 2026-09-26; was Q4_K) on **nh3-ml1**, served by **audio.cpp** (`0xShug0/audio.cpp` v0.8.2-audio8-perf-hotfix, Linux CUDA 12.8 build) on `:8031`, with direct access only. A utility seat for brokkr's dataset foundry (speech → text with inline speaker labels). Operator-approved 2026-09-26, @@ -25,6 +25,10 @@ is 3/69 = 4.35%, identical across 3 reps. Two of the three errors are "I'm" vs "I am" normalization. RTF is 0.07–0.14. ⚠ **The first request after a start takes ~31 s** (CUDA graph warmup); later ones take 0.3–1 s. VRAM ~2.0 GB. -The model card recommends Q8_0 (3.3 GB; its CUDA WER is 5.80% vs Q4_K's 7.25%, -on 69 words). To switch, change `path` in `conf/server.json`, download the file, -and recreate the container. +**Switched to Q8_0 (Prime, 2026-09-26 1616).** It is the card's recommended +package: 3.3 GB on disk, ~1.1 GB more VRAM. On the same 4 clips, WER went from +3/69 to **2/69 = 2.90%**, identical over 3 reps. The remaining 2 are the +"I'm"/"I am" normalization artifact, so no real errors are left; Q4_K's +"forwarded→folded" is gone. RTF is 0.09–0.17 (Q4_K was 0.07–0.14). This is 69 +words, so it separates working from broken, not a fine ranking. The Q4_K file was +removed. To roll back, download it again and point `conf/server.json` at it. diff --git a/stacks/vibevoice-asr-seat/conf/server.json b/stacks/vibevoice-asr-seat/conf/server.json index bcc27ba..32a21ef 100644 --- a/stacks/vibevoice-asr-seat/conf/server.json +++ b/stacks/vibevoice-asr-seat/conf/server.json @@ -7,7 +7,7 @@ { "id": "vibevoice-asr-streaming-1.5b", "family": "vibevoice_asr_streaming", - "path": "/models/vibevoice-asr-streaming-1.5b-q4_k.gguf", + "path": "/models/vibevoice-asr-streaming-1.5b-q8_0.gguf", "task": "asr", "mode": "streaming" }