feat(nh3-ml1): LFM2.5-VL-3B (llama.cpp) + VibeVoice-ASR-Streaming-1.5B (audio.cpp) utility seats

For brokkr's dataset foundry (operator-approved 2026-09-26, relayed). - stacks/lfm-vl-seat: llama.cpp server-cuda b11176 (digest-pinned), Q5_K_M + mmproj Q8_0, :8030; gateway alias lfm25-vl-3b (LiteLLM restarted, 36 s). Positive control exact; null control shows it describes a missing image. - stacks/vibevoice-asr-seat: audio.cpp v0.8.2-audio8-perf-hotfix (the GGUF's own runtime, not vibevoice.cpp) on cuda 12.8 runtime + libgomp + libsoxr, sha256-pinned; :8031 direct. LibriSpeech WER 3/69, RTF 0.07-0.14; ~31 s cold first request.
2026-09-26 00:41:01 -07:00
parent 45484a0007
commit f2792183d4
11 changed files with 235 additions and 0 deletions
@@ -0,0 +1,2 @@
+# vibevoice-asr-seat tunables (nh3-ml1). Copy to `.env` on the server.
+ASR_PORT=8031
@@ -0,0 +1,16 @@
+# audio.cpp server (0xShug0/audio.cpp) for VibeVoice-ASR-Streaming on nh3-ml1.
+# The release's Linux CUDA build ("cuda12.8-colab") is a bare binary that needs the
+# CUDA 12 runtime libs (cudart, cublas, cufft, nccl) plus libgomp. This image
+# supplies them, plus libsoxr (without it audio.cpp falls back to linear
+# resampling for the 16k→24k step). Built locally on the host; not pushed.
+FROM nvidia/cuda:12.8.1-runtime-ubuntu22.04
+ARG AUDIOCPP_TAG=v0.8.2-audio8-perf-hotfix
+ARG AUDIOCPP_SHA256=ccfb35869d67e520801981392189a1ab8b6c829151093e0816ecd6a2d828d8c8
+RUN apt-get update && apt-get install -y --no-install-recommends libgomp1 libsoxr0 curl ca-certificates \
+ && rm -rf /var/lib/apt/lists/*
+RUN curl -fsSL -o /tmp/a.tgz "https://github.com/0xShug0/audio.cpp/releases/download/${AUDIOCPP_TAG}/audio-${AUDIOCPP_TAG}-bin-ubuntu-x64-cuda12.8-colab.tar.gz" \
+ && echo "${AUDIOCPP_SHA256}  /tmp/a.tgz" | sha256sum -c - \
+ && mkdir -p /app && tar xzf /tmp/a.tgz -C /app --no-same-owner && rm /tmp/a.tgz \
+ && chmod +x /app/audiocpp_server
+WORKDIR /app
+ENTRYPOINT ["/app/audiocpp_server"]
@@ -0,0 +1,30 @@
+# vibevoice-asr-seat
+
+**Microsoft VibeVoice-ASR-Streaming-1.5B** (Q4_K) on **nh3-ml1**, served by
+**audio.cpp** (`0xShug0/audio.cpp` v0.8.2-audio8-perf-hotfix, Linux CUDA 12.8
+build) on `:8031`, with direct access only. A utility seat for brokkr's dataset
+foundry (speech → text with inline speaker labels). Operator-approved 2026-09-26,
+relayed by brokkr-smithy-dev.
+
+- `POST /v1/audio/transcriptions`: multipart `file=@x.wav`, `model=vibevoice-asr-streaming-1.5b`.
+  Returns `{"text": " \n Speaker 0:...", "timing": {"rtf": ...}}`.
+- `POST /v1/audio/transcriptions/live?model=…&sample_rate=16000&channels=1&sample_format=s16le`
+  takes chunked raw PCM.
+- `GET /v1/models`, `GET /health`.
+
+**Why audio.cpp and not vibevoice.cpp or llama.cpp.** The
+`christopherthompson81/VibeVoice-ASR-Streaming-1.5B-GGUF` files are audio.cpp
+packages with their sidecars embedded, per the model card, so no separate
+tokenizer is needed. The release binary expects the CUDA 12 runtime libraries,
+so `Dockerfile` puts it on `nvidia/cuda:12.8.1-runtime` with a SHA-256-pinned
+download, plus `libgomp1` and **`libsoxr0`**. Without libsoxr it falls back to
+linear resampling (16k→24k), and on one clip that turned "cutter" into "country".
+
+**Checks, 2026-09-26.** On the 4 LibriSpeech clips shipped with audio.cpp, WER
+is 3/69 = 4.35%, identical across 3 reps. Two of the three errors are "I'm" vs
+"I am" normalization. RTF is 0.07–0.14. ⚠ **The first request after a start
+takes ~31 s** (CUDA graph warmup); later ones take 0.3–1 s. VRAM ~2.0 GB.
+
+The model card recommends Q8_0 (3.3 GB; its CUDA WER is 5.80% vs Q4_K's 7.25%,
+on 69 words). To switch, change `path` in `conf/server.json`, download the file,
+and recreate the container.
@@ -0,0 +1,50 @@
+# vibevoice-asr-seat — Microsoft VibeVoice-ASR-Streaming-1.5B on nh3-ml1 (CT 109,
+# RTX 2000E Ada 16 GB), served by audio.cpp. Utility seat for brokkr's dataset
+# foundry (speech → text, speaker-attributed), operator-approved 2026-09-26
+# (relayed by brokkr-smithy-dev).
+#
+#   audiocpp_server  vibevoice-asr-streaming-1.5b (Q4_K)                :8031
+#     POST /v1/audio/transcriptions            (OpenAI-style, whole file)
+#     POST /v1/audio/transcriptions/live?...   (chunked 16 kHz s16le PCM)
+#     GET  /v1/models, /health
+#
+# Runtime is audio.cpp, per the GGUF's own model card (the GGUF is an audio.cpp
+# package with its sidecars embedded, so no separate tokenizer file). It is NOT
+# vanilla llama.cpp and NOT mudler's vibevoice.cpp.
+# Direct access only; no gateway entry.
+#
+# Files (not in git): /opt/aimodels/gguf/vibevoice-asr-streaming-1.5b/ from
+# christopherthompson81/VibeVoice-ASR-Streaming-1.5B-GGUF @ ff9615110299.
+name: vibevoice-asr-seat
+services:
+  audiocpp:
+    build:
+      context: .
+    image: local/audiocpp:v0.8.2-audio8-perf-hotfix
+    container_name: vibevoice-asr
+    restart: unless-stopped
+    command: ["--config", "/config/server.json", "--no-ui", "--log"]
+    ports:
+      - "${ASR_PORT}:8080"
+    volumes:
+      - /opt/aimodels/gguf/vibevoice-asr-streaming-1.5b:/models:ro
+      - /opt/docker/conf/vibevoice-asr-seat/server.json:/config/server.json:ro
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              device_ids: ["0"]
+              capabilities: [gpu]
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 120s
+    labels:
+      - homepage.group=AI - Eval & Retrieval
+      - homepage.name=ASR — VibeVoice Streaming 1.5B (audio.cpp, nh3-ml1)
+      - homepage.icon=mdi-microphone-message
+      - homepage.description=Streaming speech-to-text for the dataset foundry (direct, no gateway)
+      - homepage.href=http://10.100.50.80:${ASR_PORT}/v1/models
@@ -0,0 +1,15 @@
+{
+  "host": "0.0.0.0",
+  "port": 8080,
+  "backend": "cuda",
+  "threads": 6,
+  "models": [
+    {
+      "id": "vibevoice-asr-streaming-1.5b",
+      "family": "vibevoice_asr_streaming",
+      "path": "/models/vibevoice-asr-streaming-1.5b-q4_k.gguf",
+      "task": "asr",
+      "mode": "streaming"
+    }
+  ]
+}