vh
e0d1c44137
The 2026-09-06 headscale cutover retired irv-ml1's wg0 tunnel IP 10.100.79.3 (now 10.6.110.50). Repointed all LIVE canonical refs to the DNS NAME so the next move can't re-break them: homepage.href/siteMonitor labels across 25 stack composes, load-bearing env defaults (asset-engine INFERENCE_HOST, open-webui AUDIO_TTS_OPENAI_API_BASE_URL, skaldsong SKALDSONG_TTS_BASE_URL, zonos-gateway ZONOS_URL, dia), homepage services.yaml manual cards (Voice Design Studio, IRV-ML1), and servers/irv-ml1/ssh-target. Updated the stale 'WG tunnel' comment to the mesh reality. Left as-is: README curl-examples and .env.example comments (docs), and historical mentions in CLAUDE.md/persistent-memory. NOTE: applying the label repoints to the RUNNING irv-ml1 containers needs a recreate per service (labels read at creation); deployed .env values are separate from these canonical defaults.
speaches — OpenAI-compatible ASR (faster-whisper) on irv-ml1
Deployed 2026-08-21 for Eyra (meeting recorder) at the eyra-dev request.
- Host/GPU: irv-ml1, RTX A6000 (
device_ids: ["1"]— see the GPU note below). - Port:
:8204→http://10.100.79.3:8204. WG/LAN-internal, no auth (fleet default, agreed with the consumer for v1). - Image:
ghcr.io/speaches-ai/speachespinned by digest — see "Why the digest pin". - Models:
Systran/faster-whisper-large-v3(batch tier) +Systran/faster-distil-whisper-large-v3(low-latency tier). fp16, ~4.2 GB on disk, ~5.9 GB VRAM with both resident.
curl -X POST http://10.100.79.3:8204/v1/audio/transcriptions \
-F "file=@clip.wav;type=audio/wav" \
-F "model=Systran/faster-distil-whisper-large-v3" \
-F "response_format=verbose_json"
Why this exists next to parakeet (:8765)
Two ASR services on one box is deliberate, not drift. Parakeet is a TDT/transducer model
returning a bare {"text": ...}; it has no no_speech_prob concept at all. Eyra's
hallucination gate keys on per-segment no_speech_prob, so parakeet structurally cannot serve
it. Parakeet remains the right pick for plain text-out transcription.
The load-bearing requirement
response_format=verbose_json must return segments[].no_speech_prob intact. Verified
2026-08-21 on both tiers — every segment carries no_speech_prob, avg_logprob,
compression_ratio, temperature, tokens, and word timings.
⚠ Measured: no_speech_prob alone is a WEAK hallucination gate
Verified against synthetic speech / silence / pink-noise room tone, VAD off, distil tier:
| input | no_speech_prob |
avg_logprob |
compression_ratio |
text returned |
|---|---|---|---|---|
| speech (6.6 s) | 0.018 | -0.114 | 1.141 | correct verbatim |
| pure silence | 0.107 | -0.650 | 0.556 | "Thank you." ← hallucinated |
| pink room tone | 0.077 | -0.724 | 0.556 | "Thank you." ← hallucinated |
Both non-speech inputs produced the classic Whisper "Thank you." hallucination, and
no_speech_prob stayed under 0.11 on both — a conventional > 0.6 threshold would pass
them straight through. avg_logprob separates the same cases ~6× more decisively
(< -0.5 catches both), and compression_ratio splits cleanly at 0.556 vs 1.141.
Recommendation to any consumer: gate on a composite, not no_speech_prob alone. The field
is present and directionally correct, but its dynamic range on this stack is too compressed to
carry a threshold by itself.
Why VAD is pinned OFF
_UNSTABLE_VAD_FILTER=False, set explicitly in .env, at the consumer's request. Eyra VAD-gates
upstream on the capture edge and sends only speech segments; a second VAD here would re-chunk the
audio and therefore change what no_speech_prob means per segment, underneath a gate calibrated
against this stack.
Consequence worth stating plainly: with VAD off, this service will happily transcribe silence into text (see the table above). Upstream VAD gating is what prevents that — the service is not defending itself.
Why the digest pin
_UNSTABLE_VAD_FILTER carries a leading underscore and the literal word "unstable" — upstream
reserves the right to rename it. On a floating :latest-cuda, a routine image bump could silently
drop that variable, restore VAD, and move no_speech_prob semantics under a calibrated consumer
gate with no error and no log line. Bumping the pin is a deliberate act that requires re-running
the fidelity + discrimination checks above.
Gotcha: PRELOAD_MODELS does not download
PRELOAD_MODELS only loads models already in the HF cache — it will not fetch them. A first
boot with an empty cache starts healthy, serves /health 200, and exposes an empty
/v1/models. Download explicitly (URL-encode the / in the repo id):
curl -X POST "http://localhost:8204/v1/models/Systran%2Ffaster-whisper-large-v3"
curl -X POST "http://localhost:8204/v1/models/Systran%2Ffaster-distil-whisper-large-v3"
Related: the bind-mounted cache dir must contain a hub/ subdirectory or every
/v1/models call 500s with huggingface_hub.errors.CacheNotFound while /health still returns
200 — a healthy container serving a broken registry. mkdir -p <cache>/hub owned by uid 1000.
GPU note — the docker/native index inversion
device_ids: ["1"] is the A6000, verified empirically (the container reports
NVIDIA RTX A6000). Do not "fix" this to 0 by reading nvidia-smi from a native shell:
native CUDA on irv-ml1 enumerates the A6000 as cuda:0 while the docker view has it at 1.
0 in compose is the RTX 3090, which already hosts parakeet and the TTS stacks.
Residency
STT_MODEL_TTL=-1 keeps both tiers resident (upstream default is 300 s). Deliberate: a
mid-meeting reload would be a multi-second stall on a latency-sensitive draft-caption tier, and
~5.9 GB against 20 GB still free is cheap. Revisit when Eyra's diarization workload lands on
this box — that ask is expected at their diarize milestone and will want GPU embeddings plus
gated pyannote weights. Switching to 300 trades the latency back for VRAM in one .env line.
Deploy
scripts/deploy-stack.sh irv-ml1 speaches # diffs vs live, prompts y/N
# on host: cp .env.example .env; docker compose up -d