vh
aa5863c9a3
Deployed for Eyra (meeting recorder) per the eyra-dev request. Serves
large-v3 (batch tier) + distil-large-v3 (low-latency tier) on :8204,
fp16, both resident, ~5.9 GB VRAM against 20 GB still free.
Sits alongside the existing parakeet stack (:8765) deliberately: parakeet
is a TDT/transducer returning bare {"text": ...} and has no no_speech_prob
concept, so it structurally cannot serve this consumer.
The load-bearing requirement -- segments[].no_speech_prob surviving
response_format=verbose_json -- is VERIFIED on both tiers.
Measured finding worth more than the deployment: no_speech_prob alone is a
WEAK hallucination gate on this stack. Pure silence and pink room tone both
produced the classic Whisper 'Thank you.' hallucination while no_speech_prob
stayed under 0.11 -- a conventional >0.6 threshold passes both through.
avg_logprob separates the same cases ~6x more decisively (-0.11 speech vs
-0.65/-0.72 non-speech) and compression_ratio splits 1.141 vs 0.556.
Consumers should gate on a composite, not no_speech_prob alone. Table in
the README.
VAD pinned OFF at the consumer's request (they VAD-gate upstream on the
capture edge). Consequence stated plainly in the README: with VAD off this
service will transcribe silence into text and is not defending itself.
Image pinned BY DIGEST rather than :latest-cuda, because the VAD-off
setting rides on _UNSTABLE_VAD_FILTER -- a variable upstream explicitly
marks unstable. A floating tag could rename it on any bump, silently
restoring VAD and moving no_speech_prob semantics under a calibrated gate
with no error and no log line.
Two deployment gotchas recorded: PRELOAD_MODELS only loads models already
cached (it does not download -- use POST /v1/models/{id}), and the bind-
mounted cache needs a hub/ subdir or every /v1/models call 500s with
CacheNotFound while /health still returns 200.
44 lines
3.1 KiB
Bash
44 lines
3.1 KiB
Bash
# speaches — irv-ml1. Copy to .env on the host and adjust.
|
|
|
|
# ── Image ───────────────────────────────────────────────────────────────────
|
|
# PINNED BY DIGEST, deliberately. The VAD-off setting below rides on an
|
|
# upstream variable explicitly marked unstable (`_UNSTABLE_VAD_FILTER`); an
|
|
# unpinned `:latest-cuda` could rename it on any bump, silently restoring VAD
|
|
# and shifting `no_speech_prob` semantics under Eyra's calibrated gate.
|
|
# Bumping this tag is a deliberate act that REQUIRES re-running the
|
|
# verbose_json fidelity check afterwards. Digest captured 2026-08-21.
|
|
SPEACHES_IMAGE=ghcr.io/speaches-ai/speaches@sha256:6ec12ebf890a17e0d4b242a8ba9e0eb1fb836e60e8a3c857aea9838d541579ac
|
|
|
|
# ── Network ─────────────────────────────────────────────────────────────────
|
|
SPEACHES_PORT=8204
|
|
SPEACHES_BIND=0.0.0.0
|
|
|
|
# ── GPU ─────────────────────────────────────────────────────────────────────
|
|
# 1 = RTX A6000 (26 GB free). 0 = RTX 3090, already hosting parakeet + TTS.
|
|
SPEACHES_GPU_ID=1
|
|
|
|
# ── Models ──────────────────────────────────────────────────────────────────
|
|
# Both tiers preloaded at startup so the first call of a meeting is not a
|
|
# cold download. Repo IDs verified against the HF API 2026-08-21.
|
|
# large-v3 (~3.1 GB fp16) — batch/accuracy tier, post-meeting job
|
|
# distil-large-v3 (~1.5 GB fp16) — low-latency draft captioning
|
|
# ⚠ distil-large-v3 is ENGLISH-ONLY. Eyra is English-only for v1 by design.
|
|
# When multilingual unparks, `Systran/faster-whisper-large-v3-turbo` is the
|
|
# multilingual replacement for the fast tier — swapping it is a .env edit.
|
|
SPEACHES_PRELOAD_MODELS=["Systran/faster-whisper-large-v3","Systran/faster-distil-whisper-large-v3"]
|
|
SPEACHES_CACHE_DIR=/worktank/speaches/cache
|
|
|
|
# ── Inference ───────────────────────────────────────────────────────────────
|
|
# VAD OFF — consumer requirement, do not flip without telling eyra-dev.
|
|
SPEACHES_VAD_FILTER=False
|
|
# Ampere sm_86: fp16 is correct here; no FP8/NVFP4 on this hardware.
|
|
SPEACHES_COMPUTE_TYPE=float16
|
|
SPEACHES_NUM_WORKERS=1
|
|
# -1 = never unload (deterministic latency). 300 = upstream default, trades
|
|
# latency for VRAM — switch if the A6000 gets tight.
|
|
SPEACHES_STT_MODEL_TTL=-1
|
|
|
|
# ── Misc ────────────────────────────────────────────────────────────────────
|
|
SPEACHES_ENABLE_UI=true
|
|
SPEACHES_LOG_LEVEL=info
|