vh
131d746c92
Three fixes from the second-wave deploy attempts:
* voxtral: vllm/vllm-omni doesn't publish a `latest` tag — pull
failed with "manifest unknown". Pinned VOXTRAL_VLLM_TAG to v0.18.0
(released 2026-03-29, the day after the Voxtral 4B TTS release —
first cut with Voxtral support).
* kyutai-tts: NillPointer wrapper exposes ONLY /health (root) and
POST /v1/audio/speech. No /v1/models, no /v1/audio/voices —
those return 404. Verified by /openapi.json against the live
container. Compose healthcheck + playbook wait + verify steps
all repointed at the actual paths. POST /v1/audio/speech is now
smoke-tested with a RIFF WAV assertion (same pattern as fish-s2).
* fish-s2: added FISH_S2_MODEL env var so the model variant is
swappable via .env without rebuilding. Both s2-pro (default) and
s1-mini are pre-pulled into the bind-mount; LLAMA_CHECKPOINT_PATH
+ DECODER_CHECKPOINT_PATH now use ${FISH_S2_MODEL:-s2-pro}.
s1-mini was originally gated on fishaudio's HF org (401), but
niobures/OpenAudio-S1 mirrors the same files openly — pulled
from there via a one-shot snapshot_download.
Kyutai TTS
kyutai/tts-1.6b-en_fr — Kyutai's flagship streaming TTS (1.6B params, EN/FR bilingual, trained on 2.5M hours), served via the NillPointer/Kyutai-TTS-Server community OpenAI-compatible wrapper.
Why this stack exists
Kyutai's claim is the lowest streaming latency in this size class: 220 ms in solo setup; up to 32 simultaneous streams under 350 ms on a single L40-class GPU. Worth bench-comparing against:
| claimed latency | use case | |
|---|---|---|
| Kyutai TTS | ~220 ms | streaming EN/FR, conversational dialogue heritage |
| Kokoro | ~26 ms TTFB measured | low-latency English, fixed voice library |
| Chatterbox Turbo | ~1.2 s TTFB measured | English w/ cloning + 9 paralinguistic tags |
| Fish Audio S2-Pro | ~150 ms claimed | richest paralinguistic English |
| Voxtral | ~70 ms model latency | multilingual EN/FR/DE/ES/IT/PT/NL/HI |
Deployment notes
Kyutai's official deployment path is Rust + websockets only (no
HTTP, no OpenAI-compat). That doesn't fit the OpenAI-/v1/audio/speech
contract the rest of our TTS fleet uses. The
NillPointer/Kyutai-TTS-Server community wrapper bridges Kyutai's
native streaming to the OpenAI HTTP shape, which lets us slot it
into the same bench harness as the others.
Tradeoff: the wrapper adds Python overhead on the request path, so measured latency on this stack will be higher than Kyutai's 220 ms claim (which is for the bare Rust server). If we measure ~400-500 ms TTFB end-to-end, the wrapper is the floor — Kyutai itself is hitting its target.
Architecture heritage
Kyutai's TTS shares the Mimi neural codec + Moshi dialogue modeling framework. Both target full-duplex conversational AI (Moshi is their flagship speech-text foundation model). The TTS-only model is the "synthesis half" of the stack, distilled for low-latency streaming.
Trained on 2.5M hours — a different scaling regime from the others (CosyVoice 5k, Fish 10M, Voxtral undisclosed).
API
OpenAI-compat at http://10.100.79.3:8198:
# Single-shot synthesis.
curl -fsS -X POST http://10.100.79.3:8198/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{"model":"tts-1.6b-en_fr","input":"Hello there.","voice":"default","response_format":"wav"}' \
> out.wav
# Streaming.
curl -fsS -X POST http://10.100.79.3:8198/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{"model":"tts-1.6b-en_fr","input":"long passage…","voice":"default","stream":true}' \
| mpv --no-cache -
# Built-in voices.
curl http://10.100.79.3:8198/v1/audio/voices
Deploy
scripts/elway irv-ml1 --playbook playbooks/deploy-kyutai-tts.yaml
First boot pulls the kyutai/tts-1.6b-en_fr checkpoint (~3-6 GB) into the HF cache.
Hardware footprint
- VRAM: ~4-6 GB practical. Pinned to GPU 0 (RTX 3090) by default — the A6000 is hosting the heavier Fish S2-Pro / Voxtral.
- Disk: ~6 GB for the checkpoint + HF cache.