vh
131d746c92
Three fixes from the second-wave deploy attempts:
* voxtral: vllm/vllm-omni doesn't publish a `latest` tag — pull
failed with "manifest unknown". Pinned VOXTRAL_VLLM_TAG to v0.18.0
(released 2026-03-29, the day after the Voxtral 4B TTS release —
first cut with Voxtral support).
* kyutai-tts: NillPointer wrapper exposes ONLY /health (root) and
POST /v1/audio/speech. No /v1/models, no /v1/audio/voices —
those return 404. Verified by /openapi.json against the live
container. Compose healthcheck + playbook wait + verify steps
all repointed at the actual paths. POST /v1/audio/speech is now
smoke-tested with a RIFF WAV assertion (same pattern as fish-s2).
* fish-s2: added FISH_S2_MODEL env var so the model variant is
swappable via .env without rebuilding. Both s2-pro (default) and
s1-mini are pre-pulled into the bind-mount; LLAMA_CHECKPOINT_PATH
+ DECODER_CHECKPOINT_PATH now use ${FISH_S2_MODEL:-s2-pro}.
s1-mini was originally gated on fishaudio's HF org (401), but
niobures/OpenAudio-S1 mirrors the same files openly — pulled
from there via a one-shot snapshot_download.
59 lines
2.6 KiB
YAML
59 lines
2.6 KiB
YAML
# Kyutai TTS — 1.6B / 2B-class streaming TTS from Kyutai (the Moshi /
|
|
# Mimi team), trained on 2.5M hours. 220 ms latency in solo setup; up
|
|
# to 32 simultaneous streams under 350 ms on an L40-class GPU.
|
|
#
|
|
# Served via the NillPointer/Kyutai-TTS-Server community wrapper —
|
|
# Kyutai's official deployment is Rust + websockets only, which doesn't
|
|
# fit our OpenAI-compat fleet. The community wrapper bridges Kyutai's
|
|
# native streaming to the OpenAI /v1/audio/speech contract.
|
|
#
|
|
# Why this stack alongside the existing TTS:
|
|
# * Kyutai's claim is the lowest streaming latency in this size
|
|
# class (220 ms on a single GPU). Worth bench-comparing against
|
|
# Chatterbox (~1.2 s) and Fish S2-Pro (~150 ms claimed).
|
|
# * Trained on 2.5M hours — a different scaling regime from the
|
|
# others (CosyVoice 5k hrs, Fish 10M hrs).
|
|
# * Designed for full-duplex dialogue (Moshi heritage) — may surface
|
|
# conversational quality the others lack.
|
|
#
|
|
# All tunables live in .env — edit that, not this file.
|
|
|
|
services:
|
|
kyutai-tts:
|
|
image: local/kyutai-tts:${KYUTAI_TTS_TAG}
|
|
build:
|
|
context: https://github.com/NillPointer/Kyutai-TTS-Server.git#${KYUTAI_TTS_SHA}
|
|
dockerfile: Dockerfile
|
|
container_name: kyutai-tts
|
|
restart: unless-stopped
|
|
runtime: nvidia
|
|
ports:
|
|
- "${KYUTAI_TTS_BIND:-0.0.0.0}:${KYUTAI_TTS_PORT}:8000"
|
|
environment:
|
|
- NVIDIA_VISIBLE_DEVICES=${KYUTAI_TTS_GPU_DEVICES:-0}
|
|
# Kyutai's en/fr bilingual model on HF. Switch to a different
|
|
# checkpoint via .env without rebuilding.
|
|
- KYUTAI_MODEL=${KYUTAI_TTS_MODEL:-kyutai/tts-1.6b-en_fr}
|
|
- HF_HOME=/app/hf_cache
|
|
volumes:
|
|
- ${KYUTAI_TTS_CACHE_DIR}:/app/hf_cache
|
|
- ${KYUTAI_TTS_VOICES_DIR}:/app/voices:ro
|
|
healthcheck:
|
|
# NillPointer wrapper's actual endpoints (verified at runtime via
|
|
# /openapi.json): just /health (root) and /v1/audio/speech (POST).
|
|
# No /v1/models, no /v1/audio/voices — those 404. /health is the
|
|
# liveness signal. 127.0.0.1 explicit to dodge IPv4/IPv6 localhost
|
|
# race.
|
|
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5).status==200 else 1)\""]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
# First boot pulls the Kyutai checkpoint (~3-6 GB) + warms.
|
|
start_period: 600s
|
|
labels:
|
|
- homepage.group=AI Systems
|
|
- homepage.name=Kyutai TTS
|
|
- homepage.icon=mdi-radio-tower
|
|
- homepage.description=Ultra-low-latency streaming TTS — 220 ms on solo GPU, EN/FR (irv-ml1)
|
|
- homepage.href=http://10.100.79.3:${KYUTAI_TTS_PORT}
|