stacks/{fish-s2,voxtral,kyutai-tts}: three new TTS deploys for irv-ml1 quality A/B
Adds the three premier 2026 TTS releases we missed during the original
fleet build-out (early April), all licensed for self-host:
* Fish Audio S2-Pro (port 8195, GPU 1 / A6000) — released 2026-03-09.
4B dual-AR (Slow + Fast) trained on 10M+ hours / 80+ languages.
Headline: 15,000+ paralinguistic / emotion tags via natural language
([laugh] [whispers] [super happy] etc.) — a step-function over
Chatterbox Turbo's 9 fixed tags. 91.61% paralinguistic win rate on
EmergentTTS-Eval. ~150 ms streaming TTFB, voice cloning, MIT-style
open. ~17 GB VRAM.
* Voxtral TTS (port 8197, GPU 1 / A6000) — Mistral, released 2026-03-28.
4B open-weight, 70 ms model latency, 9.7× realtime. 68.4% blind A/B
win rate vs ElevenLabs Flash v2.5 in cloning. 8 languages
(EN/FR/DE/ES/IT/PT/NL/HI). Served via vLLM-Omni (Mistral's partner
serving stack) — published Docker image, no local build. ~16 GB VRAM.
CC BY-NC license — personal/research use only; flagged in README.
* Kyutai TTS (port 8198, GPU 0 / 3090) — kyutai/tts-1.6b-en_fr.
Trained on 2.5M hours from the Moshi/Mimi team. Claimed 220 ms in
solo setup, 32 simultaneous streams under 350 ms on L40. Kyutai's
official deploy is Rust + websockets only; using NillPointer's
community OpenAI-compat wrapper to bridge to /v1/audio/speech so
it slots into the same bench harness. ~4-6 GB VRAM.
Each stack: compose.yaml (build context, env, volumes, healthcheck,
homepage label), .env.example (all tunables documented), README.md
(why it exists, headline numbers, API, deploy + hardware notes).
Playbooks at playbooks/deploy-{fish-s2,voxtral,kyutai-tts}.yaml are
idempotent in the same shape as the existing deploy-vibevoice /
deploy-chatterbox playbooks.
Port allocations on irv-ml1 after this lands: 8188 ComfyUI, 8190
CosyVoice, 8191 Qwen3-TTS, 8192 IndexTTS-2, 8193 Kokoro, 8194
VibeVoice, 8195 Fish, 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai,
8765 Parakeet ASR.
This commit is contained in:
@@ -0,0 +1,59 @@
|
||||
# Voxtral TTS — Mistral AI's 4B open-weight streaming TTS, served via
|
||||
# vLLM-Omni (the production serving stack Mistral co-developed for
|
||||
# Voxtral). Released March 28, 2026.
|
||||
#
|
||||
# Why this stack alongside the existing TTS:
|
||||
# * 70 ms model latency, 9.7× realtime — fastest non-Kokoro option.
|
||||
# * Multilingual-first (EN strong, plus FR, DE, ES, IT, PT, NL, HI).
|
||||
# Different from CosyVoice's Chinese-leaning balance.
|
||||
# * 68.4% blind A/B win rate vs ElevenLabs Flash v2.5 in cloning.
|
||||
# * vLLM-Omni serving = continuous batching + paged attention — the
|
||||
# same mechanism that gave qwen3.6 its speed on llama-swap.
|
||||
#
|
||||
# LICENSE: CC BY-NC. Personal / research use only. Don't ship Voxtral
|
||||
# output in any commercial product without re-licensing from Mistral.
|
||||
#
|
||||
# All tunables live in .env — edit that, not this file.
|
||||
|
||||
services:
|
||||
voxtral:
|
||||
# vLLM-Omni image — Mistral's official partnership for Voxtral
|
||||
# serving. Version-pinned via .env.
|
||||
image: vllm/vllm-omni:${VOXTRAL_VLLM_TAG}
|
||||
container_name: voxtral
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${VOXTRAL_BIND:-0.0.0.0}:${VOXTRAL_PORT}:8000"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${VOXTRAL_GPU_DEVICES:-1}
|
||||
- HF_HOME=/root/.cache/huggingface
|
||||
# vLLM serving args — see https://docs.vllm.ai for full list.
|
||||
# We override the default model via cmd args below.
|
||||
volumes:
|
||||
- ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
|
||||
- ${VOXTRAL_VOICES_DIR}:/voices:ro
|
||||
# vLLM-Omni's serve command — model + dtype + port pinned.
|
||||
command:
|
||||
- --model=${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}
|
||||
- --port=8000
|
||||
- --dtype=bfloat16
|
||||
- --gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}
|
||||
healthcheck:
|
||||
# vLLM-Omni exposes /health for liveness + /v1/models for readiness.
|
||||
# /health 200 means the server's listening; /v1/models 200 means
|
||||
# the model is loaded and request-ready. Use readiness as the
|
||||
# healthy signal so we don't mark it "healthy" before it can
|
||||
# accept synthesis requests.
|
||||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# First boot pulls Voxtral-4B (~8 GB BF16) + warms vLLM. Generous.
|
||||
start_period: 600s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=Voxtral TTS
|
||||
- homepage.icon=mdi-translate
|
||||
- homepage.description=Mistral 4B multilingual streaming TTS — 70 ms latency, voice cloning (irv-ml1)
|
||||
- homepage.href=http://10.100.79.3:${VOXTRAL_PORT}
|
||||
Reference in New Issue
Block a user