stacks/{fish-s2,voxtral,kyutai-tts}: three new TTS deploys for irv-ml1 quality A/B

Adds the three premier 2026 TTS releases we missed during the original fleet build-out (early April), all licensed for self-host: * Fish Audio S2-Pro (port 8195, GPU 1 / A6000) — released 2026-03-09. 4B dual-AR (Slow + Fast) trained on 10M+ hours / 80+ languages. Headline: 15,000+ paralinguistic / emotion tags via natural language ([laugh] [whispers] [super happy] etc.) — a step-function over Chatterbox Turbo's 9 fixed tags. 91.61% paralinguistic win rate on EmergentTTS-Eval. ~150 ms streaming TTFB, voice cloning, MIT-style open. ~17 GB VRAM. * Voxtral TTS (port 8197, GPU 1 / A6000) — Mistral, released 2026-03-28. 4B open-weight, 70 ms model latency, 9.7× realtime. 68.4% blind A/B win rate vs ElevenLabs Flash v2.5 in cloning. 8 languages (EN/FR/DE/ES/IT/PT/NL/HI). Served via vLLM-Omni (Mistral's partner serving stack) — published Docker image, no local build. ~16 GB VRAM. CC BY-NC license — personal/research use only; flagged in README. * Kyutai TTS (port 8198, GPU 0 / 3090) — kyutai/tts-1.6b-en_fr. Trained on 2.5M hours from the Moshi/Mimi team. Claimed 220 ms in solo setup, 32 simultaneous streams under 350 ms on L40. Kyutai's official deploy is Rust + websockets only; using NillPointer's community OpenAI-compat wrapper to bridge to /v1/audio/speech so it slots into the same bench harness. ~4-6 GB VRAM. Each stack: compose.yaml (build context, env, volumes, healthcheck, homepage label), .env.example (all tunables documented), README.md (why it exists, headline numbers, API, deploy + hardware notes). Playbooks at playbooks/deploy-{fish-s2,voxtral,kyutai-tts}.yaml are idempotent in the same shape as the existing deploy-vibevoice / deploy-chatterbox playbooks. Port allocations on irv-ml1 after this lands: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS, 8192 IndexTTS-2, 8193 Kokoro, 8194 VibeVoice, 8195 Fish, 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai, 8765 Parakeet ASR.
2026-04-27 22:40:10 -07:00
parent db42a7cc17
commit 16d018ff96
12 changed files with 872 additions and 0 deletions
@@ -0,0 +1,38 @@
+# Voxtral TTS stack tunables. Copy to `.env` on irv-ml1 before
+# deploying.
+
+# ── image pin ────────────────────────────────────────────────────────
+# vLLM-Omni image tag (Mistral's partner serving stack for Voxtral).
+# Use a specific version rather than `latest` — vLLM moves fast and
+# Voxtral has version-specific compatibility.
+VOXTRAL_VLLM_TAG=latest
+
+# Voxtral model on Hugging Face. The 4B variant is the only released
+# checkpoint as of 2026-04. Default BF16 weights are ~8 GB.
+VOXTRAL_MODEL=mistralai/Voxtral-4B-TTS-2603
+
+# ── network ──────────────────────────────────────────────────────────
+# Host port (container listens on 8000 internally).
+VOXTRAL_PORT=8197
+VOXTRAL_BIND=0.0.0.0
+
+# ── runtime / GPU ────────────────────────────────────────────────────
+# GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB).
+# Voxtral 4B BF16 needs ~16 GB practical (model + KV + activation).
+# Pinned to A6000 by default for headroom. The 3090 fits but is tight
+# for long streaming sessions.
+VOXTRAL_GPU_DEVICES=1
+
+# vLLM GPU memory utilization fraction (0.0-1.0). 0.85 = leave 15%
+# headroom for other processes / KV cache spikes. Lower if running
+# alongside other GPU workloads on the same device.
+VOXTRAL_GPU_UTIL=0.85
+
+# ── persistent storage on the host ───────────────────────────────────
+# HF cache — first start pulls the Voxtral checkpoint (~8 GB) into
+# this dir. Persistent across container recreates.
+VOXTRAL_CACHE_DIR=/worktank/voxtral/hf_cache
+
+# Reference voices for cloning. Read-only mount inside the container.
+# Drop ~5-15 s WAV / FLAC clips here.
+VOXTRAL_VOICES_DIR=/worktank/voxtral/voices
@@ -0,0 +1,85 @@
+# Voxtral TTS
+
+[mistralai/Voxtral-4B-TTS-2603](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603)
+— Mistral AI's 4B open-weight streaming TTS, served via the
+[vLLM-Omni](https://github.com/vllm-project/vllm-omni) production
+serving stack (Mistral co-developed). Released March 28, 2026.
+
+## ⚠️ License
+
+**CC BY-NC.** Personal use, research, and internal tooling are fine.
+**Don't ship Voxtral output in any commercial product** without
+re-licensing from Mistral. The other TTS in this fleet (Kokoro,
+Chatterbox, Fish S2-Pro, IndexTTS-2, Qwen3-TTS, CosyVoice) are all
+open-licensed and clean for commercial work.
+
+## Why this stack exists
+
+Multilingual streaming with serious speed:
+
+| | use case |
+|---|---|
+| **Voxtral** | multilingual EN/FR/DE/ES/IT/PT/NL/HI streaming, 70 ms model latency |
+| Kokoro | low-latency English, fixed voice library |
+| Chatterbox Turbo | low-latency English w/ cloning + 9 paralinguistic tags |
+| Fish Audio S2-Pro | richest paralinguistic English (15k+ tags) |
+| IndexTTS-2 | English voice cloning + emotion vector / text control |
+| Qwen3-TTS-1.7B | English voice cloning (slow on official backend) |
+| CosyVoice 3 | multilingual (Chinese-leaning) |
+| VibeVoice 1.5B | long-form / multi-speaker dialogue |
+
+Voxtral fills the **multilingual + low-latency + cloning** slot
+that's been weak in the fleet (CosyVoice is multilingual but slow on
+English; nothing else is multilingual at all).
+
+## Headline numbers
+
+- **70 ms** model latency for a typical 10 s sample (500-char input)
+- **9.7×** realtime factor
+- **68.4%** blind A/B win rate vs ElevenLabs Flash v2.5 in voice
+  cloning evaluations
+- **8 languages**: EN, FR, DE, ES, IT, PT, NL, HI
+
+## API
+
+vLLM-Omni serves an OpenAI-compatible API at
+`http://10.100.79.3:8197/v1`:
+
+```bash
+# Single-shot synthesis.
+curl -fsS -X POST http://10.100.79.3:8197/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"mistralai/Voxtral-4B-TTS-2603","input":"Hello there.","voice":"alloy","response_format":"wav"}' \
+  > out.wav
+
+# Streaming.
+curl -fsS -X POST http://10.100.79.3:8197/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"mistralai/Voxtral-4B-TTS-2603","input":"long passage…","voice":"alloy","stream":true}' \
+  | mpv --no-cache -
+
+# vLLM-Omni standard endpoints.
+curl http://10.100.79.3:8197/v1/models    # confirms model loaded
+curl http://10.100.79.3:8197/v1/audio/voices  # built-in + cloned voices
+```
+
+## Deploy
+
+```bash
+scripts/elway irv-ml1 --playbook playbooks/deploy-voxtral.yaml
+```
+
+First boot pulls Voxtral-4B (~8 GB BF16) into the HF cache + warms
+vLLM. Both are cached afterwards.
+
+## Hardware footprint
+
+- **VRAM**: ~16 GB practical (8 GB weights + KV + activation). Pinned
+  to GPU 1 (RTX A6000) by default — comfortable headroom. The 3090's
+  24 GB CAN fit but it's tight for long streaming sessions.
+- **Disk**: ~8 GB for the Voxtral checkpoint + HF cache.
+
+## Voice library
+
+Drop reference WAV / FLAC into `/worktank/voxtral/voices/` on the
+host. The wrapper scans on request — no restart needed.
@@ -0,0 +1,59 @@
+# Voxtral TTS — Mistral AI's 4B open-weight streaming TTS, served via
+# vLLM-Omni (the production serving stack Mistral co-developed for
+# Voxtral). Released March 28, 2026.
+#
+# Why this stack alongside the existing TTS:
+#   * 70 ms model latency, 9.7× realtime — fastest non-Kokoro option.
+#   * Multilingual-first (EN strong, plus FR, DE, ES, IT, PT, NL, HI).
+#     Different from CosyVoice's Chinese-leaning balance.
+#   * 68.4% blind A/B win rate vs ElevenLabs Flash v2.5 in cloning.
+#   * vLLM-Omni serving = continuous batching + paged attention — the
+#     same mechanism that gave qwen3.6 its speed on llama-swap.
+#
+# LICENSE: CC BY-NC. Personal / research use only. Don't ship Voxtral
+# output in any commercial product without re-licensing from Mistral.
+#
+# All tunables live in .env — edit that, not this file.
+
+services:
+  voxtral:
+    # vLLM-Omni image — Mistral's official partnership for Voxtral
+    # serving. Version-pinned via .env.
+    image: vllm/vllm-omni:${VOXTRAL_VLLM_TAG}
+    container_name: voxtral
+    restart: unless-stopped
+    runtime: nvidia
+    ports:
+      - "${VOXTRAL_BIND:-0.0.0.0}:${VOXTRAL_PORT}:8000"
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=${VOXTRAL_GPU_DEVICES:-1}
+      - HF_HOME=/root/.cache/huggingface
+      # vLLM serving args — see https://docs.vllm.ai for full list.
+      # We override the default model via cmd args below.
+    volumes:
+      - ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
+      - ${VOXTRAL_VOICES_DIR}:/voices:ro
+    # vLLM-Omni's serve command — model + dtype + port pinned.
+    command:
+      - --model=${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}
+      - --port=8000
+      - --dtype=bfloat16
+      - --gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}
+    healthcheck:
+      # vLLM-Omni exposes /health for liveness + /v1/models for readiness.
+      # /health 200 means the server's listening; /v1/models 200 means
+      # the model is loaded and request-ready. Use readiness as the
+      # healthy signal so we don't mark it "healthy" before it can
+      # accept synthesis requests.
+      test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      # First boot pulls Voxtral-4B (~8 GB BF16) + warms vLLM. Generous.
+      start_period: 600s
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=Voxtral TTS
+      - homepage.icon=mdi-translate
+      - homepage.description=Mistral 4B multilingual streaming TTS — 70 ms latency, voice cloning (irv-ml1)
+      - homepage.href=http://10.100.79.3:${VOXTRAL_PORT}