- chatterbox-fast experimental -> ready: browser audition verified end-to-end
(operator confirmed progressive playback "excellent" 2026-06-02).
- vibevoice ready -> down: no container running on irv-ml1 (connection refused);
catalog status was stale.
- voxtral: NOT a stale typo — its stack genuinely claimed :8197, the port now held
by the live chatterbox-fast. voxtral is down, so moved IT to :8201 (catalog
endpoint + source_url, stacks/voxtral/.env.example + README, host .env) rather
than disturb the live service. No live clash existed (voxtral down) but it was a
latent deploy-time collision I introduced by placing chatterbox-fast on 8197.
No catalog_version bump (status changes + endpoint correction, additive). Validates
against the schema.
Voxtral final fix (8th iteration):
* The bundled voxtral_tts.yaml hardcodes gpu_memory_utilization: 0.8
on the language_model stage — overrides the CLI flag. Mounted a
patched copy (0.4) at /etc/voxtral/voxtral_tts.yaml and pointed
--stage-configs-path there.
* With Kyutai stopped to free 5 GB on the 3090, both stages fit
(target 9.4 + 2.4 GB ≈ 11.8 GB; 17 GB free post-kyutai-stop).
* Voxtral now healthy on GPU 0 — bench: 1.9-2.7 s TTFB, real WAV.
Fish s2-pro optimization (per-request sweep, no model swap):
* `streaming: true` in request body drops TTFB from 7.7 s → 0.026 s
(300×). Total time goes up ~1 s (chunked HTTP overhead) but
perceived latency = TTFB. Use stream:true for any interactive use.
* `latency: "balanced"` actually slower than default — bad name; skip.
* `use_memory_cache: "on"` no measurable benefit.
* `chunk_length: 100` (default 200) no TTFB benefit non-streaming.
* Server-side `--half` (fp16 inference) added via compose `command`
override — passes through start_server.sh's $@ unchanged into
api_server.py. Should reduce total time too. Validation pending
the post-restart bench.
Kyutai stopped to free GPU 0 budget — the bench numbers earlier
(3.4 s avg) were unimpressive vs Voxtral's 2.3 s in the same
multilingual slot. Kept the stack files for future re-deploy if
needed; just the running container is gone.
Fourth attempt finally found the right invocation. Voxtral is a
two-stage TTS pipeline (language_model → acoustic_transformer →
audio output), not a flat MistralForCausalLM. Standard `vllm serve`
errored with "no module named 'acoustic_transformer'" because it
loads the model as a vanilla Mistral causal LM.
Pattern from /workspace/vllm-omni/examples/online_serving/
qwen3_tts/run_server.sh (closest in-image analog):
vllm-omni serve <MODEL> \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--host 0.0.0.0 --port 8000 \
--gpu-memory-utilization 0.45 \
--trust-remote-code --omni
Key differences from previous attempt:
* `vllm-omni` binary, not `vllm`
* `--omni` flag activates multi-stage pipeline
* `--stage-configs-path` points at the bundled YAML that maps
stages to GPU + scheduler + worker classes
* Dropped --load-format/--tokenizer-mode/--config-format=mistral
flags — the stage config handles tokenizer_mode internally
* --trust-remote-code is required for the acoustic_transformer
custom code path
Default .env.example now: GPU 0 (3090) with util 0.45 (~10.6 GB
target on 24 GB GPU). The A6000 is fully booked by Fish s2-pro.
Third voxtral attempt: image pulled clean (3 min, v0.18.0), entrypoint
parsed correctly, vLLM started, but engine init failed two ways:
1. HF rate-limited the irv-ml1 IP (38.120.94.3) during the metadata
fetch — 429 Too Many Requests from too many large unauthenticated
pulls today (heretic, 27b, fish-s2, fish-s1-mini, voxtral). Added
HF_TOKEN env passthrough; user generates a token at
https://huggingface.co/settings/tokens and sets VOXTRAL_HF_TOKEN
in .env.
2. Voxtral uses Mistral's native model format (params.json +
tekken.json tokenizer + consolidated.safetensors single file),
NOT HF transformers format (config.json + tokenizer.json + sharded
.safetensors). vLLM errored with "ensure presence of params.json
for Mistral models." Fix: pass --load-format=mistral
--tokenizer-mode=mistral --config-format=mistral to vllm serve.
Confirmed by inspecting the Voxtral-4B-TTS-2603 HF tree:
25 files, ships params.json + tekken.json + consolidated.safetensors.
Both fixes baked into compose. User needs to drop their HF_TOKEN into
.env once and recreate.
Side note discovered while debugging: fish-s2 s1-mini variant uses
the tiktoken tokenizer format; the wrapper can't load it (errors with
"NoneType has no attribute encode" on warmup). So s1-mini isn't a
drop-in optimization for s2-pro — different code path needed. Fish
back on s2-pro for now.
Second voxtral attempt got past the image pull (v0.18.0 published,
~3 min download) but container init failed:
unable to start container process: error during container init:
exec: "--model=mistralai/Voxtral-4B-TTS-2603": stat ...: no such file
vllm/vllm-omni:v0.18.0 has Entrypoint=null AND Cmd=null — there's no
default executable. The compose's `command:` array becomes the full
exec invocation, with --model=... interpreted as the binary name.
Standard vLLM serving CLI is `vllm serve <model> [flags]`. The
binary's at /usr/local/bin/vllm. Set entrypoint: ["vllm", "serve"]
and pass the model as a positional arg.
While we're here: HF cache was empty too (Voxtral 4B BF16 ~8 GB
download on first start) — vLLM auto-downloads from HF on model
load, so no separate pre-pull step needed.
Three fixes from the second-wave deploy attempts:
* voxtral: vllm/vllm-omni doesn't publish a `latest` tag — pull
failed with "manifest unknown". Pinned VOXTRAL_VLLM_TAG to v0.18.0
(released 2026-03-29, the day after the Voxtral 4B TTS release —
first cut with Voxtral support).
* kyutai-tts: NillPointer wrapper exposes ONLY /health (root) and
POST /v1/audio/speech. No /v1/models, no /v1/audio/voices —
those return 404. Verified by /openapi.json against the live
container. Compose healthcheck + playbook wait + verify steps
all repointed at the actual paths. POST /v1/audio/speech is now
smoke-tested with a RIFF WAV assertion (same pattern as fish-s2).
* fish-s2: added FISH_S2_MODEL env var so the model variant is
swappable via .env without rebuilding. Both s2-pro (default) and
s1-mini are pre-pulled into the bind-mount; LLAMA_CHECKPOINT_PATH
+ DECODER_CHECKPOINT_PATH now use ${FISH_S2_MODEL:-s2-pro}.
s1-mini was originally gated on fishaudio's HF org (401), but
niobures/OpenAudio-S1 mirrors the same files openly — pulled
from there via a one-shot snapshot_download.
Adds the three premier 2026 TTS releases we missed during the original
fleet build-out (early April), all licensed for self-host:
* Fish Audio S2-Pro (port 8195, GPU 1 / A6000) — released 2026-03-09.
4B dual-AR (Slow + Fast) trained on 10M+ hours / 80+ languages.
Headline: 15,000+ paralinguistic / emotion tags via natural language
([laugh] [whispers] [super happy] etc.) — a step-function over
Chatterbox Turbo's 9 fixed tags. 91.61% paralinguistic win rate on
EmergentTTS-Eval. ~150 ms streaming TTFB, voice cloning, MIT-style
open. ~17 GB VRAM.
* Voxtral TTS (port 8197, GPU 1 / A6000) — Mistral, released 2026-03-28.
4B open-weight, 70 ms model latency, 9.7× realtime. 68.4% blind A/B
win rate vs ElevenLabs Flash v2.5 in cloning. 8 languages
(EN/FR/DE/ES/IT/PT/NL/HI). Served via vLLM-Omni (Mistral's partner
serving stack) — published Docker image, no local build. ~16 GB VRAM.
CC BY-NC license — personal/research use only; flagged in README.
* Kyutai TTS (port 8198, GPU 0 / 3090) — kyutai/tts-1.6b-en_fr.
Trained on 2.5M hours from the Moshi/Mimi team. Claimed 220 ms in
solo setup, 32 simultaneous streams under 350 ms on L40. Kyutai's
official deploy is Rust + websockets only; using NillPointer's
community OpenAI-compat wrapper to bridge to /v1/audio/speech so
it slots into the same bench harness. ~4-6 GB VRAM.
Each stack: compose.yaml (build context, env, volumes, healthcheck,
homepage label), .env.example (all tunables documented), README.md
(why it exists, headline numbers, API, deploy + hardware notes).
Playbooks at playbooks/deploy-{fish-s2,voxtral,kyutai-tts}.yaml are
idempotent in the same shape as the existing deploy-vibevoice /
deploy-chatterbox playbooks.
Port allocations on irv-ml1 after this lands: 8188 ComfyUI, 8190
CosyVoice, 8191 Qwen3-TTS, 8192 IndexTTS-2, 8193 Kokoro, 8194
VibeVoice, 8195 Fish, 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai,
8765 Parakeet ASR.
vh