stacks/{fish-s2,voxtral,kyutai-tts}: three new TTS deploys for irv-ml1 quality A/B

Adds the three premier 2026 TTS releases we missed during the original
fleet build-out (early April), all licensed for self-host:

* Fish Audio S2-Pro (port 8195, GPU 1 / A6000) — released 2026-03-09.
  4B dual-AR (Slow + Fast) trained on 10M+ hours / 80+ languages.
  Headline: 15,000+ paralinguistic / emotion tags via natural language
  ([laugh] [whispers] [super happy] etc.) — a step-function over
  Chatterbox Turbo's 9 fixed tags. 91.61% paralinguistic win rate on
  EmergentTTS-Eval. ~150 ms streaming TTFB, voice cloning, MIT-style
  open. ~17 GB VRAM.

* Voxtral TTS (port 8197, GPU 1 / A6000) — Mistral, released 2026-03-28.
  4B open-weight, 70 ms model latency, 9.7× realtime. 68.4% blind A/B
  win rate vs ElevenLabs Flash v2.5 in cloning. 8 languages
  (EN/FR/DE/ES/IT/PT/NL/HI). Served via vLLM-Omni (Mistral's partner
  serving stack) — published Docker image, no local build. ~16 GB VRAM.
  CC BY-NC license — personal/research use only; flagged in README.

* Kyutai TTS (port 8198, GPU 0 / 3090) — kyutai/tts-1.6b-en_fr.
  Trained on 2.5M hours from the Moshi/Mimi team. Claimed 220 ms in
  solo setup, 32 simultaneous streams under 350 ms on L40. Kyutai's
  official deploy is Rust + websockets only; using NillPointer's
  community OpenAI-compat wrapper to bridge to /v1/audio/speech so
  it slots into the same bench harness. ~4-6 GB VRAM.

Each stack: compose.yaml (build context, env, volumes, healthcheck,
homepage label), .env.example (all tunables documented), README.md
(why it exists, headline numbers, API, deploy + hardware notes).
Playbooks at playbooks/deploy-{fish-s2,voxtral,kyutai-tts}.yaml are
idempotent in the same shape as the existing deploy-vibevoice /
deploy-chatterbox playbooks.

Port allocations on irv-ml1 after this lands: 8188 ComfyUI, 8190
CosyVoice, 8191 Qwen3-TTS, 8192 IndexTTS-2, 8193 Kokoro, 8194
VibeVoice, 8195 Fish, 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai,
8765 Parakeet ASR.

playbooks/deploy-fish-s2.yaml

@@ -0,0 +1,99 @@
+# Deploy Fish Audio S2-Pro (richest paralinguistic open-source TTS) to
+# irv-ml1.
+#
+# Builds the image locally from fishaudio/fish-speech via docker buildx
+# git URL context. ~10-15 min cold build (CUDA 12.x + torch + flash-attn
+# + Fish's training/inference deps). First start downloads s2-pro
+# (~9 GB BF16) into the bind-mounted HF cache. Generous /v1/health
+# wait deadline accommodates both.
+#
+# Usage:
+#   scripts/elway irv-ml1 --playbook playbooks/deploy-fish-s2.yaml
+#
+# Idempotent — every step is creates-/when-gated; rerun is safe.
+
+vars:
+  compose_dir: /opt/docker/compose/fish-s2
+  references_dir: /worktank/fish-s2/references
+  checkpoints_dir: /worktank/fish-s2/checkpoints
+  cache_dir: /worktank/fish-s2/hf_cache
+  host_port: "8195"
+
+steps:
+  # ── host-side dirs ──────────────────────────────────────────────────
+
+  - name: Ensure /worktank/fish-s2 root exists (one-time, sudo)
+    shell: mkdir -p /worktank/fish-s2
+    sudo: true
+    creates: /worktank/fish-s2
+
+  - name: Chown /worktank/fish-s2 to lkraven
+    shell: chown -R lkraven:lkraven /worktank/fish-s2
+    sudo: true
+    when: "[ \"$(stat -c %U /worktank/fish-s2)\" != \"lkraven\" ]"
+
+  - name: Ensure references dir exists
+    shell: mkdir -p {{ references_dir }}
+    creates: "{{ references_dir }}"
+
+  - name: Ensure checkpoints dir exists
+    shell: mkdir -p {{ checkpoints_dir }}
+    creates: "{{ checkpoints_dir }}"
+
+  - name: Ensure HF cache dir exists
+    shell: mkdir -p {{ cache_dir }}
+    creates: "{{ cache_dir }}"
+
+  - name: Ensure compose dir exists
+    shell: mkdir -p {{ compose_dir }}
+    creates: "{{ compose_dir }}"
+
+  # ── deploy compose + env ────────────────────────────────────────────
+
+  - name: Upload compose.yaml
+    upload:
+      src: stacks/fish-s2/compose.yaml
+      dest: "{{ compose_dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Seed .env from template (only if absent)
+    upload:
+      src: stacks/fish-s2/.env.example
+      dest: "{{ compose_dir }}/.env"
+      mode: "0644"
+    when: "[ ! -f {{ compose_dir }}/.env ]"
+
+  # ── build + bring up ────────────────────────────────────────────────
+
+  - name: docker compose build (~10-15 min first time; cached after)
+    shell: |
+      set -o pipefail
+      cd {{ compose_dir }} && docker compose build 2>&1 \
+        | grep -vE '^#[0-9]+ |^ => |^=> |Collecting|Downloading|Requirement|Using cached|Installing collected|Successfully (installed|built)|━'
+
+  - name: docker compose up -d
+    shell: cd {{ compose_dir }} && docker compose up -d
+
+  - name: Wait for /v1/health to respond (allow ~15 min for first model download + warmup)
+    shell: |
+      for i in $(seq 1 180); do
+        curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/health && exit 0
+        sleep 5
+      done
+      exit 1
+    changed_when: "false"
+
+verify:
+  - name: /v1/health returns 200
+    shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/v1/health
+    changed_when: "false"
+
+  - name: /v1/audio/voices returns valid JSON
+    shell: |
+      curl -sf http://localhost:{{ host_port }}/v1/audio/voices \
+        | python3 -c "import json,sys; json.load(sys.stdin)"
+    changed_when: "false"
+
+  - name: Container is running
+    shell: docker inspect fish-s2 --format '{{.State.Status}}' | grep -q running
+    changed_when: "false"

playbooks/deploy-kyutai-tts.yaml

@@ -0,0 +1,100 @@
+# Deploy Kyutai TTS (1.6B EN/FR streaming, 220 ms claimed latency) to
+# irv-ml1.
+#
+# Builds the image locally from NillPointer/Kyutai-TTS-Server via
+# docker buildx git URL context. ~5-8 min cold build (CUDA + torch +
+# moshi + Kyutai's Mimi codec deps). First start downloads
+# kyutai/tts-1.6b-en_fr (~3-6 GB) into the bind-mounted HF cache.
+#
+# Usage:
+#   scripts/elway irv-ml1 --playbook playbooks/deploy-kyutai-tts.yaml
+#
+# Note: Kyutai's official deploy is Rust + websockets only. This stack
+# uses the NillPointer community wrapper to bridge to OpenAI-compat
+# HTTP — adds Python overhead on the request path, so measured TTFB
+# will be higher than the bare-Rust 220 ms claim. See README.
+#
+# Idempotent — every step is creates-/when-gated; rerun is safe.
+
+vars:
+  compose_dir: /opt/docker/compose/kyutai-tts
+  voices_dir: /worktank/kyutai-tts/voices
+  cache_dir: /worktank/kyutai-tts/hf_cache
+  host_port: "8198"
+
+steps:
+  # ── host-side dirs ──────────────────────────────────────────────────
+
+  - name: Ensure /worktank/kyutai-tts root exists (one-time, sudo)
+    shell: mkdir -p /worktank/kyutai-tts
+    sudo: true
+    creates: /worktank/kyutai-tts
+
+  - name: Chown /worktank/kyutai-tts to lkraven
+    shell: chown -R lkraven:lkraven /worktank/kyutai-tts
+    sudo: true
+    when: "[ \"$(stat -c %U /worktank/kyutai-tts)\" != \"lkraven\" ]"
+
+  - name: Ensure voices dir exists
+    shell: mkdir -p {{ voices_dir }}
+    creates: "{{ voices_dir }}"
+
+  - name: Ensure HF cache dir exists
+    shell: mkdir -p {{ cache_dir }}
+    creates: "{{ cache_dir }}"
+
+  - name: Ensure compose dir exists
+    shell: mkdir -p {{ compose_dir }}
+    creates: "{{ compose_dir }}"
+
+  # ── deploy compose + env ────────────────────────────────────────────
+
+  - name: Upload compose.yaml
+    upload:
+      src: stacks/kyutai-tts/compose.yaml
+      dest: "{{ compose_dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Seed .env from template (only if absent)
+    upload:
+      src: stacks/kyutai-tts/.env.example
+      dest: "{{ compose_dir }}/.env"
+      mode: "0644"
+    when: "[ ! -f {{ compose_dir }}/.env ]"
+
+  # ── build + bring up ────────────────────────────────────────────────
+
+  - name: docker compose build (~5-8 min first time; cached after)
+    shell: |
+      set -o pipefail
+      cd {{ compose_dir }} && docker compose build 2>&1 \
+        | grep -vE '^#[0-9]+ |^ => |^=> |Collecting|Downloading|Requirement|Using cached|Installing collected|Successfully (installed|built)|━'
+
+  - name: docker compose up -d
+    shell: cd {{ compose_dir }} && docker compose up -d
+
+  - name: Wait for /v1/models to respond (allow ~10 min for first download + warmup)
+    shell: |
+      for i in $(seq 1 120); do
+        curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/models && exit 0
+        sleep 5
+      done
+      exit 1
+    changed_when: "false"
+
+verify:
+  - name: /v1/models returns valid JSON
+    shell: |
+      curl -sf http://localhost:{{ host_port }}/v1/models \
+        | python3 -c "import json,sys; json.load(sys.stdin)"
+    changed_when: "false"
+
+  - name: /v1/audio/voices returns valid JSON
+    shell: |
+      curl -sf http://localhost:{{ host_port }}/v1/audio/voices \
+        | python3 -c "import json,sys; json.load(sys.stdin)"
+    changed_when: "false"
+
+  - name: Container is running
+    shell: docker inspect kyutai-tts --format '{{.State.Status}}' | grep -q running
+    changed_when: "false"

playbooks/deploy-voxtral.yaml

@@ -0,0 +1,89 @@
+# Deploy Voxtral TTS (Mistral 4B multilingual streaming) to irv-ml1.
+#
+# Pulls the official vllm/vllm-omni image (Mistral's partner serving
+# stack) — no local build. ~3-5 min cold pull on first deploy. First
+# container start downloads Voxtral-4B-TTS-2603 (~8 GB BF16) into the
+# bind-mounted HF cache. Generous /v1/models wait deadline accommodates
+# the first model pull + vLLM warmup.
+#
+# Usage:
+#   scripts/elway irv-ml1 --playbook playbooks/deploy-voxtral.yaml
+#
+# License caveat: Voxtral is CC BY-NC. Personal / research use only.
+# See stacks/voxtral/README.md.
+#
+# Idempotent — every step is creates-/when-gated; rerun is safe.
+
+vars:
+  compose_dir: /opt/docker/compose/voxtral
+  voices_dir: /worktank/voxtral/voices
+  cache_dir: /worktank/voxtral/hf_cache
+  host_port: "8197"
+
+steps:
+  # ── host-side dirs ──────────────────────────────────────────────────
+
+  - name: Ensure /worktank/voxtral root exists (one-time, sudo)
+    shell: mkdir -p /worktank/voxtral
+    sudo: true
+    creates: /worktank/voxtral
+
+  - name: Chown /worktank/voxtral to lkraven
+    shell: chown -R lkraven:lkraven /worktank/voxtral
+    sudo: true
+    when: "[ \"$(stat -c %U /worktank/voxtral)\" != \"lkraven\" ]"
+
+  - name: Ensure voices dir exists
+    shell: mkdir -p {{ voices_dir }}
+    creates: "{{ voices_dir }}"
+
+  - name: Ensure HF cache dir exists
+    shell: mkdir -p {{ cache_dir }}
+    creates: "{{ cache_dir }}"
+
+  - name: Ensure compose dir exists
+    shell: mkdir -p {{ compose_dir }}
+    creates: "{{ compose_dir }}"
+
+  # ── deploy compose + env ────────────────────────────────────────────
+
+  - name: Upload compose.yaml
+    upload:
+      src: stacks/voxtral/compose.yaml
+      dest: "{{ compose_dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Seed .env from template (only if absent)
+    upload:
+      src: stacks/voxtral/.env.example
+      dest: "{{ compose_dir }}/.env"
+      mode: "0644"
+    when: "[ ! -f {{ compose_dir }}/.env ]"
+
+  # ── pull + bring up ─────────────────────────────────────────────────
+
+  - name: docker compose pull (~3-5 min cold)
+    shell: cd {{ compose_dir }} && docker compose pull 2>&1 | tail -20
+
+  - name: docker compose up -d
+    shell: cd {{ compose_dir }} && docker compose up -d
+
+  - name: Wait for /v1/models to report model loaded (allow ~10 min for first pull + warmup)
+    shell: |
+      for i in $(seq 1 120); do
+        curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/models && exit 0
+        sleep 5
+      done
+      exit 1
+    changed_when: "false"
+
+verify:
+  - name: /v1/models reports loaded model
+    shell: |
+      curl -sf http://localhost:{{ host_port }}/v1/models \
+        | python3 -c "import json,sys; d=json.load(sys.stdin); assert any('Voxtral' in m.get('id','') for m in d.get('data',[]))"
+    changed_when: "false"
+
+  - name: Container is running
+    shell: docker inspect voxtral --format '{{.State.Status}}' | grep -q running
+    changed_when: "false"

stacks/fish-s2/.env.example

@@ -0,0 +1,50 @@
+# Fish Audio S2-Pro stack tunables. Copy to `.env` on irv-ml1 before
+# deploying.
+
+# ── build pin ────────────────────────────────────────────────────────
+# SHA of fishaudio/fish-speech to build from. Bump + rebuild when you
+# want upstream wrapper updates. Use the FULL 40-char SHA — docker
+# buildx's git source resolver doesn't accept short hashes.
+FISH_S2_SHA=main
+
+# Local image tag — bump when you change build context to force a
+# fresh layer build.
+FISH_S2_TAG=v1
+
+# ── network ──────────────────────────────────────────────────────────
+# Host port (container listens on 8080 internally; we map to 8195
+# externally to fit alongside the rest of the irv-ml1 TTS slate).
+# Port reservations on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191
+# Qwen3-TTS, 8192 IndexTTS-2, 8193 Kokoro, 8194 VibeVoice, 8196
+# Chatterbox, 8765 Parakeet ASR.
+FISH_S2_PORT=8195
+FISH_S2_BIND=0.0.0.0
+
+# ── runtime / GPU ────────────────────────────────────────────────────
+# GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB), "all"
+# = both visible. Fish S2-Pro consumes ~17 GB during inference (4B
+# model + KV cache), so the A6000 is the right home with comfortable
+# headroom. The 3090's 24 GB CAN fit it but leaves ~5 GB for KV which
+# is tight for long contexts.
+FISH_S2_GPU_DEVICES=1
+
+# torch.compile on first inference of each shape adds ~60 s warmup but
+# unlocks ~10× speedup per upstream. Worth it for sustained use; flip
+# to 0 to disable if you hit a torch.compile bug on a future
+# checkpoint.
+FISH_S2_COMPILE=1
+
+# ── persistent storage on the host ───────────────────────────────────
+# Model checkpoints — Fish auto-downloads s2-pro on first run (~9 GB
+# at BF16) and caches under here. Persistent across container
+# recreates so we don't re-pull.
+FISH_S2_CHECKPOINT_DIR=/worktank/fish-s2/checkpoints
+
+# Reference audio for voice cloning. Drop clean ~5-15 s clips here
+# (WAV / FLAC / MP3); the wrapper scans on request. One clean clip
+# per voice; name them descriptively.
+FISH_S2_REFERENCE_DIR=/worktank/fish-s2/references
+
+# HF cache — persistent across container recreates to skip the model
+# re-pull. Worth ~9 GB on disk.
+FISH_S2_CACHE_DIR=/worktank/fish-s2/hf_cache

stacks/fish-s2/README.md

@@ -0,0 +1,105 @@
+# Fish Audio S2-Pro
+
+[fishaudio/s2-pro](https://huggingface.co/fishaudio/s2-pro) — the most
+expressive open-source TTS model as of 2026-04, served via the
+official [fishaudio/fish-speech](https://github.com/fishaudio/fish-speech)
+inference engine.
+
+## Why this stack exists
+
+Three of the existing TTS already cover the basics — Kokoro for raw
+speed, Chatterbox for speed-with-cloning, IndexTTS-2 for precision
+emotion control. Fish Audio S2-Pro fills a different slot:
+**dramatically richer paralinguistic control via natural-language
+tags** (15,000+ vs Chatterbox Turbo's 9 fixed tags), with comparable
+latency (~150 ms streaming) and voice cloning.
+
+Released March 9, 2026; we missed it during the original irv-ml1
+build-out in early April.
+
+| | use case |
+|---|---|
+| **Fish Audio S2-Pro** | richest emotive / paralinguistic English TTS — 15k+ tags |
+| Kokoro | low-latency English, fixed voice library |
+| Chatterbox Turbo | low-latency English w/ cloning + 9 paralinguistic tags |
+| IndexTTS-2 | English voice cloning + emotion vector / text control |
+| Qwen3-TTS-1.7B | English voice cloning (slow on official backend) |
+| CosyVoice 3 | multilingual (Chinese-leaning) |
+| VibeVoice 1.5B | long-form / multi-speaker dialogue |
+
+## Architecture
+
+Dual-AR (Slow + Fast):
+- **Slow AR** operates along the time axis, predicts the primary
+  semantic codebook.
+- **Fast AR** generates the remaining 9 residual codebooks per time
+  step, reconstructing fine-grained acoustic detail.
+
+Trained on 10M+ hours of audio across 80+ languages with
+reinforcement-learning alignment. Win rates per upstream:
+
+| benchmark | S2-Pro |
+|---|---|
+| EmergentTTS-Eval paralinguistics | 91.61% |
+| Blind A/B vs ElevenLabs Flash v2.5 (multilingual) | strong |
+
+## Headline features
+
+- **15,000+ paralinguistic / emotion tags** via natural language:
+  ```
+  [laugh] [whispers] [super happy] [sigh] [excited] [heavy breathing]
+  [angry] [sleepy] [crying] [surprise] ...
+  ```
+  Drop them inline in the input text. Different shape from
+  IndexTTS-2's 8-vector emotion control — this is "say it like this"
+  markup directly in the prompt, with a far larger vocabulary.
+- **Voice cloning** from ~5-15 s reference WAV.
+- **Multi-speaker / multi-turn** generation natively supported.
+- **80+ languages** (English-strong, not Chinese-leaning like
+  CosyVoice).
+
+## API
+
+OpenAI-compat at `http://10.100.79.3:8195`:
+
+```bash
+# Single-shot synthesis with paralinguistic tags inline.
+curl -fsS -X POST http://10.100.79.3:8195/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"fish-s2","input":"Oh wow [super happy] I cannot believe it. [laugh] What a day.","voice":"glados","response_format":"wav"}' \
+  > out.wav
+
+# Built-in voices.
+curl http://10.100.79.3:8195/v1/audio/voices
+
+# Streaming.
+curl -fsS -X POST http://10.100.79.3:8195/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"fish-s2","input":"long passage…","voice":"glados","stream":true}' \
+  | mpv --no-cache -
+```
+
+WebUI at `/`. OpenAPI / docs at `/docs`. Healthcheck at `/v1/health`.
+
+## Voice library
+
+Drop reference WAV / MP3 / FLAC into
+`/worktank/fish-s2/references/` on the host. The wrapper scans on
+request — no restart needed. Use clean ~5-15 s clips, single
+speaker, ideally with diverse intonation samples.
+
+## Deploy
+
+```bash
+scripts/elway irv-ml1 --playbook playbooks/deploy-fish-s2.yaml
+```
+
+First boot pulls the s2-pro checkpoint (~9 GB BF16) into the HF
+cache + warms torch.compile (adds ~60 s). Both are cached afterwards.
+
+## Hardware footprint
+
+- **VRAM**: ~17 GB practical, 24 GB recommended. Pinned to GPU 1
+  (RTX A6000) by default — plenty of headroom for long contexts and
+  large mmproj if a future checkpoint adds vision.
+- **Disk**: ~9 GB for the model checkpoint + HF cache.

stacks/fish-s2/compose.yaml

@@ -0,0 +1,76 @@
+# Fish Audio S2-Pro — the most expressive open-source TTS as of
+# 2026-04. 4B params, dual-AR architecture (Slow AR for semantic
+# codebook + Fast AR for 9 residual codebooks), trained on 10M+ hours
+# across 80+ languages. ~150 ms streaming TTFB on warm GPU.
+#
+# Why this stack alongside the existing TTS:
+#   * Headline feature: 15,000+ paralinguistic / emotion tags via
+#     natural language, e.g. [laugh] [whispers] [super happy] [sigh].
+#     Chatterbox Turbo only has 9 fixed tags — Fish's vocabulary is
+#     dramatically richer for any emotive use case.
+#   * 91.61% paralinguistic win rate on EmergentTTS-Eval — currently
+#     the leader on that benchmark.
+#   * Voice cloning + native multi-speaker / multi-turn generation.
+#   * MIT-style license (weights, training code, inference engine all
+#     open).
+#
+# Image is built locally from upstream's repo via docker buildx
+# git-context. Upstream ships a compose with `--profile server` for
+# the API path; we adapt that to our `restart: unless-stopped`
+# convention + bind-mount layout.
+#
+# All tunables live in .env — edit that, not this file.
+
+services:
+  fish-s2:
+    image: local/fish-s2:${FISH_S2_TAG}
+    build:
+      context: https://github.com/fishaudio/fish-speech.git#${FISH_S2_SHA}
+      dockerfile: dockerfile
+      args:
+        # Upstream's Dockerfile reads BACKEND to choose CUDA vs CPU
+        # paths during pip install. We always want CUDA on irv-ml1.
+        BACKEND: cuda
+    container_name: fish-s2
+    restart: unless-stopped
+    runtime: nvidia
+    ports:
+      - "${FISH_S2_BIND:-0.0.0.0}:${FISH_S2_PORT}:8080"
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=${FISH_S2_GPU_DEVICES:-1}
+      - BACKEND=cuda
+      # COMPILE=1 enables torch.compile — upstream claims ~10× speedup
+      # on the autoregressive forward, at the cost of ~60 s warmup the
+      # first time each input shape is seen. Worth the speedup; turn
+      # off via .env if you hit a torch.compile bug on a future model
+      # checkpoint.
+      - COMPILE=${FISH_S2_COMPILE:-1}
+      - API_PORT=8080
+      # Hugging Face cache for model weights — first start pulls
+      # fishaudio/s2-pro (~9 GB BF16) into this dir.
+      - HF_HOME=/app/hf_cache
+    volumes:
+      # Model checkpoints (auto-downloaded on first run, then cached).
+      - ${FISH_S2_CHECKPOINT_DIR}:/app/checkpoints
+      # Reference audio for voice cloning — drop ~5–15 s WAV clips here.
+      - ${FISH_S2_REFERENCE_DIR}:/app/references
+      # Persistent HF cache so model re-pull only happens on first deploy.
+      - ${FISH_S2_CACHE_DIR}:/app/hf_cache
+    healthcheck:
+      # Fish ships /v1/health on the API server. python urllib instead
+      # of curl because the upstream image is python-based and may not
+      # carry curl. 127.0.0.1 explicit to dodge the IPv6-first
+      # localhost trap we hit on chatterbox + news-digest.
+      test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/v1/health', timeout=5).status==200 else 1)\""]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      # First boot: torch.compile warmup + first-pull HF download +
+      # checkpoint load. Generous deadline.
+      start_period: 900s
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=Fish Audio S2-Pro
+      - homepage.icon=mdi-fish
+      - homepage.description=Most expressive open-source TTS — 15k+ paralinguistic tags, voice cloning, 80+ languages (irv-ml1)
+      - homepage.href=http://10.100.79.3:${FISH_S2_PORT}

stacks/kyutai-tts/.env.example

@@ -0,0 +1,32 @@
+# Kyutai TTS stack tunables. Copy to `.env` on irv-ml1 before
+# deploying.
+
+# ── build pin ────────────────────────────────────────────────────────
+# SHA of NillPointer/Kyutai-TTS-Server to build from. Use the FULL
+# 40-char SHA — docker buildx's git source resolver doesn't accept
+# short hashes.
+KYUTAI_TTS_SHA=main
+
+# Local image tag — bump when you change build context to force a
+# fresh layer build.
+KYUTAI_TTS_TAG=v1
+
+# Kyutai model on HF. Available variants:
+#   kyutai/tts-1.6b-en_fr — bilingual EN/FR, 1.6B params (default)
+#   kyutai/pocket-tts     — 100M, CPU-realtime, EN-only (lighter alt)
+KYUTAI_TTS_MODEL=kyutai/tts-1.6b-en_fr
+
+# ── network ──────────────────────────────────────────────────────────
+KYUTAI_TTS_PORT=8198
+KYUTAI_TTS_BIND=0.0.0.0
+
+# ── runtime / GPU ────────────────────────────────────────────────────
+# GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB).
+# 1.6B Kyutai needs ~4-6 GB practical, fits comfortably on either.
+# Pinned to GPU 0 (3090) by default since the A6000 is hosting the
+# heavier Fish S2-Pro / Voxtral.
+KYUTAI_TTS_GPU_DEVICES=0
+
+# ── persistent storage on the host ───────────────────────────────────
+KYUTAI_TTS_CACHE_DIR=/worktank/kyutai-tts/hf_cache
+KYUTAI_TTS_VOICES_DIR=/worktank/kyutai-tts/voices

stacks/kyutai-tts/README.md

@@ -0,0 +1,83 @@
+# Kyutai TTS
+
+[kyutai/tts-1.6b-en_fr](https://huggingface.co/kyutai/tts-1.6b-en_fr)
+— Kyutai's flagship streaming TTS (1.6B params, EN/FR bilingual,
+trained on 2.5M hours), served via the
+[NillPointer/Kyutai-TTS-Server](https://github.com/NillPointer/Kyutai-TTS-Server)
+community OpenAI-compatible wrapper.
+
+## Why this stack exists
+
+Kyutai's claim is the **lowest streaming latency in this size class**:
+220 ms in solo setup; up to 32 simultaneous streams under 350 ms on
+a single L40-class GPU. Worth bench-comparing against:
+
+| | claimed latency | use case |
+|---|---|---|
+| **Kyutai TTS** | **~220 ms** | streaming EN/FR, conversational dialogue heritage |
+| Kokoro | ~26 ms TTFB measured | low-latency English, fixed voice library |
+| Chatterbox Turbo | ~1.2 s TTFB measured | English w/ cloning + 9 paralinguistic tags |
+| Fish Audio S2-Pro | ~150 ms claimed | richest paralinguistic English |
+| Voxtral | ~70 ms model latency | multilingual EN/FR/DE/ES/IT/PT/NL/HI |
+
+## Deployment notes
+
+Kyutai's official deployment path is **Rust + websockets only** (no
+HTTP, no OpenAI-compat). That doesn't fit the OpenAI-`/v1/audio/speech`
+contract the rest of our TTS fleet uses. The
+NillPointer/Kyutai-TTS-Server community wrapper bridges Kyutai's
+native streaming to the OpenAI HTTP shape, which lets us slot it
+into the same bench harness as the others.
+
+**Tradeoff**: the wrapper adds Python overhead on the request path,
+so measured latency on this stack will be *higher* than Kyutai's
+220 ms claim (which is for the bare Rust server). If we measure
+~400-500 ms TTFB end-to-end, the wrapper is the floor — Kyutai itself
+is hitting its target.
+
+## Architecture heritage
+
+Kyutai's TTS shares the **Mimi** neural codec + **Moshi** dialogue
+modeling framework. Both target full-duplex conversational AI (Moshi
+is their flagship speech-text foundation model). The TTS-only model
+is the "synthesis half" of the stack, distilled for low-latency
+streaming.
+
+Trained on **2.5M hours** — a different scaling regime from the
+others (CosyVoice 5k, Fish 10M, Voxtral undisclosed).
+
+## API
+
+OpenAI-compat at `http://10.100.79.3:8198`:
+
+```bash
+# Single-shot synthesis.
+curl -fsS -X POST http://10.100.79.3:8198/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"tts-1.6b-en_fr","input":"Hello there.","voice":"default","response_format":"wav"}' \
+  > out.wav
+
+# Streaming.
+curl -fsS -X POST http://10.100.79.3:8198/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"tts-1.6b-en_fr","input":"long passage…","voice":"default","stream":true}' \
+  | mpv --no-cache -
+
+# Built-in voices.
+curl http://10.100.79.3:8198/v1/audio/voices
+```
+
+## Deploy
+
+```bash
+scripts/elway irv-ml1 --playbook playbooks/deploy-kyutai-tts.yaml
+```
+
+First boot pulls the kyutai/tts-1.6b-en_fr checkpoint (~3-6 GB) into
+the HF cache.
+
+## Hardware footprint
+
+- **VRAM**: ~4-6 GB practical. Pinned to GPU 0 (RTX 3090) by default
+  — the A6000 is hosting the heavier Fish S2-Pro / Voxtral.
+- **Disk**: ~6 GB for the checkpoint + HF cache.

stacks/kyutai-tts/compose.yaml

@@ -0,0 +1,56 @@
+# Kyutai TTS — 1.6B / 2B-class streaming TTS from Kyutai (the Moshi /
+# Mimi team), trained on 2.5M hours. 220 ms latency in solo setup; up
+# to 32 simultaneous streams under 350 ms on an L40-class GPU.
+#
+# Served via the NillPointer/Kyutai-TTS-Server community wrapper —
+# Kyutai's official deployment is Rust + websockets only, which doesn't
+# fit our OpenAI-compat fleet. The community wrapper bridges Kyutai's
+# native streaming to the OpenAI /v1/audio/speech contract.
+#
+# Why this stack alongside the existing TTS:
+#   * Kyutai's claim is the lowest streaming latency in this size
+#     class (220 ms on a single GPU). Worth bench-comparing against
+#     Chatterbox (~1.2 s) and Fish S2-Pro (~150 ms claimed).
+#   * Trained on 2.5M hours — a different scaling regime from the
+#     others (CosyVoice 5k hrs, Fish 10M hrs).
+#   * Designed for full-duplex dialogue (Moshi heritage) — may surface
+#     conversational quality the others lack.
+#
+# All tunables live in .env — edit that, not this file.
+
+services:
+  kyutai-tts:
+    image: local/kyutai-tts:${KYUTAI_TTS_TAG}
+    build:
+      context: https://github.com/NillPointer/Kyutai-TTS-Server.git#${KYUTAI_TTS_SHA}
+      dockerfile: Dockerfile
+    container_name: kyutai-tts
+    restart: unless-stopped
+    runtime: nvidia
+    ports:
+      - "${KYUTAI_TTS_BIND:-0.0.0.0}:${KYUTAI_TTS_PORT}:8000"
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=${KYUTAI_TTS_GPU_DEVICES:-0}
+      # Kyutai's en/fr bilingual model on HF. Switch to a different
+      # checkpoint via .env without rebuilding.
+      - KYUTAI_MODEL=${KYUTAI_TTS_MODEL:-kyutai/tts-1.6b-en_fr}
+      - HF_HOME=/app/hf_cache
+    volumes:
+      - ${KYUTAI_TTS_CACHE_DIR}:/app/hf_cache
+      - ${KYUTAI_TTS_VOICES_DIR}:/app/voices:ro
+    healthcheck:
+      # The wrapper exposes /v1/models for OpenAI-compat — same shape
+      # as Voxtral / Qwen3-TTS. Use that as the readiness signal.
+      # 127.0.0.1 explicit to dodge IPv4/IPv6 localhost race.
+      test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      # First boot pulls the Kyutai checkpoint (~3-6 GB) + warms.
+      start_period: 600s
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=Kyutai TTS
+      - homepage.icon=mdi-radio-tower
+      - homepage.description=Ultra-low-latency streaming TTS — 220 ms on solo GPU, EN/FR (irv-ml1)
+      - homepage.href=http://10.100.79.3:${KYUTAI_TTS_PORT}

stacks/voxtral/.env.example

@@ -0,0 +1,38 @@
+# Voxtral TTS stack tunables. Copy to `.env` on irv-ml1 before
+# deploying.
+
+# ── image pin ────────────────────────────────────────────────────────
+# vLLM-Omni image tag (Mistral's partner serving stack for Voxtral).
+# Use a specific version rather than `latest` — vLLM moves fast and
+# Voxtral has version-specific compatibility.
+VOXTRAL_VLLM_TAG=latest
+
+# Voxtral model on Hugging Face. The 4B variant is the only released
+# checkpoint as of 2026-04. Default BF16 weights are ~8 GB.
+VOXTRAL_MODEL=mistralai/Voxtral-4B-TTS-2603
+
+# ── network ──────────────────────────────────────────────────────────
+# Host port (container listens on 8000 internally).
+VOXTRAL_PORT=8197
+VOXTRAL_BIND=0.0.0.0
+
+# ── runtime / GPU ────────────────────────────────────────────────────
+# GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB).
+# Voxtral 4B BF16 needs ~16 GB practical (model + KV + activation).
+# Pinned to A6000 by default for headroom. The 3090 fits but is tight
+# for long streaming sessions.
+VOXTRAL_GPU_DEVICES=1
+
+# vLLM GPU memory utilization fraction (0.0-1.0). 0.85 = leave 15%
+# headroom for other processes / KV cache spikes. Lower if running
+# alongside other GPU workloads on the same device.
+VOXTRAL_GPU_UTIL=0.85
+
+# ── persistent storage on the host ───────────────────────────────────
+# HF cache — first start pulls the Voxtral checkpoint (~8 GB) into
+# this dir. Persistent across container recreates.
+VOXTRAL_CACHE_DIR=/worktank/voxtral/hf_cache
+
+# Reference voices for cloning. Read-only mount inside the container.
+# Drop ~5-15 s WAV / FLAC clips here.
+VOXTRAL_VOICES_DIR=/worktank/voxtral/voices

stacks/voxtral/README.md

@@ -0,0 +1,85 @@
+# Voxtral TTS
+
+[mistralai/Voxtral-4B-TTS-2603](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603)
+— Mistral AI's 4B open-weight streaming TTS, served via the
+[vLLM-Omni](https://github.com/vllm-project/vllm-omni) production
+serving stack (Mistral co-developed). Released March 28, 2026.
+
+## ⚠️ License
+
+**CC BY-NC.** Personal use, research, and internal tooling are fine.
+**Don't ship Voxtral output in any commercial product** without
+re-licensing from Mistral. The other TTS in this fleet (Kokoro,
+Chatterbox, Fish S2-Pro, IndexTTS-2, Qwen3-TTS, CosyVoice) are all
+open-licensed and clean for commercial work.
+
+## Why this stack exists
+
+Multilingual streaming with serious speed:
+
+| | use case |
+|---|---|
+| **Voxtral** | multilingual EN/FR/DE/ES/IT/PT/NL/HI streaming, 70 ms model latency |
+| Kokoro | low-latency English, fixed voice library |
+| Chatterbox Turbo | low-latency English w/ cloning + 9 paralinguistic tags |
+| Fish Audio S2-Pro | richest paralinguistic English (15k+ tags) |
+| IndexTTS-2 | English voice cloning + emotion vector / text control |
+| Qwen3-TTS-1.7B | English voice cloning (slow on official backend) |
+| CosyVoice 3 | multilingual (Chinese-leaning) |
+| VibeVoice 1.5B | long-form / multi-speaker dialogue |
+
+Voxtral fills the **multilingual + low-latency + cloning** slot
+that's been weak in the fleet (CosyVoice is multilingual but slow on
+English; nothing else is multilingual at all).
+
+## Headline numbers
+
+- **70 ms** model latency for a typical 10 s sample (500-char input)
+- **9.7×** realtime factor
+- **68.4%** blind A/B win rate vs ElevenLabs Flash v2.5 in voice
+  cloning evaluations
+- **8 languages**: EN, FR, DE, ES, IT, PT, NL, HI
+
+## API
+
+vLLM-Omni serves an OpenAI-compatible API at
+`http://10.100.79.3:8197/v1`:
+
+```bash
+# Single-shot synthesis.
+curl -fsS -X POST http://10.100.79.3:8197/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"mistralai/Voxtral-4B-TTS-2603","input":"Hello there.","voice":"alloy","response_format":"wav"}' \
+  > out.wav
+
+# Streaming.
+curl -fsS -X POST http://10.100.79.3:8197/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"mistralai/Voxtral-4B-TTS-2603","input":"long passage…","voice":"alloy","stream":true}' \
+  | mpv --no-cache -
+
+# vLLM-Omni standard endpoints.
+curl http://10.100.79.3:8197/v1/models    # confirms model loaded
+curl http://10.100.79.3:8197/v1/audio/voices  # built-in + cloned voices
+```
+
+## Deploy
+
+```bash
+scripts/elway irv-ml1 --playbook playbooks/deploy-voxtral.yaml
+```
+
+First boot pulls Voxtral-4B (~8 GB BF16) into the HF cache + warms
+vLLM. Both are cached afterwards.
+
+## Hardware footprint
+
+- **VRAM**: ~16 GB practical (8 GB weights + KV + activation). Pinned
+  to GPU 1 (RTX A6000) by default — comfortable headroom. The 3090's
+  24 GB CAN fit but it's tight for long streaming sessions.
+- **Disk**: ~8 GB for the Voxtral checkpoint + HF cache.
+
+## Voice library
+
+Drop reference WAV / FLAC into `/worktank/voxtral/voices/` on the
+host. The wrapper scans on request — no restart needed.

stacks/voxtral/compose.yaml

@@ -0,0 +1,59 @@
+# Voxtral TTS — Mistral AI's 4B open-weight streaming TTS, served via
+# vLLM-Omni (the production serving stack Mistral co-developed for
+# Voxtral). Released March 28, 2026.
+#
+# Why this stack alongside the existing TTS:
+#   * 70 ms model latency, 9.7× realtime — fastest non-Kokoro option.
+#   * Multilingual-first (EN strong, plus FR, DE, ES, IT, PT, NL, HI).
+#     Different from CosyVoice's Chinese-leaning balance.
+#   * 68.4% blind A/B win rate vs ElevenLabs Flash v2.5 in cloning.
+#   * vLLM-Omni serving = continuous batching + paged attention — the
+#     same mechanism that gave qwen3.6 its speed on llama-swap.
+#
+# LICENSE: CC BY-NC. Personal / research use only. Don't ship Voxtral
+# output in any commercial product without re-licensing from Mistral.
+#
+# All tunables live in .env — edit that, not this file.
+
+services:
+  voxtral:
+    # vLLM-Omni image — Mistral's official partnership for Voxtral
+    # serving. Version-pinned via .env.
+    image: vllm/vllm-omni:${VOXTRAL_VLLM_TAG}
+    container_name: voxtral
+    restart: unless-stopped
+    runtime: nvidia
+    ports:
+      - "${VOXTRAL_BIND:-0.0.0.0}:${VOXTRAL_PORT}:8000"
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=${VOXTRAL_GPU_DEVICES:-1}
+      - HF_HOME=/root/.cache/huggingface
+      # vLLM serving args — see https://docs.vllm.ai for full list.
+      # We override the default model via cmd args below.
+    volumes:
+      - ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
+      - ${VOXTRAL_VOICES_DIR}:/voices:ro
+    # vLLM-Omni's serve command — model + dtype + port pinned.
+    command:
+      - --model=${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}
+      - --port=8000
+      - --dtype=bfloat16
+      - --gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}
+    healthcheck:
+      # vLLM-Omni exposes /health for liveness + /v1/models for readiness.
+      # /health 200 means the server's listening; /v1/models 200 means
+      # the model is loaded and request-ready. Use readiness as the
+      # healthy signal so we don't mark it "healthy" before it can
+      # accept synthesis requests.
+      test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      # First boot pulls Voxtral-4B (~8 GB BF16) + warms vLLM. Generous.
+      start_period: 600s
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=Voxtral TTS
+      - homepage.icon=mdi-translate
+      - homepage.description=Mistral 4B multilingual streaming TTS — 70 ms latency, voice cloning (irv-ml1)
+      - homepage.href=http://10.100.79.3:${VOXTRAL_PORT}