stacks/{fish-s2,voxtral,kyutai-tts}: three new TTS deploys for irv-ml1 quality A/B
Adds the three premier 2026 TTS releases we missed during the original
fleet build-out (early April), all licensed for self-host:
* Fish Audio S2-Pro (port 8195, GPU 1 / A6000) — released 2026-03-09.
4B dual-AR (Slow + Fast) trained on 10M+ hours / 80+ languages.
Headline: 15,000+ paralinguistic / emotion tags via natural language
([laugh] [whispers] [super happy] etc.) — a step-function over
Chatterbox Turbo's 9 fixed tags. 91.61% paralinguistic win rate on
EmergentTTS-Eval. ~150 ms streaming TTFB, voice cloning, MIT-style
open. ~17 GB VRAM.
* Voxtral TTS (port 8197, GPU 1 / A6000) — Mistral, released 2026-03-28.
4B open-weight, 70 ms model latency, 9.7× realtime. 68.4% blind A/B
win rate vs ElevenLabs Flash v2.5 in cloning. 8 languages
(EN/FR/DE/ES/IT/PT/NL/HI). Served via vLLM-Omni (Mistral's partner
serving stack) — published Docker image, no local build. ~16 GB VRAM.
CC BY-NC license — personal/research use only; flagged in README.
* Kyutai TTS (port 8198, GPU 0 / 3090) — kyutai/tts-1.6b-en_fr.
Trained on 2.5M hours from the Moshi/Mimi team. Claimed 220 ms in
solo setup, 32 simultaneous streams under 350 ms on L40. Kyutai's
official deploy is Rust + websockets only; using NillPointer's
community OpenAI-compat wrapper to bridge to /v1/audio/speech so
it slots into the same bench harness. ~4-6 GB VRAM.
Each stack: compose.yaml (build context, env, volumes, healthcheck,
homepage label), .env.example (all tunables documented), README.md
(why it exists, headline numbers, API, deploy + hardware notes).
Playbooks at playbooks/deploy-{fish-s2,voxtral,kyutai-tts}.yaml are
idempotent in the same shape as the existing deploy-vibevoice /
deploy-chatterbox playbooks.
Port allocations on irv-ml1 after this lands: 8188 ComfyUI, 8190
CosyVoice, 8191 Qwen3-TTS, 8192 IndexTTS-2, 8193 Kokoro, 8194
VibeVoice, 8195 Fish, 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai,
8765 Parakeet ASR.
This commit is contained in:
@@ -0,0 +1,99 @@
|
||||
# Deploy Fish Audio S2-Pro (richest paralinguistic open-source TTS) to
|
||||
# irv-ml1.
|
||||
#
|
||||
# Builds the image locally from fishaudio/fish-speech via docker buildx
|
||||
# git URL context. ~10-15 min cold build (CUDA 12.x + torch + flash-attn
|
||||
# + Fish's training/inference deps). First start downloads s2-pro
|
||||
# (~9 GB BF16) into the bind-mounted HF cache. Generous /v1/health
|
||||
# wait deadline accommodates both.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/elway irv-ml1 --playbook playbooks/deploy-fish-s2.yaml
|
||||
#
|
||||
# Idempotent — every step is creates-/when-gated; rerun is safe.
|
||||
|
||||
vars:
|
||||
compose_dir: /opt/docker/compose/fish-s2
|
||||
references_dir: /worktank/fish-s2/references
|
||||
checkpoints_dir: /worktank/fish-s2/checkpoints
|
||||
cache_dir: /worktank/fish-s2/hf_cache
|
||||
host_port: "8195"
|
||||
|
||||
steps:
|
||||
# ── host-side dirs ──────────────────────────────────────────────────
|
||||
|
||||
- name: Ensure /worktank/fish-s2 root exists (one-time, sudo)
|
||||
shell: mkdir -p /worktank/fish-s2
|
||||
sudo: true
|
||||
creates: /worktank/fish-s2
|
||||
|
||||
- name: Chown /worktank/fish-s2 to lkraven
|
||||
shell: chown -R lkraven:lkraven /worktank/fish-s2
|
||||
sudo: true
|
||||
when: "[ \"$(stat -c %U /worktank/fish-s2)\" != \"lkraven\" ]"
|
||||
|
||||
- name: Ensure references dir exists
|
||||
shell: mkdir -p {{ references_dir }}
|
||||
creates: "{{ references_dir }}"
|
||||
|
||||
- name: Ensure checkpoints dir exists
|
||||
shell: mkdir -p {{ checkpoints_dir }}
|
||||
creates: "{{ checkpoints_dir }}"
|
||||
|
||||
- name: Ensure HF cache dir exists
|
||||
shell: mkdir -p {{ cache_dir }}
|
||||
creates: "{{ cache_dir }}"
|
||||
|
||||
- name: Ensure compose dir exists
|
||||
shell: mkdir -p {{ compose_dir }}
|
||||
creates: "{{ compose_dir }}"
|
||||
|
||||
# ── deploy compose + env ────────────────────────────────────────────
|
||||
|
||||
- name: Upload compose.yaml
|
||||
upload:
|
||||
src: stacks/fish-s2/compose.yaml
|
||||
dest: "{{ compose_dir }}/compose.yaml"
|
||||
mode: "0644"
|
||||
|
||||
- name: Seed .env from template (only if absent)
|
||||
upload:
|
||||
src: stacks/fish-s2/.env.example
|
||||
dest: "{{ compose_dir }}/.env"
|
||||
mode: "0644"
|
||||
when: "[ ! -f {{ compose_dir }}/.env ]"
|
||||
|
||||
# ── build + bring up ────────────────────────────────────────────────
|
||||
|
||||
- name: docker compose build (~10-15 min first time; cached after)
|
||||
shell: |
|
||||
set -o pipefail
|
||||
cd {{ compose_dir }} && docker compose build 2>&1 \
|
||||
| grep -vE '^#[0-9]+ |^ => |^=> |Collecting|Downloading|Requirement|Using cached|Installing collected|Successfully (installed|built)|━'
|
||||
|
||||
- name: docker compose up -d
|
||||
shell: cd {{ compose_dir }} && docker compose up -d
|
||||
|
||||
- name: Wait for /v1/health to respond (allow ~15 min for first model download + warmup)
|
||||
shell: |
|
||||
for i in $(seq 1 180); do
|
||||
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/health && exit 0
|
||||
sleep 5
|
||||
done
|
||||
exit 1
|
||||
changed_when: "false"
|
||||
|
||||
verify:
|
||||
- name: /v1/health returns 200
|
||||
shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/v1/health
|
||||
changed_when: "false"
|
||||
|
||||
- name: /v1/audio/voices returns valid JSON
|
||||
shell: |
|
||||
curl -sf http://localhost:{{ host_port }}/v1/audio/voices \
|
||||
| python3 -c "import json,sys; json.load(sys.stdin)"
|
||||
changed_when: "false"
|
||||
|
||||
- name: Container is running
|
||||
shell: docker inspect fish-s2 --format '{{.State.Status}}' | grep -q running
|
||||
changed_when: "false"
|
||||
@@ -0,0 +1,100 @@
|
||||
# Deploy Kyutai TTS (1.6B EN/FR streaming, 220 ms claimed latency) to
|
||||
# irv-ml1.
|
||||
#
|
||||
# Builds the image locally from NillPointer/Kyutai-TTS-Server via
|
||||
# docker buildx git URL context. ~5-8 min cold build (CUDA + torch +
|
||||
# moshi + Kyutai's Mimi codec deps). First start downloads
|
||||
# kyutai/tts-1.6b-en_fr (~3-6 GB) into the bind-mounted HF cache.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/elway irv-ml1 --playbook playbooks/deploy-kyutai-tts.yaml
|
||||
#
|
||||
# Note: Kyutai's official deploy is Rust + websockets only. This stack
|
||||
# uses the NillPointer community wrapper to bridge to OpenAI-compat
|
||||
# HTTP — adds Python overhead on the request path, so measured TTFB
|
||||
# will be higher than the bare-Rust 220 ms claim. See README.
|
||||
#
|
||||
# Idempotent — every step is creates-/when-gated; rerun is safe.
|
||||
|
||||
vars:
|
||||
compose_dir: /opt/docker/compose/kyutai-tts
|
||||
voices_dir: /worktank/kyutai-tts/voices
|
||||
cache_dir: /worktank/kyutai-tts/hf_cache
|
||||
host_port: "8198"
|
||||
|
||||
steps:
|
||||
# ── host-side dirs ──────────────────────────────────────────────────
|
||||
|
||||
- name: Ensure /worktank/kyutai-tts root exists (one-time, sudo)
|
||||
shell: mkdir -p /worktank/kyutai-tts
|
||||
sudo: true
|
||||
creates: /worktank/kyutai-tts
|
||||
|
||||
- name: Chown /worktank/kyutai-tts to lkraven
|
||||
shell: chown -R lkraven:lkraven /worktank/kyutai-tts
|
||||
sudo: true
|
||||
when: "[ \"$(stat -c %U /worktank/kyutai-tts)\" != \"lkraven\" ]"
|
||||
|
||||
- name: Ensure voices dir exists
|
||||
shell: mkdir -p {{ voices_dir }}
|
||||
creates: "{{ voices_dir }}"
|
||||
|
||||
- name: Ensure HF cache dir exists
|
||||
shell: mkdir -p {{ cache_dir }}
|
||||
creates: "{{ cache_dir }}"
|
||||
|
||||
- name: Ensure compose dir exists
|
||||
shell: mkdir -p {{ compose_dir }}
|
||||
creates: "{{ compose_dir }}"
|
||||
|
||||
# ── deploy compose + env ────────────────────────────────────────────
|
||||
|
||||
- name: Upload compose.yaml
|
||||
upload:
|
||||
src: stacks/kyutai-tts/compose.yaml
|
||||
dest: "{{ compose_dir }}/compose.yaml"
|
||||
mode: "0644"
|
||||
|
||||
- name: Seed .env from template (only if absent)
|
||||
upload:
|
||||
src: stacks/kyutai-tts/.env.example
|
||||
dest: "{{ compose_dir }}/.env"
|
||||
mode: "0644"
|
||||
when: "[ ! -f {{ compose_dir }}/.env ]"
|
||||
|
||||
# ── build + bring up ────────────────────────────────────────────────
|
||||
|
||||
- name: docker compose build (~5-8 min first time; cached after)
|
||||
shell: |
|
||||
set -o pipefail
|
||||
cd {{ compose_dir }} && docker compose build 2>&1 \
|
||||
| grep -vE '^#[0-9]+ |^ => |^=> |Collecting|Downloading|Requirement|Using cached|Installing collected|Successfully (installed|built)|━'
|
||||
|
||||
- name: docker compose up -d
|
||||
shell: cd {{ compose_dir }} && docker compose up -d
|
||||
|
||||
- name: Wait for /v1/models to respond (allow ~10 min for first download + warmup)
|
||||
shell: |
|
||||
for i in $(seq 1 120); do
|
||||
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/models && exit 0
|
||||
sleep 5
|
||||
done
|
||||
exit 1
|
||||
changed_when: "false"
|
||||
|
||||
verify:
|
||||
- name: /v1/models returns valid JSON
|
||||
shell: |
|
||||
curl -sf http://localhost:{{ host_port }}/v1/models \
|
||||
| python3 -c "import json,sys; json.load(sys.stdin)"
|
||||
changed_when: "false"
|
||||
|
||||
- name: /v1/audio/voices returns valid JSON
|
||||
shell: |
|
||||
curl -sf http://localhost:{{ host_port }}/v1/audio/voices \
|
||||
| python3 -c "import json,sys; json.load(sys.stdin)"
|
||||
changed_when: "false"
|
||||
|
||||
- name: Container is running
|
||||
shell: docker inspect kyutai-tts --format '{{.State.Status}}' | grep -q running
|
||||
changed_when: "false"
|
||||
@@ -0,0 +1,89 @@
|
||||
# Deploy Voxtral TTS (Mistral 4B multilingual streaming) to irv-ml1.
|
||||
#
|
||||
# Pulls the official vllm/vllm-omni image (Mistral's partner serving
|
||||
# stack) — no local build. ~3-5 min cold pull on first deploy. First
|
||||
# container start downloads Voxtral-4B-TTS-2603 (~8 GB BF16) into the
|
||||
# bind-mounted HF cache. Generous /v1/models wait deadline accommodates
|
||||
# the first model pull + vLLM warmup.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/elway irv-ml1 --playbook playbooks/deploy-voxtral.yaml
|
||||
#
|
||||
# License caveat: Voxtral is CC BY-NC. Personal / research use only.
|
||||
# See stacks/voxtral/README.md.
|
||||
#
|
||||
# Idempotent — every step is creates-/when-gated; rerun is safe.
|
||||
|
||||
vars:
|
||||
compose_dir: /opt/docker/compose/voxtral
|
||||
voices_dir: /worktank/voxtral/voices
|
||||
cache_dir: /worktank/voxtral/hf_cache
|
||||
host_port: "8197"
|
||||
|
||||
steps:
|
||||
# ── host-side dirs ──────────────────────────────────────────────────
|
||||
|
||||
- name: Ensure /worktank/voxtral root exists (one-time, sudo)
|
||||
shell: mkdir -p /worktank/voxtral
|
||||
sudo: true
|
||||
creates: /worktank/voxtral
|
||||
|
||||
- name: Chown /worktank/voxtral to lkraven
|
||||
shell: chown -R lkraven:lkraven /worktank/voxtral
|
||||
sudo: true
|
||||
when: "[ \"$(stat -c %U /worktank/voxtral)\" != \"lkraven\" ]"
|
||||
|
||||
- name: Ensure voices dir exists
|
||||
shell: mkdir -p {{ voices_dir }}
|
||||
creates: "{{ voices_dir }}"
|
||||
|
||||
- name: Ensure HF cache dir exists
|
||||
shell: mkdir -p {{ cache_dir }}
|
||||
creates: "{{ cache_dir }}"
|
||||
|
||||
- name: Ensure compose dir exists
|
||||
shell: mkdir -p {{ compose_dir }}
|
||||
creates: "{{ compose_dir }}"
|
||||
|
||||
# ── deploy compose + env ────────────────────────────────────────────
|
||||
|
||||
- name: Upload compose.yaml
|
||||
upload:
|
||||
src: stacks/voxtral/compose.yaml
|
||||
dest: "{{ compose_dir }}/compose.yaml"
|
||||
mode: "0644"
|
||||
|
||||
- name: Seed .env from template (only if absent)
|
||||
upload:
|
||||
src: stacks/voxtral/.env.example
|
||||
dest: "{{ compose_dir }}/.env"
|
||||
mode: "0644"
|
||||
when: "[ ! -f {{ compose_dir }}/.env ]"
|
||||
|
||||
# ── pull + bring up ─────────────────────────────────────────────────
|
||||
|
||||
- name: docker compose pull (~3-5 min cold)
|
||||
shell: cd {{ compose_dir }} && docker compose pull 2>&1 | tail -20
|
||||
|
||||
- name: docker compose up -d
|
||||
shell: cd {{ compose_dir }} && docker compose up -d
|
||||
|
||||
- name: Wait for /v1/models to report model loaded (allow ~10 min for first pull + warmup)
|
||||
shell: |
|
||||
for i in $(seq 1 120); do
|
||||
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/models && exit 0
|
||||
sleep 5
|
||||
done
|
||||
exit 1
|
||||
changed_when: "false"
|
||||
|
||||
verify:
|
||||
- name: /v1/models reports loaded model
|
||||
shell: |
|
||||
curl -sf http://localhost:{{ host_port }}/v1/models \
|
||||
| python3 -c "import json,sys; d=json.load(sys.stdin); assert any('Voxtral' in m.get('id','') for m in d.get('data',[]))"
|
||||
changed_when: "false"
|
||||
|
||||
- name: Container is running
|
||||
shell: docker inspect voxtral --format '{{.State.Status}}' | grep -q running
|
||||
changed_when: "false"
|
||||
Reference in New Issue
Block a user