voxtral + kyutai-tts: fix wrong image tag / wrong endpoint paths; fish-s2: env-selectable model variant

Three fixes from the second-wave deploy attempts:

* voxtral: vllm/vllm-omni doesn't publish a `latest` tag — pull
  failed with "manifest unknown". Pinned VOXTRAL_VLLM_TAG to v0.18.0
  (released 2026-03-29, the day after the Voxtral 4B TTS release —
  first cut with Voxtral support).

* kyutai-tts: NillPointer wrapper exposes ONLY /health (root) and
  POST /v1/audio/speech. No /v1/models, no /v1/audio/voices —
  those return 404. Verified by /openapi.json against the live
  container. Compose healthcheck + playbook wait + verify steps
  all repointed at the actual paths. POST /v1/audio/speech is now
  smoke-tested with a RIFF WAV assertion (same pattern as fish-s2).

* fish-s2: added FISH_S2_MODEL env var so the model variant is
  swappable via .env without rebuilding. Both s2-pro (default) and
  s1-mini are pre-pulled into the bind-mount; LLAMA_CHECKPOINT_PATH
  + DECODER_CHECKPOINT_PATH now use ${FISH_S2_MODEL:-s2-pro}.
  s1-mini was originally gated on fishaudio's HF org (401), but
  niobures/OpenAudio-S1 mirrors the same files openly — pulled
  from there via a one-shot snapshot_download.
This commit is contained in:
vh
2026-04-27 23:54:59 -07:00
parent 01c1ae2605
commit 131d746c92
5 changed files with 42 additions and 16 deletions
+15 -9
View File
@@ -73,26 +73,32 @@ steps:
- name: docker compose up -d
shell: cd {{ compose_dir }} && docker compose up -d
- name: Wait for /v1/models to respond (allow ~10 min for first download + warmup)
- name: Wait for /health to respond (allow ~10 min for first download + warmup)
shell: |
for i in $(seq 1 120); do
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/models && exit 0
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/health && exit 0
sleep 5
done
exit 1
changed_when: "false"
verify:
- name: /v1/models returns valid JSON
shell: |
curl -sf http://localhost:{{ host_port }}/v1/models \
| python3 -c "import json,sys; json.load(sys.stdin)"
- name: /health returns 200
shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/health
changed_when: "false"
- name: /v1/audio/voices returns valid JSON
- name: /v1/audio/speech returns a real WAV (POST with text body)
# NillPointer wrapper exposes ONLY /health and POST /v1/audio/speech
# (no /v1/models, no /v1/audio/voices). Smoke by POSTing and
# asserting a RIFF WAV comes back.
shell: |
curl -sf http://localhost:{{ host_port }}/v1/audio/voices \
| python3 -c "import json,sys; json.load(sys.stdin)"
out=$(mktemp --suffix=.wav)
curl -sf -X POST http://localhost:{{ host_port }}/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{"model":"tts-1.6b-en_fr","input":"Verify."}' \
-o "$out" --max-time 30
file -b "$out" | grep -q '^RIFF.*WAVE'
rm -f "$out"
changed_when: "false"
- name: Container is running
+9
View File
@@ -34,6 +34,15 @@ FISH_S2_GPU_DEVICES=1
# checkpoint.
FISH_S2_COMPILE=1
# Model variant. Both checkpoints are pre-pulled by the deploy
# playbook into /worktank/fish-s2/checkpoints/. Swap by editing this
# value + `docker compose up -d --force-recreate` (no rebuild needed).
# s2-pro — 4B-class, highest quality, ~7-8 s TTFB on a long phrase
# s1-mini — lighter, ~3-5× smaller model.pth, expected ~2-3 s TTFB
# fishaudio/s1-mini on HF is gated, but niobures/OpenAudio-S1 mirrors
# the same files openly — playbook pulls from there.
FISH_S2_MODEL=s2-pro
# ── persistent storage on the host ───────────────────────────────────
# Model checkpoints — Fish auto-downloads s2-pro on first run (~9 GB
# at BF16) and caches under here. Persistent across container
+7
View File
@@ -63,6 +63,13 @@ services:
# checkpoint.
- COMPILE=${FISH_S2_COMPILE:-1}
- API_PORT=8080
# Model selection. Override Fish's Dockerfile defaults so we can
# swap variants via .env without rebuilding. Both checkpoints are
# pre-pulled by the deploy playbook into bind-mounted checkpoints/.
# s2-pro — 4B class, ~17 GB VRAM, slow but highest quality
# s1-mini — lighter, ~3-5× smaller model.pth, much faster
- LLAMA_CHECKPOINT_PATH=checkpoints/${FISH_S2_MODEL:-s2-pro}
- DECODER_CHECKPOINT_PATH=checkpoints/${FISH_S2_MODEL:-s2-pro}/codec.pth
# Hugging Face cache for model weights — first start pulls
# fishaudio/s2-pro (~9 GB BF16) into this dir.
- HF_HOME=/app/hf_cache
+6 -4
View File
@@ -39,10 +39,12 @@ services:
- ${KYUTAI_TTS_CACHE_DIR}:/app/hf_cache
- ${KYUTAI_TTS_VOICES_DIR}:/app/voices:ro
healthcheck:
# The wrapper exposes /v1/models for OpenAI-compat — same shape
# as Voxtral / Qwen3-TTS. Use that as the readiness signal.
# 127.0.0.1 explicit to dodge IPv4/IPv6 localhost race.
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
# NillPointer wrapper's actual endpoints (verified at runtime via
# /openapi.json): just /health (root) and /v1/audio/speech (POST).
# No /v1/models, no /v1/audio/voices — those 404. /health is the
# liveness signal. 127.0.0.1 explicit to dodge IPv4/IPv6 localhost
# race.
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5).status==200 else 1)\""]
interval: 30s
timeout: 10s
retries: 3
+5 -3
View File
@@ -3,9 +3,11 @@
# ── image pin ────────────────────────────────────────────────────────
# vLLM-Omni image tag (Mistral's partner serving stack for Voxtral).
# Use a specific version rather than `latest` — vLLM moves fast and
# Voxtral has version-specific compatibility.
VOXTRAL_VLLM_TAG=latest
# Pin a specific version — vllm/vllm-omni does NOT publish `latest`;
# `:latest` 404s with "manifest unknown". v0.18.0 was released
# 2026-03-29, one day after the Voxtral 4B TTS release, and is the
# first vLLM-Omni cut with Voxtral support.
VOXTRAL_VLLM_TAG=v0.18.0
# Voxtral model on Hugging Face. The 4B variant is the only released
# checkpoint as of 2026-04. Default BF16 weights are ~8 GB.