voxtral + kyutai-tts: fix wrong image tag / wrong endpoint paths; fish-s2: env-selectable model variant
Three fixes from the second-wave deploy attempts:
* voxtral: vllm/vllm-omni doesn't publish a `latest` tag — pull
failed with "manifest unknown". Pinned VOXTRAL_VLLM_TAG to v0.18.0
(released 2026-03-29, the day after the Voxtral 4B TTS release —
first cut with Voxtral support).
* kyutai-tts: NillPointer wrapper exposes ONLY /health (root) and
POST /v1/audio/speech. No /v1/models, no /v1/audio/voices —
those return 404. Verified by /openapi.json against the live
container. Compose healthcheck + playbook wait + verify steps
all repointed at the actual paths. POST /v1/audio/speech is now
smoke-tested with a RIFF WAV assertion (same pattern as fish-s2).
* fish-s2: added FISH_S2_MODEL env var so the model variant is
swappable via .env without rebuilding. Both s2-pro (default) and
s1-mini are pre-pulled into the bind-mount; LLAMA_CHECKPOINT_PATH
+ DECODER_CHECKPOINT_PATH now use ${FISH_S2_MODEL:-s2-pro}.
s1-mini was originally gated on fishaudio's HF org (401), but
niobures/OpenAudio-S1 mirrors the same files openly — pulled
from there via a one-shot snapshot_download.
This commit is contained in:
@@ -73,26 +73,32 @@ steps:
|
||||
- name: docker compose up -d
|
||||
shell: cd {{ compose_dir }} && docker compose up -d
|
||||
|
||||
- name: Wait for /v1/models to respond (allow ~10 min for first download + warmup)
|
||||
- name: Wait for /health to respond (allow ~10 min for first download + warmup)
|
||||
shell: |
|
||||
for i in $(seq 1 120); do
|
||||
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/models && exit 0
|
||||
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/health && exit 0
|
||||
sleep 5
|
||||
done
|
||||
exit 1
|
||||
changed_when: "false"
|
||||
|
||||
verify:
|
||||
- name: /v1/models returns valid JSON
|
||||
shell: |
|
||||
curl -sf http://localhost:{{ host_port }}/v1/models \
|
||||
| python3 -c "import json,sys; json.load(sys.stdin)"
|
||||
- name: /health returns 200
|
||||
shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/health
|
||||
changed_when: "false"
|
||||
|
||||
- name: /v1/audio/voices returns valid JSON
|
||||
- name: /v1/audio/speech returns a real WAV (POST with text body)
|
||||
# NillPointer wrapper exposes ONLY /health and POST /v1/audio/speech
|
||||
# (no /v1/models, no /v1/audio/voices). Smoke by POSTing and
|
||||
# asserting a RIFF WAV comes back.
|
||||
shell: |
|
||||
curl -sf http://localhost:{{ host_port }}/v1/audio/voices \
|
||||
| python3 -c "import json,sys; json.load(sys.stdin)"
|
||||
out=$(mktemp --suffix=.wav)
|
||||
curl -sf -X POST http://localhost:{{ host_port }}/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"tts-1.6b-en_fr","input":"Verify."}' \
|
||||
-o "$out" --max-time 30
|
||||
file -b "$out" | grep -q '^RIFF.*WAVE'
|
||||
rm -f "$out"
|
||||
changed_when: "false"
|
||||
|
||||
- name: Container is running
|
||||
|
||||
@@ -34,6 +34,15 @@ FISH_S2_GPU_DEVICES=1
|
||||
# checkpoint.
|
||||
FISH_S2_COMPILE=1
|
||||
|
||||
# Model variant. Both checkpoints are pre-pulled by the deploy
|
||||
# playbook into /worktank/fish-s2/checkpoints/. Swap by editing this
|
||||
# value + `docker compose up -d --force-recreate` (no rebuild needed).
|
||||
# s2-pro — 4B-class, highest quality, ~7-8 s TTFB on a long phrase
|
||||
# s1-mini — lighter, ~3-5× smaller model.pth, expected ~2-3 s TTFB
|
||||
# fishaudio/s1-mini on HF is gated, but niobures/OpenAudio-S1 mirrors
|
||||
# the same files openly — playbook pulls from there.
|
||||
FISH_S2_MODEL=s2-pro
|
||||
|
||||
# ── persistent storage on the host ───────────────────────────────────
|
||||
# Model checkpoints — Fish auto-downloads s2-pro on first run (~9 GB
|
||||
# at BF16) and caches under here. Persistent across container
|
||||
|
||||
@@ -63,6 +63,13 @@ services:
|
||||
# checkpoint.
|
||||
- COMPILE=${FISH_S2_COMPILE:-1}
|
||||
- API_PORT=8080
|
||||
# Model selection. Override Fish's Dockerfile defaults so we can
|
||||
# swap variants via .env without rebuilding. Both checkpoints are
|
||||
# pre-pulled by the deploy playbook into bind-mounted checkpoints/.
|
||||
# s2-pro — 4B class, ~17 GB VRAM, slow but highest quality
|
||||
# s1-mini — lighter, ~3-5× smaller model.pth, much faster
|
||||
- LLAMA_CHECKPOINT_PATH=checkpoints/${FISH_S2_MODEL:-s2-pro}
|
||||
- DECODER_CHECKPOINT_PATH=checkpoints/${FISH_S2_MODEL:-s2-pro}/codec.pth
|
||||
# Hugging Face cache for model weights — first start pulls
|
||||
# fishaudio/s2-pro (~9 GB BF16) into this dir.
|
||||
- HF_HOME=/app/hf_cache
|
||||
|
||||
@@ -39,10 +39,12 @@ services:
|
||||
- ${KYUTAI_TTS_CACHE_DIR}:/app/hf_cache
|
||||
- ${KYUTAI_TTS_VOICES_DIR}:/app/voices:ro
|
||||
healthcheck:
|
||||
# The wrapper exposes /v1/models for OpenAI-compat — same shape
|
||||
# as Voxtral / Qwen3-TTS. Use that as the readiness signal.
|
||||
# 127.0.0.1 explicit to dodge IPv4/IPv6 localhost race.
|
||||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
|
||||
# NillPointer wrapper's actual endpoints (verified at runtime via
|
||||
# /openapi.json): just /health (root) and /v1/audio/speech (POST).
|
||||
# No /v1/models, no /v1/audio/voices — those 404. /health is the
|
||||
# liveness signal. 127.0.0.1 explicit to dodge IPv4/IPv6 localhost
|
||||
# race.
|
||||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5).status==200 else 1)\""]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
@@ -3,9 +3,11 @@
|
||||
|
||||
# ── image pin ────────────────────────────────────────────────────────
|
||||
# vLLM-Omni image tag (Mistral's partner serving stack for Voxtral).
|
||||
# Use a specific version rather than `latest` — vLLM moves fast and
|
||||
# Voxtral has version-specific compatibility.
|
||||
VOXTRAL_VLLM_TAG=latest
|
||||
# Pin a specific version — vllm/vllm-omni does NOT publish `latest`;
|
||||
# `:latest` 404s with "manifest unknown". v0.18.0 was released
|
||||
# 2026-03-29, one day after the Voxtral 4B TTS release, and is the
|
||||
# first vLLM-Omni cut with Voxtral support.
|
||||
VOXTRAL_VLLM_TAG=v0.18.0
|
||||
|
||||
# Voxtral model on Hugging Face. The 4B variant is the only released
|
||||
# checkpoint as of 2026-04. Default BF16 weights are ~8 GB.
|
||||
|
||||
Reference in New Issue
Block a user