vh
131d746c92
Three fixes from the second-wave deploy attempts:
* voxtral: vllm/vllm-omni doesn't publish a `latest` tag — pull
failed with "manifest unknown". Pinned VOXTRAL_VLLM_TAG to v0.18.0
(released 2026-03-29, the day after the Voxtral 4B TTS release —
first cut with Voxtral support).
* kyutai-tts: NillPointer wrapper exposes ONLY /health (root) and
POST /v1/audio/speech. No /v1/models, no /v1/audio/voices —
those return 404. Verified by /openapi.json against the live
container. Compose healthcheck + playbook wait + verify steps
all repointed at the actual paths. POST /v1/audio/speech is now
smoke-tested with a RIFF WAV assertion (same pattern as fish-s2).
* fish-s2: added FISH_S2_MODEL env var so the model variant is
swappable via .env without rebuilding. Both s2-pro (default) and
s1-mini are pre-pulled into the bind-mount; LLAMA_CHECKPOINT_PATH
+ DECODER_CHECKPOINT_PATH now use ${FISH_S2_MODEL:-s2-pro}.
s1-mini was originally gated on fishaudio's HF org (401), but
niobures/OpenAudio-S1 mirrors the same files openly — pulled
from there via a one-shot snapshot_download.
41 lines
2.1 KiB
Bash
41 lines
2.1 KiB
Bash
# Voxtral TTS stack tunables. Copy to `.env` on irv-ml1 before
|
|
# deploying.
|
|
|
|
# ── image pin ────────────────────────────────────────────────────────
|
|
# vLLM-Omni image tag (Mistral's partner serving stack for Voxtral).
|
|
# Pin a specific version — vllm/vllm-omni does NOT publish `latest`;
|
|
# `:latest` 404s with "manifest unknown". v0.18.0 was released
|
|
# 2026-03-29, one day after the Voxtral 4B TTS release, and is the
|
|
# first vLLM-Omni cut with Voxtral support.
|
|
VOXTRAL_VLLM_TAG=v0.18.0
|
|
|
|
# Voxtral model on Hugging Face. The 4B variant is the only released
|
|
# checkpoint as of 2026-04. Default BF16 weights are ~8 GB.
|
|
VOXTRAL_MODEL=mistralai/Voxtral-4B-TTS-2603
|
|
|
|
# ── network ──────────────────────────────────────────────────────────
|
|
# Host port (container listens on 8000 internally).
|
|
VOXTRAL_PORT=8197
|
|
VOXTRAL_BIND=0.0.0.0
|
|
|
|
# ── runtime / GPU ────────────────────────────────────────────────────
|
|
# GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB).
|
|
# Voxtral 4B BF16 needs ~16 GB practical (model + KV + activation).
|
|
# Pinned to A6000 by default for headroom. The 3090 fits but is tight
|
|
# for long streaming sessions.
|
|
VOXTRAL_GPU_DEVICES=1
|
|
|
|
# vLLM GPU memory utilization fraction (0.0-1.0). 0.85 = leave 15%
|
|
# headroom for other processes / KV cache spikes. Lower if running
|
|
# alongside other GPU workloads on the same device.
|
|
VOXTRAL_GPU_UTIL=0.85
|
|
|
|
# ── persistent storage on the host ───────────────────────────────────
|
|
# HF cache — first start pulls the Voxtral checkpoint (~8 GB) into
|
|
# this dir. Persistent across container recreates.
|
|
VOXTRAL_CACHE_DIR=/worktank/voxtral/hf_cache
|
|
|
|
# Reference voices for cloning. Read-only mount inside the container.
|
|
# Drop ~5-15 s WAV / FLAC clips here.
|
|
VOXTRAL_VOICES_DIR=/worktank/voxtral/voices
|