# Voxtral TTS stack tunables. Copy to `.env` on irv-ml1 before # deploying. # ── image pin ──────────────────────────────────────────────────────── # vLLM-Omni image tag (Mistral's partner serving stack for Voxtral). # Pin a specific version — vllm/vllm-omni does NOT publish `latest`; # `:latest` 404s with "manifest unknown". v0.18.0 was released # 2026-03-29, one day after the Voxtral 4B TTS release, and is the # first vLLM-Omni cut with Voxtral support. VOXTRAL_VLLM_TAG=v0.18.0 # Voxtral model on Hugging Face. The 4B variant is the only released # checkpoint as of 2026-04. Default BF16 weights are ~8 GB. VOXTRAL_MODEL=mistralai/Voxtral-4B-TTS-2603 # ── network ────────────────────────────────────────────────────────── # Host port (container listens on 8000 internally). # Was 8197 — moved to 8201 (2026-06-02): 8197 is taken by the live # chatterbox-fast stack. Don't reuse 8197. VOXTRAL_PORT=8201 VOXTRAL_BIND=0.0.0.0 # ── runtime / GPU ──────────────────────────────────────────────────── # GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB). # Pinned to GPU 0 (3090) — the A6000 is fully booked by Fish S2-Pro # (~17 GB) + Qwen3-TTS / IndexTTS-2 / VibeVoice slots. Voxtral 4B BF16 # needs ~10-12 GB practical (model + small KV); the 3090's 24 GB is # enough alongside Kokoro (~1 GB) + Chatterbox (~3 GB) + Kyutai (~6 GB). VOXTRAL_GPU_DEVICES=0 # vLLM GPU memory utilization fraction (0.0-1.0). On the 3090 alongside # Kokoro/Chatterbox/Kyutai (~10 GB used), 0.5 = ~12 GB target gives # Voxtral enough room for weights + KV. Bump to 0.85 if Voxtral ever # moves to a dedicated GPU. VOXTRAL_GPU_UTIL=0.5 # ── HuggingFace auth ───────────────────────────────────────────────── # HF_TOKEN — required to dodge 429 rate limits on Voxtral download. # HF aggressively throttles unauthenticated IPs that pull large repos. # Generate one at https://huggingface.co/settings/tokens (a read-only # token is sufficient). Without this, the first model download fails # mid-stream and vLLM aborts engine init. VOXTRAL_HF_TOKEN= # ── persistent storage on the host ─────────────────────────────────── # HF cache — first start pulls the Voxtral checkpoint (~8 GB) into # this dir. Persistent across container recreates. VOXTRAL_CACHE_DIR=/worktank/voxtral/hf_cache # Reference voices for cloning. Read-only mount inside the container. # Drop ~5-15 s WAV / FLAC clips here. VOXTRAL_VOICES_DIR=/worktank/voxtral/voices