vh
099e1d7418
- chatterbox-fast experimental -> ready: browser audition verified end-to-end (operator confirmed progressive playback "excellent" 2026-06-02). - vibevoice ready -> down: no container running on irv-ml1 (connection refused); catalog status was stale. - voxtral: NOT a stale typo — its stack genuinely claimed :8197, the port now held by the live chatterbox-fast. voxtral is down, so moved IT to :8201 (catalog endpoint + source_url, stacks/voxtral/.env.example + README, host .env) rather than disturb the live service. No live clash existed (voxtral down) but it was a latent deploy-time collision I introduced by placing chatterbox-fast on 8197. No catalog_version bump (status changes + endpoint correction, additive). Validates against the schema.
53 lines
2.9 KiB
Bash
53 lines
2.9 KiB
Bash
# Voxtral TTS stack tunables. Copy to `.env` on irv-ml1 before
|
|
# deploying.
|
|
|
|
# ── image pin ────────────────────────────────────────────────────────
|
|
# vLLM-Omni image tag (Mistral's partner serving stack for Voxtral).
|
|
# Pin a specific version — vllm/vllm-omni does NOT publish `latest`;
|
|
# `:latest` 404s with "manifest unknown". v0.18.0 was released
|
|
# 2026-03-29, one day after the Voxtral 4B TTS release, and is the
|
|
# first vLLM-Omni cut with Voxtral support.
|
|
VOXTRAL_VLLM_TAG=v0.18.0
|
|
|
|
# Voxtral model on Hugging Face. The 4B variant is the only released
|
|
# checkpoint as of 2026-04. Default BF16 weights are ~8 GB.
|
|
VOXTRAL_MODEL=mistralai/Voxtral-4B-TTS-2603
|
|
|
|
# ── network ──────────────────────────────────────────────────────────
|
|
# Host port (container listens on 8000 internally).
|
|
# Was 8197 — moved to 8201 (2026-06-02): 8197 is taken by the live
|
|
# chatterbox-fast stack. Don't reuse 8197.
|
|
VOXTRAL_PORT=8201
|
|
VOXTRAL_BIND=0.0.0.0
|
|
|
|
# ── runtime / GPU ────────────────────────────────────────────────────
|
|
# GPU pinning. "0" = RTX 3090 (24 GB), "1" = RTX A6000 (48 GB).
|
|
# Pinned to GPU 0 (3090) — the A6000 is fully booked by Fish S2-Pro
|
|
# (~17 GB) + Qwen3-TTS / IndexTTS-2 / VibeVoice slots. Voxtral 4B BF16
|
|
# needs ~10-12 GB practical (model + small KV); the 3090's 24 GB is
|
|
# enough alongside Kokoro (~1 GB) + Chatterbox (~3 GB) + Kyutai (~6 GB).
|
|
VOXTRAL_GPU_DEVICES=0
|
|
|
|
# vLLM GPU memory utilization fraction (0.0-1.0). On the 3090 alongside
|
|
# Kokoro/Chatterbox/Kyutai (~10 GB used), 0.5 = ~12 GB target gives
|
|
# Voxtral enough room for weights + KV. Bump to 0.85 if Voxtral ever
|
|
# moves to a dedicated GPU.
|
|
VOXTRAL_GPU_UTIL=0.5
|
|
|
|
# ── HuggingFace auth ─────────────────────────────────────────────────
|
|
# HF_TOKEN — required to dodge 429 rate limits on Voxtral download.
|
|
# HF aggressively throttles unauthenticated IPs that pull large repos.
|
|
# Generate one at https://huggingface.co/settings/tokens (a read-only
|
|
# token is sufficient). Without this, the first model download fails
|
|
# mid-stream and vLLM aborts engine init.
|
|
VOXTRAL_HF_TOKEN=
|
|
|
|
# ── persistent storage on the host ───────────────────────────────────
|
|
# HF cache — first start pulls the Voxtral checkpoint (~8 GB) into
|
|
# this dir. Persistent across container recreates.
|
|
VOXTRAL_CACHE_DIR=/worktank/voxtral/hf_cache
|
|
|
|
# Reference voices for cloning. Read-only mount inside the container.
|
|
# Drop ~5-15 s WAV / FLAC clips here.
|
|
VOXTRAL_VOICES_DIR=/worktank/voxtral/voices
|