vh
e0d1c44137
The 2026-09-06 headscale cutover retired irv-ml1's wg0 tunnel IP 10.100.79.3 (now 10.6.110.50). Repointed all LIVE canonical refs to the DNS NAME so the next move can't re-break them: homepage.href/siteMonitor labels across 25 stack composes, load-bearing env defaults (asset-engine INFERENCE_HOST, open-webui AUDIO_TTS_OPENAI_API_BASE_URL, skaldsong SKALDSONG_TTS_BASE_URL, zonos-gateway ZONOS_URL, dia), homepage services.yaml manual cards (Voice Design Studio, IRV-ML1), and servers/irv-ml1/ssh-target. Updated the stale 'WG tunnel' comment to the mesh reality. Left as-is: README curl-examples and .env.example comments (docs), and historical mentions in CLAUDE.md/persistent-memory. NOTE: applying the label repoints to the RUNNING irv-ml1 containers needs a recreate per service (labels read at creation); deployed .env values are separate from these canonical defaults.
87 lines
4.1 KiB
YAML
87 lines
4.1 KiB
YAML
# Voxtral TTS — Mistral AI's 4B open-weight streaming TTS, served via
|
||
# vLLM-Omni (the production serving stack Mistral co-developed for
|
||
# Voxtral). Released March 28, 2026.
|
||
#
|
||
# Why this stack alongside the existing TTS:
|
||
# * 70 ms model latency, 9.7× realtime — fastest non-Kokoro option.
|
||
# * Multilingual-first (EN strong, plus FR, DE, ES, IT, PT, NL, HI).
|
||
# Different from CosyVoice's Chinese-leaning balance.
|
||
# * 68.4% blind A/B win rate vs ElevenLabs Flash v2.5 in cloning.
|
||
# * vLLM-Omni serving = continuous batching + paged attention — the
|
||
# same mechanism that gave qwen3.6 its speed on llama-swap.
|
||
#
|
||
# LICENSE: CC BY-NC. Personal / research use only. Don't ship Voxtral
|
||
# output in any commercial product without re-licensing from Mistral.
|
||
#
|
||
# All tunables live in .env — edit that, not this file.
|
||
|
||
services:
|
||
voxtral:
|
||
# vLLM-Omni image — Mistral's official partnership for Voxtral
|
||
# serving. Version-pinned via .env.
|
||
image: vllm/vllm-omni:${VOXTRAL_VLLM_TAG}
|
||
container_name: voxtral
|
||
restart: unless-stopped
|
||
runtime: nvidia
|
||
ports:
|
||
- "${VOXTRAL_BIND:-0.0.0.0}:${VOXTRAL_PORT}:8000"
|
||
environment:
|
||
- NVIDIA_VISIBLE_DEVICES=${VOXTRAL_GPU_DEVICES:-1}
|
||
- HF_HOME=/root/.cache/huggingface
|
||
# HF_TOKEN required to dodge 429 rate limits on the model
|
||
# download (HF aggressively throttles unauthenticated IPs that
|
||
# pull large repos repeatedly). Set in .env — see .env.example.
|
||
- HF_TOKEN=${VOXTRAL_HF_TOKEN}
|
||
# vLLM serving args — see https://docs.vllm.ai for full list.
|
||
# We override the default model via cmd args below.
|
||
volumes:
|
||
- ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
|
||
- ${VOXTRAL_VOICES_DIR}:/voices:ro
|
||
# Patched stage config — bundled YAML hardcodes
|
||
# gpu_memory_utilization: 0.8 on the language_model stage which
|
||
# OOMs anywhere we have other models resident on the same GPU.
|
||
# Mount our own copy at a custom path; the --stage-configs-path
|
||
# flag below points at it.
|
||
- /opt/docker/conf/voxtral/voxtral_tts.yaml:/etc/voxtral/voxtral_tts.yaml:ro
|
||
# vllm/vllm-omni image has no default ENTRYPOINT or CMD — the
|
||
# container init expected --model=... as argv[0]. Set entrypoint
|
||
# to `vllm serve` (the standard CLI) and pass model as positional
|
||
# + tuning flags via command.
|
||
# Voxtral TTS is a STAGE-BASED pipeline (language_model →
|
||
# acoustic_transformer → audio output), not a flat
|
||
# MistralForCausalLM. vllm-omni's `--omni` mode + a stage config
|
||
# YAML drives this. The standard `vllm serve` errors with
|
||
# "no module named 'acoustic_transformer'" because it tries to
|
||
# load Voxtral as a vanilla Mistral causal LM.
|
||
#
|
||
# Pattern lifted from /workspace/vllm-omni/examples/online_serving/
|
||
# qwen3_tts/run_server.sh (closest analog example in the image).
|
||
# Stage config path is relative to WORKDIR=/workspace/vllm-omni.
|
||
entrypoint: ["vllm-omni", "serve"]
|
||
command:
|
||
- "${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}"
|
||
- "--stage-configs-path=/etc/voxtral/voxtral_tts.yaml"
|
||
- "--host=0.0.0.0"
|
||
- "--port=8000"
|
||
- "--gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.45}"
|
||
- "--trust-remote-code"
|
||
- "--omni"
|
||
healthcheck:
|
||
# vLLM-Omni exposes /health for liveness + /v1/models for readiness.
|
||
# /health 200 means the server's listening; /v1/models 200 means
|
||
# the model is loaded and request-ready. Use readiness as the
|
||
# healthy signal so we don't mark it "healthy" before it can
|
||
# accept synthesis requests.
|
||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/v1/models', timeout=5).status==200 else 1)\""]
|
||
interval: 30s
|
||
timeout: 10s
|
||
retries: 3
|
||
# First boot pulls Voxtral-4B (~8 GB BF16) + warms vLLM. Generous.
|
||
start_period: 600s
|
||
labels:
|
||
- homepage.group=AI Systems
|
||
- homepage.name=Voxtral TTS
|
||
- homepage.icon=mdi-translate
|
||
- homepage.description=Mistral 4B multilingual streaming TTS — 70 ms latency, voice cloning (irv-ml1)
|
||
- homepage.href=http://irv-ml1.nh3.internal:${VOXTRAL_PORT}
|