Files

T

vh 7875382aed qwen3-tts: add stack + deploy playbook for irv-ml1

Alibaba's open-weight TTS (Apache 2.0, Jan 2026), deployed via
groxaxo/Qwen3-TTS-Openai-Fastapi wrapper. Built locally from a
pinned git SHA via docker buildx's git context — no source
vendored. 1.7B flagship model by default; 0.6B available via
QWEN3_TTS_MODEL env override.

Why we need a second TTS stack: cosyvoice 3 emits Chinese phonemes
for English content per upstream FunAudioLLM/CosyVoice#1790
(unfixed). Qwen3-TTS is from the same Alibaba team but with
English first-class in the checkpoint — 10 languages, 97 ms
streaming TTFB, instruction-driven emotion. Coexists with cosyvoice
on irv-ml1 (port 8191; cosyvoice keeps 8190).

Voice cloning shape DIFFERS from cosyvoice: profile-based, not
voice-id. Profiles live under voice_library/profiles/<name>/ and
are referenced as voice="clone:<name>".

Path layout: /worktank/qwen3-tts/{cache,voices}/, with cache excluded
from restic (regenerable from HF Hub) and voices included (cloned
profiles need original reference audio to recreate).

playbooks/deploy-qwen3-tts.yaml: 10 steps + 5 verify, idempotent;
the wait step polls /health for up to ~10 min to absorb first-run
model download.

Stack only — restic profile update for /worktank/qwen3-tts/voices/
to follow when this is empirically validated against the GLaDOS
voice (the "did Qwen inherit the Chinese-bias bug?" question).

2026-04-24 16:58:19 -07:00

3.7 KiB

Raw Blame History

Qwen3-TTS

Alibaba's open-weight TTS (Apache 2.0, released Jan 2026), deployed via the groxaxo/Qwen3-TTS-Openai-Fastapi OpenAI-compatible wrapper.

Server: irv-ml1 (Irvine, WireGuard-only) Port: 8191 (container 8880) GPUs: both exposed (NVIDIA_VISIBLE_DEVICES=all); 1.7B model fits on either the RTX 3090 (24 GB) or A6000 (48 GB) with headroom Image: local/qwen3-tts:v1 — built locally from a pinned git SHA of the wrapper repo via docker buildx's git URL context

Why this stack alongside cosyvoice

CosyVoice 3 (the other stack on this host) emits Chinese-flavored phonemes when given English content. Confirmed against upstream issue FunAudioLLM/CosyVoice#1790 — closed without a fix as of 2026-04-14.

Qwen3-TTS is from the same Alibaba team but built English-first into the checkpoint: 10 languages, 97 ms streaming TTFB, instruction-driven emotional expression, voice cloning. It's the better choice for any English-narration use; keep CosyVoice 3 around for Chinese / dialect work where it shines.

API endpoints

Method + path	Purpose
`POST /v1/audio/speech`	OpenAI-compatible TTS (drop-in for OpenAI clients)
`GET /v1/voices`	List cloned voice profiles
`GET /v1/models`	List available model checkpoints
`GET /health`	Healthcheck
`GET /`	Web UI
`GET /voice-studio`	Gradio voice-cloning UI (when `ENABLE_VOICE_STUDIO=true`)

Voice cloning — different shape from cosyvoice

Profile-based, not voice-id-based:

Clone a voice via the /voice-studio web UI (uploads reference audio + transcript, names the profile, e.g. glados).
The wrapper writes /root/qwen3-tts/voice_library/profiles/<name>/{meta.json,reference.wav} inside the container (bind-mounted to /worktank/qwen3-tts/voices/profiles/<name>/ on the host).
Reference the profile in synthesis requests as voice="clone:<name>".

# OpenAI-shape, English with emotion via instruction
curl -X POST http://10.100.79.3:8191/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-TTS-12Hz-1.7B",
    "voice": "clone:glados",
    "input": "You should leave now. Visitors are not welcome.",
    "instructions": "speak with cold contempt",
    "response_format": "wav"
  }' \
  -o glados_en_angry.wav

(Note instructions field name; OpenAI standard. Whether English instructions actually drive emotion correctly here is the open question this stack exists to test — see the deploy playbook's verify section.)

Path layout

Host path	Container path	Purpose	Restic?
`/worktank/qwen3-tts/cache/`	`/root/.cache/huggingface`	Model cache (~5 GB after first run)	excluded
`/worktank/qwen3-tts/voices/`	`/root/qwen3-tts/voice_library`	Cloned profiles	included (precious)

Deploy

Via elway — see playbooks/deploy-qwen3-tts.yaml in the eshpfi-management root. The playbook builds the image, ensures the host dirs, installs compose + .env, brings up. First boot downloads the 1.7B model from HF (~6 GB), which is the slowest step.

scripts/elway irv-ml1 --playbook playbooks/deploy-qwen3-tts.yaml

To pin a different upstream wrapper SHA, set --var sha=<new-sha> or edit .env on the server and rebuild.

Switching to the smaller model

If 1.7B is too heavy or you need to share GPUs more aggressively:

ssh irv-ml1 '
  cd /opt/docker/compose/qwen3-tts && \
  sed -i "s|^QWEN3_TTS_MODEL=.*|QWEN3_TTS_MODEL=Qwen/Qwen3-TTS-12Hz-0.6B|" .env && \
  docker compose up -d
'

The new model auto-downloads on next start (~2–3 GB).

3.7 KiB Raw Blame History Unescape Escape