qwen3-tts/.env.example: document the optimized-backend crashloop trap

Investigation of the slow (8-12s) qwen3-tts TTFB found the upstream
wrapper has 5 backend options. The advertised path to fast TTFB is
TTS_BACKEND=optimized (torch.compile + CUDA graphs + real-time
streaming). It loads cleanly but crashes the container during its
hardcoded warmup phase — silent exit (ExitCode 0, no traceback,
no OOM kill), repeats every ~22s under restart policy.

TTS_WARMUP_ON_START=false suppresses the factory-level warmup but
the optimized backend has its own internal warmup that fires
regardless and triggers the crash.

Updated the .env.example block to enumerate all 5 backend options
with their actual current behavior so future-Claude doesn't re-walk
this path. official is staying as the default.
This commit is contained in:
2026-04-27 22:12:04 -07:00
parent 5fd69f06b1
commit db42a7cc17
+17 -3
View File
@@ -20,9 +20,23 @@ QWEN3_TTS_PORT=8191
QWEN3_TTS_BIND=0.0.0.0 QWEN3_TTS_BIND=0.0.0.0
# ── runtime ────────────────────────────────────────────────────────── # ── runtime ──────────────────────────────────────────────────────────
# Inference backend. `official` = default upstream; `optimized` = # Inference backend. Five options exposed by /app/api/backends/factory.py:
# faster but slightly less robust; `vllm_omni` = vLLM-backed (needs # official — default. Naive HF transformers autoregressive loop.
# more VRAM); `pytorch` = bare pytorch path. # Reliable, but the GPU sits at ~27% utilization and
# TTFB on a sentence is 8-12s (no streaming endpoint
# tuning). 2026-04-27 bench confirmed.
# optimized — torch.compile + CUDA graphs + real-time streaming.
# Should be the path to upstream's claimed 97 ms TTFB.
# CRASHES the container during its hardcoded warmup as
# of 2026-04-27 (silent ExitCode 0, no Traceback, repeats
# every ~22s under restart policy). TTS_WARMUP_ON_START=
# false doesn't help — the optimized backend's warmup is
# separate from the factory-level warmup. Don't use until
# upstream root-cause is identified or worked around.
# vllm_omni — vLLM-Omni-backed. Faster but needs more VRAM and a
# vLLM dependency we haven't pulled. Untested.
# pytorch — CPU-only fallback.
# openvino — Intel CPU experimental.
QWEN3_TTS_BACKEND=official QWEN3_TTS_BACKEND=official
# Model variant. Upstream publishes four checkpoints on HF: # Model variant. Upstream publishes four checkpoints on HF: