qwen3-tts/.env.example: document the optimized-backend crashloop trap
Investigation of the slow (8-12s) qwen3-tts TTFB found the upstream wrapper has 5 backend options. The advertised path to fast TTFB is TTS_BACKEND=optimized (torch.compile + CUDA graphs + real-time streaming). It loads cleanly but crashes the container during its hardcoded warmup phase — silent exit (ExitCode 0, no traceback, no OOM kill), repeats every ~22s under restart policy. TTS_WARMUP_ON_START=false suppresses the factory-level warmup but the optimized backend has its own internal warmup that fires regardless and triggers the crash. Updated the .env.example block to enumerate all 5 backend options with their actual current behavior so future-Claude doesn't re-walk this path. official is staying as the default.
This commit is contained in:
@@ -20,9 +20,23 @@ QWEN3_TTS_PORT=8191
|
||||
QWEN3_TTS_BIND=0.0.0.0
|
||||
|
||||
# ── runtime ──────────────────────────────────────────────────────────
|
||||
# Inference backend. `official` = default upstream; `optimized` =
|
||||
# faster but slightly less robust; `vllm_omni` = vLLM-backed (needs
|
||||
# more VRAM); `pytorch` = bare pytorch path.
|
||||
# Inference backend. Five options exposed by /app/api/backends/factory.py:
|
||||
# official — default. Naive HF transformers autoregressive loop.
|
||||
# Reliable, but the GPU sits at ~27% utilization and
|
||||
# TTFB on a sentence is 8-12s (no streaming endpoint
|
||||
# tuning). 2026-04-27 bench confirmed.
|
||||
# optimized — torch.compile + CUDA graphs + real-time streaming.
|
||||
# Should be the path to upstream's claimed 97 ms TTFB.
|
||||
# CRASHES the container during its hardcoded warmup as
|
||||
# of 2026-04-27 (silent ExitCode 0, no Traceback, repeats
|
||||
# every ~22s under restart policy). TTS_WARMUP_ON_START=
|
||||
# false doesn't help — the optimized backend's warmup is
|
||||
# separate from the factory-level warmup. Don't use until
|
||||
# upstream root-cause is identified or worked around.
|
||||
# vllm_omni — vLLM-Omni-backed. Faster but needs more VRAM and a
|
||||
# vLLM dependency we haven't pulled. Untested.
|
||||
# pytorch — CPU-only fallback.
|
||||
# openvino — Intel CPU experimental.
|
||||
QWEN3_TTS_BACKEND=official
|
||||
|
||||
# Model variant. Upstream publishes four checkpoints on HF:
|
||||
|
||||
Reference in New Issue
Block a user