qwen3-tts/.env.example: document the optimized-backend crashloop trap
Investigation of the slow (8-12s) qwen3-tts TTFB found the upstream wrapper has 5 backend options. The advertised path to fast TTFB is TTS_BACKEND=optimized (torch.compile + CUDA graphs + real-time streaming). It loads cleanly but crashes the container during its hardcoded warmup phase — silent exit (ExitCode 0, no traceback, no OOM kill), repeats every ~22s under restart policy. TTS_WARMUP_ON_START=false suppresses the factory-level warmup but the optimized backend has its own internal warmup that fires regardless and triggers the crash. Updated the .env.example block to enumerate all 5 backend options with their actual current behavior so future-Claude doesn't re-walk this path. official is staying as the default.
This commit is contained in:
@@ -20,9 +20,23 @@ QWEN3_TTS_PORT=8191
|
|||||||
QWEN3_TTS_BIND=0.0.0.0
|
QWEN3_TTS_BIND=0.0.0.0
|
||||||
|
|
||||||
# ── runtime ──────────────────────────────────────────────────────────
|
# ── runtime ──────────────────────────────────────────────────────────
|
||||||
# Inference backend. `official` = default upstream; `optimized` =
|
# Inference backend. Five options exposed by /app/api/backends/factory.py:
|
||||||
# faster but slightly less robust; `vllm_omni` = vLLM-backed (needs
|
# official — default. Naive HF transformers autoregressive loop.
|
||||||
# more VRAM); `pytorch` = bare pytorch path.
|
# Reliable, but the GPU sits at ~27% utilization and
|
||||||
|
# TTFB on a sentence is 8-12s (no streaming endpoint
|
||||||
|
# tuning). 2026-04-27 bench confirmed.
|
||||||
|
# optimized — torch.compile + CUDA graphs + real-time streaming.
|
||||||
|
# Should be the path to upstream's claimed 97 ms TTFB.
|
||||||
|
# CRASHES the container during its hardcoded warmup as
|
||||||
|
# of 2026-04-27 (silent ExitCode 0, no Traceback, repeats
|
||||||
|
# every ~22s under restart policy). TTS_WARMUP_ON_START=
|
||||||
|
# false doesn't help — the optimized backend's warmup is
|
||||||
|
# separate from the factory-level warmup. Don't use until
|
||||||
|
# upstream root-cause is identified or worked around.
|
||||||
|
# vllm_omni — vLLM-Omni-backed. Faster but needs more VRAM and a
|
||||||
|
# vLLM dependency we haven't pulled. Untested.
|
||||||
|
# pytorch — CPU-only fallback.
|
||||||
|
# openvino — Intel CPU experimental.
|
||||||
QWEN3_TTS_BACKEND=official
|
QWEN3_TTS_BACKEND=official
|
||||||
|
|
||||||
# Model variant. Upstream publishes four checkpoints on HF:
|
# Model variant. Upstream publishes four checkpoints on HF:
|
||||||
|
|||||||
Reference in New Issue
Block a user