Move the ~22-service flat "AI Systems" group off the Main tab into a new
four-tab layout (Main / AI / Infrastructure / Toolchain). The AI tab sorts
the inference fleet by function into seven groups:
AI - Inference gen, char-rp, char-rp-reasoning, Granite summarizer
AI - Eval & Retrieval Selene, Skywork Reward, Qwen3 rerank/embed, image-bench
AI - Gateways & Chat LiteLLM, Asset Engine, Gateway Chat, Open WebUI, ...
AI - Speech (TTS) Chatterbox Fast, Kokoro, mOrpheus
AI - Audio Tools Parakeet ASR, YT Voice Clipper
AI - Image & Media ComfyUI, Arbo
AI - Dormant stopped rollback seats + retired auditions
Relabel each stack's homepage.group so canonical stacks/ matches the live
containers on ana-ml2, ana-docker, and irv-ml1. Dormant stacks were refreshed
with `docker compose up --no-start` so they carry the new label while staying
stopped (compose-start rollback preserved). settings.yaml drives tab/order/
columns; services.yaml and README updated to the new scheme.
IndexTTS-2's tts.infer(stream_return=True) is a generator that yields
audio chunks per text segment as they finish, plus inter-segment
silence. Expose this via the existing POST /v1/audio/speech with a new
"stream": true field on the request body.
Wire-up:
- 44-byte WAV header emitted up front with placeholder data length
(0xFFFFFFFF) so chunks can be written before total samples are
known. Players that read until EOF (mpv, ffplay, aplay, sox,
browsers via <audio>) handle this fine.
- Each yielded chunk goes through _chunk_to_pcm_bytes(), which
handles torch tensors / numpy arrays in either int16 or float
(-1..1) form.
- 22050 Hz mono int16 — IndexTTS-2's hardcoded output shape.
Time-to-first-audio drops from full-file latency to ~one-segment
latency. Single-sentence inputs barely benefit; long passages /
multi-paragraph reads benefit a lot. Strict metadata parsers may
balk at the placeholder size — request without stream for a
closed-length WAV in that case.
INDEX_TTS_TAG bumped to v2 to force a rebuild.
budget hazard + media-CDN workaround
Tried adding git-lfs install + git lfs pull to the build to get
real example WAVs into the image — failed with:
Error downloading object: examples/emo_hate.wav: Smudge error:
batch response: This repository exceeded its LFS budget. The
account responsible for the budget should increase it to
restore access.
The index-tts org's LFS bandwidth quota is exhausted upstream and
out of our control. Reverting the Dockerfile change. The examples
aren't needed for the wrapper to work; emotion_text and
emotion_vector are sufficient for end-to-end testing without any
WAV file at all.
For users who want the bundled example clips as starter audio,
README now documents the media-CDN URL trick — same LFS objects
served via a different code path that doesn't count against the
LFS API budget. INDEX_TTS_TAG stays at v1.
The IndexTTS-2 repo stores examples/emo_*.wav and examples/voice_*.wav
as Git LFS objects. v1 of our image cloned the repo without an LFS
pull, leaving those paths as ~130-byte pointer text files — unusable
for `docker cp` into /worktank/index-tts/{voices,emotions}/ as starter
references. (Caught when an emotion_voice="hate" call returned audio
that was actually the pointer text round-tripped through file IO.)
v2 adds git-lfs to the apt list, calls `git lfs install --system`
once, and `git lfs pull` after the checkout. Adds ~1-2 MB to the
image (the examples are small audio clips). INDEX_TTS_TAG bumped to
v2 to force a clean rebuild.
Adds a third TTS to the irv-ml1 fleet. IndexTTS-2 is Bilibili's
emotion-controllable zero-shot TTS (paper 2506.21619). Distinguishing
capability vs the existing two: timbre and emotion are disentangled —
clone a voice's timbre from one reference and the emotion from a
different reference, OR set emotion via 8-vector, OR derive it from a
text description. Neither CosyVoice 3 nor Qwen3-TTS-1.7B-Base does
this cleanly in English.
Wrapper is owned end-to-end (~150 lines in app.py) — the only existing
FastAPI fork (csllpr/index-tts-fastapi) targets v1 and is a dormant
single-commit repo. Upstream IndexTTS-2 ships only a Gradio webui.
Layout follows the qwen3-tts pattern:
stacks/index-tts/
Dockerfile — CUDA 12.8 base, IndexTTS pinned to a SHA
app.py — FastAPI: POST /v1/audio/speech + /v1/voices
entrypoint.sh — one-time HF snapshot_download of the weights
compose.yaml — env-driven, GPU pinning support, bind mounts
.env.example — port 8192, fp16, paths
README.md — API examples + comparison vs the other TTS
playbooks/deploy-index-tts.yaml — elway playbook for irv-ml1
Voice and emotion libraries are flat host dirs of WAVs, bind-mounted.
Drop a new <name>.wav and /v1/voices picks it up immediately.
License caveat: IndexTTS-2 weights ship under a custom Bilibili
license (free at our scale, not OSI-open). README documents it.
vh