stacks/index-tts: own FastAPI wrapper for IndexTTS-2 + deploy playbook
Adds a third TTS to the irv-ml1 fleet. IndexTTS-2 is Bilibili's
emotion-controllable zero-shot TTS (paper 2506.21619). Distinguishing
capability vs the existing two: timbre and emotion are disentangled —
clone a voice's timbre from one reference and the emotion from a
different reference, OR set emotion via 8-vector, OR derive it from a
text description. Neither CosyVoice 3 nor Qwen3-TTS-1.7B-Base does
this cleanly in English.
Wrapper is owned end-to-end (~150 lines in app.py) — the only existing
FastAPI fork (csllpr/index-tts-fastapi) targets v1 and is a dormant
single-commit repo. Upstream IndexTTS-2 ships only a Gradio webui.
Layout follows the qwen3-tts pattern:
stacks/index-tts/
Dockerfile — CUDA 12.8 base, IndexTTS pinned to a SHA
app.py — FastAPI: POST /v1/audio/speech + /v1/voices
entrypoint.sh — one-time HF snapshot_download of the weights
compose.yaml — env-driven, GPU pinning support, bind mounts
.env.example — port 8192, fp16, paths
README.md — API examples + comparison vs the other TTS
playbooks/deploy-index-tts.yaml — elway playbook for irv-ml1
Voice and emotion libraries are flat host dirs of WAVs, bind-mounted.
Drop a new <name>.wav and /v1/voices picks it up immediately.
License caveat: IndexTTS-2 weights ship under a custom Bilibili
license (free at our scale, not OSI-open). README documents it.
This commit is contained in:
@@ -0,0 +1,134 @@
|
||||
# IndexTTS-2
|
||||
|
||||
Bilibili's emotion-controllable zero-shot TTS
|
||||
([paper](https://arxiv.org/abs/2506.21619),
|
||||
[code](https://github.com/index-tts/index-tts),
|
||||
[weights](https://huggingface.co/IndexTeam/IndexTTS-2)) served behind
|
||||
our own thin FastAPI wrapper.
|
||||
|
||||
## Why this stack exists alongside the other two TTS
|
||||
|
||||
| | CosyVoice 3 | Qwen3-TTS-1.7B-Base | **IndexTTS-2** |
|
||||
|---|---|---|---|
|
||||
| Voice cloning | ✅ | ✅ (`-Base` variant only) | ✅ |
|
||||
| English quality | medium (Chinese-leaning) | high (English-first) | medium (better than CosyVoice) |
|
||||
| Emotion control | `instruct` mode is Chinese-only | inline tags | **explicit: audio / 8-vector / text** |
|
||||
| Duration control | implicit | implicit | **explicit token-count mode** |
|
||||
| License | Apache 2.0 | Apache 2.0 | custom (Bilibili — free at our scale) |
|
||||
| Wrapper | bare CosyVoice CLI | groxaxo upstream FastAPI | ours, in this dir |
|
||||
|
||||
The differentiator is **disentangled emotion**. IndexTTS-2 lets you
|
||||
clone a voice's timbre from one reference and the emotion from a
|
||||
different reference — or skip emotion-audio entirely and supply an
|
||||
8-vector or a text description. Neither of the other two does this
|
||||
cleanly in English.
|
||||
|
||||
## API
|
||||
|
||||
OpenAI-compat-ish:
|
||||
|
||||
```bash
|
||||
# List available voices + emotions
|
||||
curl http://10.100.79.3:8192/v1/voices
|
||||
|
||||
# Basic synthesis (uses speaker WAV's natural emotion)
|
||||
curl -X POST http://10.100.79.3:8192/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"input": "I have all the time in the world.",
|
||||
"voice": "glados"
|
||||
}' > glados.wav
|
||||
|
||||
# Same speaker, emotion taken from a separate reference WAV
|
||||
curl -X POST http://10.100.79.3:8192/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"input": "I have all the time in the world.",
|
||||
"voice": "glados",
|
||||
"emotion_voice": "menacing",
|
||||
"emotion_alpha": 0.9
|
||||
}' > glados-menacing.wav
|
||||
|
||||
# Same speaker, emotion as 8-vector
|
||||
# Order: happy, angry, sad, afraid, disgusted, melancholic, surprised, calm
|
||||
curl -X POST http://10.100.79.3:8192/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"input": "I have all the time in the world.",
|
||||
"voice": "glados",
|
||||
"emotion_vector": [0, 0.7, 0, 0, 0.2, 0, 0, 0]
|
||||
}' > glados-angry.wav
|
||||
|
||||
# Same speaker, emotion derived from text by bundled QwenEmotion model
|
||||
curl -X POST http://10.100.79.3:8192/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"input": "I have all the time in the world.",
|
||||
"voice": "glados",
|
||||
"emotion_text": "she said with quiet menace"
|
||||
}' > glados-menacing.wav
|
||||
|
||||
# Health
|
||||
curl http://10.100.79.3:8192/healthz
|
||||
```
|
||||
|
||||
Output is always WAV (PCM_16, 22050 Hz — IndexTTS-2's native rate).
|
||||
`response_format` other than `wav` is rejected.
|
||||
|
||||
## Voice library
|
||||
|
||||
Flat dirs on the host (bind-mounted; survives container recreates):
|
||||
|
||||
```
|
||||
/worktank/index-tts/voices/<name>.wav # timbre references
|
||||
/worktank/index-tts/emotions/<name>.wav # emotion references
|
||||
```
|
||||
|
||||
Drop a new WAV in either dir and `/v1/voices` picks it up immediately —
|
||||
no restart. Use clean reference clips, 5-30 s each, single speaker.
|
||||
Cloned voices live under restic; cache (model weights) is excluded.
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
scripts/elway irv-ml1 --playbook playbooks/deploy-index-tts.yaml
|
||||
```
|
||||
|
||||
First build: ~5-10 min for the docker image (CUDA torch + IndexTTS
|
||||
deps), plus ~5-7 GB model download on first container start. Subsequent
|
||||
starts: ~30 s warmup.
|
||||
|
||||
## Switching GPUs
|
||||
|
||||
irv-ml1 has an RTX 3090 (cuda:0) + RTX A6000 (cuda:1). Default is
|
||||
auto-pick (cuda:0). To pin to the A6000 alongside Qwen3-TTS-on-3090:
|
||||
|
||||
```bash
|
||||
ssh irv-ml1 '
|
||||
cd /opt/docker/compose/index-tts
|
||||
sed -i "s|^INDEX_TTS_DEVICE=.*|INDEX_TTS_DEVICE=cuda:1|" .env
|
||||
docker compose up -d
|
||||
'
|
||||
```
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **License** — IndexTeam/IndexTTS-2 ships under a custom Bilibili
|
||||
license, not Apache/MIT. Free at our scale (the commercial tier kicks
|
||||
in at 100M MAU / RMB 1B revenue). Restricts using outputs to train
|
||||
other AI models. Read `INDEX_MODEL_LICENSE` in the HF repo before
|
||||
using outputs anywhere external.
|
||||
- **Sample rate** — 22050 Hz is hardcoded upstream. If you need 24 kHz
|
||||
or 48 kHz, resample in the caller.
|
||||
- **Emotion-source precedence** — if multiple emotion controls are
|
||||
specified in one request, the first non-empty one wins in this order:
|
||||
`emotion_voice` > `emotion_vector` > `emotion_text`. The others are
|
||||
silently ignored.
|
||||
- **Model download** — happens in the entrypoint on first start; the
|
||||
config.yaml file in the cache dir is the gate. To force a re-download,
|
||||
delete that file and recreate the container.
|
||||
- **HF cache pinning** — `infer_v2.py` pins `HF_HUB_CACHE` at import
|
||||
time to `./checkpoints/hf_cache`. The wrapper sets this env var
|
||||
before importing, so auxiliary HF assets (MaskGCT, campplus, BigVGAN,
|
||||
w2v-bert) land alongside the IndexTTS-2 weights and are excluded
|
||||
from restic together.
|
||||
Reference in New Issue
Block a user