# Chatterbox Turbo

Resemble AI's 350M-param low-latency English TTS with zero-shot voice
cloning, served via [devnen/Chatterbox-TTS-Server](https://github.com/devnen/Chatterbox-TTS-Server)
— the most actively-maintained OpenAI-compat wrapper supporting Turbo.

Model: [ResembleAI/chatterbox-turbo](https://huggingface.co/ResembleAI/chatterbox-turbo)
— released April 2026, ~6× realtime, ~75 ms latency, MIT-licensed.

## Why this stack exists

Fills the **low-latency English voice-cloning** slot none of the other
TTS own cleanly: Kokoro is fast but fixed-voice; IndexTTS-2 clones
beautifully but is slow; Qwen3-TTS-Base clones at a higher quality bar
but isn't tuned for sub-second latency. Chatterbox Turbo trades some
fidelity for **6× realtime + 5-second-reference cloning**, ideal for
real-time voice-agent use cases.

| | use case |
|---|---|
| **Chatterbox Turbo** | low-latency English w/ voice cloning + paralinguistic tags |
| Kokoro | low-latency English, fixed voice library |
| IndexTTS-2 | English voice cloning + emotion vector / text control |
| Qwen3-TTS-1.7B-Base | high-quality English voice cloning |
| CosyVoice 3 | multilingual (Chinese-leaning) |
| VibeVoice 1.5B | long-form / multi-speaker dialogue |

## Headline features

- **Zero-shot voice cloning from ~5 s reference** (base Chatterbox
  needs ~10 s; Turbo cuts that in half).
- **Native paralinguistic tags inline in text** — drop these into your
  prompt and the model honors them. Turbo ships exactly **9** tags
  (verified against the live model at `/api/model-info`):
  ```
  [laugh] [chuckle] [sigh] [gasp] [cough] [clear throat] [sniff] [groan] [shush]
  ```
  Note: base Chatterbox docs list `[whisper]` and `[breath]` — those
  are NOT in the Turbo set, ignore them or you'll burn time wondering
  why nothing happens. Different shape from IndexTTS-2's 8-vector
  emotion control: cleaner for "say it like this" markup directly in
  the prompt.
- **Mandatory PerTh watermark** on outputs (Resemble policy, cannot
  be disabled). Non-issue for internal use; mention it if you ever
  ship Chatterbox-generated audio externally.

## API

OpenAI-compat at `http://10.100.79.3:8196`:

```bash
# Built-in voices.
curl http://10.100.79.3:8196/v1/audio/voices

# Single-shot synthesis with a built-in voice.
curl -fsS -X POST http://10.100.79.3:8196/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"chatterbox-turbo","input":"Hello there. [laugh] What a day.","voice":"alloy","response_format":"wav"}' \
  > out.wav

# Voice cloning — drop a 5 s reference WAV into
# /worktank/chatterbox/reference_audio/glados.wav, then:
curl -fsS -X POST http://10.100.79.3:8196/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"chatterbox-turbo","input":"I have all the time in the world.","voice":"glados"}' \
  > glados.wav

# Streaming (where supported by the wrapper).
curl -fsS -X POST http://10.100.79.3:8196/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"chatterbox-turbo","input":"long passage…","voice":"alloy","stream":true}' \
  | mpv --no-cache -
```

Native `/tts` endpoint (devnen wrapper extension) and OpenAPI at `/docs`.
Healthcheck at `/health`.

## Voice library

Drop reference WAV / MP3 / FLAC into
`/worktank/chatterbox/reference_audio/` on the host. The wrapper
discovers new files on next request — no restart needed. Use clean
~5 s clips, single speaker.

Built-in OpenAI-style voice aliases (alloy, echo, fable, onyx, nova,
shimmer) map to bundled presets — useful for OpenAI SDK clients that
hardcode those names.

## Deploy

```bash
scripts/elway irv-ml1 --playbook playbooks/deploy-chatterbox.yaml
```

Cold deploy budget:
- Image build: ~5-8 GB (CUDA + torch + Chatterbox deps)
- Model download: ~6 GB (Chatterbox Turbo weights, first run)
- **Total: ~12 GB on /worktank/chatterbox/**

First build: ~8-10 min. First synthesis: ~10-30 s warmup.

## Switching the model

```bash
ssh irv-ml1 '
  cd /opt/docker/compose/chatterbox
  sed -i "s|^CHATTERBOX_MODEL_REPO=.*|CHATTERBOX_MODEL_REPO=ResembleAI/chatterbox|" .env
  docker compose up -d
'
```

Options for `CHATTERBOX_MODEL_REPO`:
- `ResembleAI/chatterbox-turbo` — flagship Turbo (default, fastest)
- `ResembleAI/chatterbox` — base Chatterbox, 500M, slower but with
  exaggeration / CFG-weight knobs Turbo doesn't expose
- `ResembleAI/chatterbox-multilingual` — 23 languages (slower than
  Turbo, useful if you need non-English on this stack vs CosyVoice 3)

## Gotchas

- **Python 3.10 only** (devnen wrapper). Image bakes that in; not
  something you'd hit unless you fork the Dockerfile.
- **PerTh watermark** is unconditional. Can't disable.
- **Turbo loses some knobs vs base Chatterbox** — no `exaggeration`
  or CFG-weight tuning. If you need expressive amplitude control,
  flip to base Chatterbox via the config swap above.
- **Repo is fresh** (~weekly commits). Pin to a SHA in `.env`
  (`CHATTERBOX_SHA=...`) and rebuild monthly to ride upstream
  bug-fix progress.
- **License**: wrapper MIT; weights MIT (Resemble) — including the
  watermark requirement.