stacks: add Kokoro, VibeVoice 1.5B, Chatterbox Turbo (TTS slate fill-in)
Three TTS additions to round out coverage on irv-ml1, each filling a
distinct niche the existing slate doesn't own.
Final coverage matrix (all on irv-ml1):
Kokoro — low-latency English, fixed voice library, ~300ms TTFA
Chatterbox Turbo — low-latency English w/ voice cloning + paralinguistic tags
IndexTTS-2 — English voice cloning + emotion vector / text control
Qwen3-TTS-1.7B-Base — high-quality English voice cloning
CosyVoice 3 — multilingual (Chinese-leaning)
VibeVoice 1.5B — long-form / multi-speaker dialogue
stacks/kokoro:
- port 8193, GPU device 0 (3090)
- pulls ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master (no Dockerfile,
no first-run model download — models baked in)
- 60+ built-in voices, OpenAI-compat with stream=true over chunked HTTP
- Apache-2.0 weights + code, ~1 GB VRAM
stacks/vibevoice:
- port 8194, GPU device 1 (A6000 — for 7B headroom)
- builds groxaxo/VibeVoice-FastAPI1 (more current fork of ncoder-ai)
pinned to 7614c469a145
- default model microsoft/VibeVoice-1.5B (~7 GB bf16 VRAM); env var
swap to rsxdalv/VibeVoice-Large (7B) or FabioSarracino/VibeVoice-Large-Q8
- multi-speaker dialogue via /v1/vibevoice/generate with Speaker N: format
- long-form niche only — not low-latency
stacks/chatterbox:
- port 8196, GPU device 0 (3090)
- builds devnen/Chatterbox-TTS-Server (most active Turbo-supporting wrapper)
- default model ResembleAI/chatterbox-turbo (~2.5 GB fp16, ~75ms latency)
- paralinguistic tags inline ([laugh] [whisper] etc) — different shape
from IndexTTS-2's emotion vector; fills the speed+cloning niche
Kokoro/IndexTTS don't cover together
- mandatory PerTh watermark on outputs (Resemble policy)
Three matching playbooks under playbooks/deploy-{kokoro,vibevoice,
chatterbox}.yaml. All idempotent, creates-/when-gated.
Cold-deploy disk on /worktank/: ~7 GB Kokoro + ~19 GB VibeVoice 1.5B
+ ~12 GB Chatterbox = ~38 GB total. VRAM concurrent: ~10-11 GB across
both GPUs.
Skipped from the original four-stack proposal: VibeVoice Realtime
(overlaps Kokoro's niche; Kokoro wins on latency, license, and not
needing a build).
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# Chatterbox Turbo stack tunables. Copy to `.env` on irv-ml1 before
|
||||
# deploying.
|
||||
|
||||
# ── build pin ────────────────────────────────────────────────────────
|
||||
# SHA of devnen/Chatterbox-TTS-Server. Bump + rebuild when you want
|
||||
# upstream wrapper updates. Pin a SHA — the wrapper has no tagged
|
||||
# releases yet.
|
||||
CHATTERBOX_SHA=main
|
||||
|
||||
# Local image tag — bump when you change build context to force a
|
||||
# fresh layer build.
|
||||
CHATTERBOX_TAG=v1
|
||||
|
||||
# ── network ──────────────────────────────────────────────────────────
|
||||
# Host port. Container listens on 8004 internally.
|
||||
# Reserved on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS,
|
||||
# 8192 IndexTTS-2, 8193 Kokoro, 8194 VibeVoice, 8765 Parakeet.
|
||||
# 8196 picked here.
|
||||
CHATTERBOX_PORT=8196
|
||||
|
||||
# Bind address. 0.0.0.0 exposes on all interfaces (incl. WG tunnel
|
||||
# interface 10.100.79.3); 127.0.0.1 restricts to local-only.
|
||||
CHATTERBOX_BIND=0.0.0.0
|
||||
|
||||
# ── runtime / GPU ────────────────────────────────────────────────────
|
||||
# Devices visible inside the container. "0" pins to the RTX 3090.
|
||||
# Chatterbox Turbo is small (~2.5 GB fp16) — the 3090 is plenty.
|
||||
CHATTERBOX_GPU_DEVICES=0
|
||||
|
||||
# Model checkpoint. Options:
|
||||
# ResembleAI/chatterbox-turbo — flagship Turbo, 350M, EN-only,
|
||||
# ~2.5 GB fp16, ~75 ms latency
|
||||
# ResembleAI/chatterbox — base Chatterbox, 500M, EN-only,
|
||||
# ~3.5 GB fp16, slower but with
|
||||
# exaggeration/CFG-weight knobs
|
||||
# ResembleAI/chatterbox-multilingual — 23 languages, slower than Turbo
|
||||
CHATTERBOX_MODEL_REPO=ResembleAI/chatterbox-turbo
|
||||
|
||||
# ── persistent storage on the host ───────────────────────────────────
|
||||
# Reference audio dir for voice cloning. Drop short (~5 s) reference
|
||||
# WAVs in here; the wrapper picks them up by filename. Cloned voices
|
||||
# need the original reference to recreate — included in restic.
|
||||
CHATTERBOX_REFERENCE_DIR=/worktank/chatterbox/reference_audio
|
||||
|
||||
# HuggingFace cache — Chatterbox Turbo weights (~6 GB) land here on
|
||||
# first start. Bind-mounted so they survive container recreate.
|
||||
# Excluded from restic (regenerable from HF).
|
||||
CHATTERBOX_CACHE_DIR=/worktank/chatterbox/cache
|
||||
|
||||
# Optional: mount a host config.yaml for hot-edit. Leave commented
|
||||
# out in compose.yaml unless you actively want this.
|
||||
# CHATTERBOX_CONFIG=/worktank/chatterbox/config.yaml
|
||||
@@ -0,0 +1,127 @@
|
||||
# Chatterbox Turbo
|
||||
|
||||
Resemble AI's 350M-param low-latency English TTS with zero-shot voice
|
||||
cloning, served via [devnen/Chatterbox-TTS-Server](https://github.com/devnen/Chatterbox-TTS-Server)
|
||||
— the most actively-maintained OpenAI-compat wrapper supporting Turbo.
|
||||
|
||||
Model: [ResembleAI/chatterbox-turbo](https://huggingface.co/ResembleAI/chatterbox-turbo)
|
||||
— released April 2026, ~6× realtime, ~75 ms latency, MIT-licensed.
|
||||
|
||||
## Why this stack exists
|
||||
|
||||
Fills the **low-latency English voice-cloning** slot none of the other
|
||||
TTS own cleanly: Kokoro is fast but fixed-voice; IndexTTS-2 clones
|
||||
beautifully but is slow; Qwen3-TTS-Base clones at a higher quality bar
|
||||
but isn't tuned for sub-second latency. Chatterbox Turbo trades some
|
||||
fidelity for **6× realtime + 5-second-reference cloning**, ideal for
|
||||
real-time voice-agent use cases.
|
||||
|
||||
| | use case |
|
||||
|---|---|
|
||||
| **Chatterbox Turbo** | low-latency English w/ voice cloning + paralinguistic tags |
|
||||
| Kokoro | low-latency English, fixed voice library |
|
||||
| IndexTTS-2 | English voice cloning + emotion vector / text control |
|
||||
| Qwen3-TTS-1.7B-Base | high-quality English voice cloning |
|
||||
| CosyVoice 3 | multilingual (Chinese-leaning) |
|
||||
| VibeVoice 1.5B | long-form / multi-speaker dialogue |
|
||||
|
||||
## Headline features
|
||||
|
||||
- **Zero-shot voice cloning from ~5 s reference** (base Chatterbox
|
||||
needs ~10 s; Turbo cuts that in half).
|
||||
- **Native paralinguistic tags inline in text** — drop these into your
|
||||
prompt and the model honors them:
|
||||
```
|
||||
[laugh] [cough] [sigh] [gasp] [whisper] [breath]
|
||||
```
|
||||
Different shape from IndexTTS-2's 8-vector emotion control: cleaner
|
||||
for "say it like this" markup directly in the prompt.
|
||||
- **Mandatory PerTh watermark** on outputs (Resemble policy, cannot
|
||||
be disabled). Non-issue for internal use; mention it if you ever
|
||||
ship Chatterbox-generated audio externally.
|
||||
|
||||
## API
|
||||
|
||||
OpenAI-compat at `http://10.100.79.3:8196`:
|
||||
|
||||
```bash
|
||||
# Built-in voices.
|
||||
curl http://10.100.79.3:8196/v1/audio/voices
|
||||
|
||||
# Single-shot synthesis with a built-in voice.
|
||||
curl -fsS -X POST http://10.100.79.3:8196/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"chatterbox-turbo","input":"Hello there. [laugh] What a day.","voice":"alloy","response_format":"wav"}' \
|
||||
> out.wav
|
||||
|
||||
# Voice cloning — drop a 5 s reference WAV into
|
||||
# /worktank/chatterbox/reference_audio/glados.wav, then:
|
||||
curl -fsS -X POST http://10.100.79.3:8196/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"chatterbox-turbo","input":"I have all the time in the world.","voice":"glados"}' \
|
||||
> glados.wav
|
||||
|
||||
# Streaming (where supported by the wrapper).
|
||||
curl -fsS -X POST http://10.100.79.3:8196/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"chatterbox-turbo","input":"long passage…","voice":"alloy","stream":true}' \
|
||||
| mpv --no-cache -
|
||||
```
|
||||
|
||||
Native `/tts` endpoint (devnen wrapper extension) and OpenAPI at `/docs`.
|
||||
Healthcheck at `/health`.
|
||||
|
||||
## Voice library
|
||||
|
||||
Drop reference WAV / MP3 / FLAC into
|
||||
`/worktank/chatterbox/reference_audio/` on the host. The wrapper
|
||||
discovers new files on next request — no restart needed. Use clean
|
||||
~5 s clips, single speaker.
|
||||
|
||||
Built-in OpenAI-style voice aliases (alloy, echo, fable, onyx, nova,
|
||||
shimmer) map to bundled presets — useful for OpenAI SDK clients that
|
||||
hardcode those names.
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
scripts/elway irv-ml1 --playbook playbooks/deploy-chatterbox.yaml
|
||||
```
|
||||
|
||||
Cold deploy budget:
|
||||
- Image build: ~5-8 GB (CUDA + torch + Chatterbox deps)
|
||||
- Model download: ~6 GB (Chatterbox Turbo weights, first run)
|
||||
- **Total: ~12 GB on /worktank/chatterbox/**
|
||||
|
||||
First build: ~8-10 min. First synthesis: ~10-30 s warmup.
|
||||
|
||||
## Switching the model
|
||||
|
||||
```bash
|
||||
ssh irv-ml1 '
|
||||
cd /opt/docker/compose/chatterbox
|
||||
sed -i "s|^CHATTERBOX_MODEL_REPO=.*|CHATTERBOX_MODEL_REPO=ResembleAI/chatterbox|" .env
|
||||
docker compose up -d
|
||||
'
|
||||
```
|
||||
|
||||
Options for `CHATTERBOX_MODEL_REPO`:
|
||||
- `ResembleAI/chatterbox-turbo` — flagship Turbo (default, fastest)
|
||||
- `ResembleAI/chatterbox` — base Chatterbox, 500M, slower but with
|
||||
exaggeration / CFG-weight knobs Turbo doesn't expose
|
||||
- `ResembleAI/chatterbox-multilingual` — 23 languages (slower than
|
||||
Turbo, useful if you need non-English on this stack vs CosyVoice 3)
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Python 3.10 only** (devnen wrapper). Image bakes that in; not
|
||||
something you'd hit unless you fork the Dockerfile.
|
||||
- **PerTh watermark** is unconditional. Can't disable.
|
||||
- **Turbo loses some knobs vs base Chatterbox** — no `exaggeration`
|
||||
or CFG-weight tuning. If you need expressive amplitude control,
|
||||
flip to base Chatterbox via the config swap above.
|
||||
- **Repo is fresh** (~weekly commits). Pin to a SHA in `.env`
|
||||
(`CHATTERBOX_SHA=...`) and rebuild monthly to ride upstream
|
||||
bug-fix progress.
|
||||
- **License**: wrapper MIT; weights MIT (Resemble) — including the
|
||||
watermark requirement.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Chatterbox Turbo (Resemble AI's 350M-param low-latency English TTS
|
||||
# with zero-shot voice cloning) served via devnen/Chatterbox-TTS-Server
|
||||
# — the most actively-maintained OpenAI-compat wrapper that supports
|
||||
# the Turbo checkpoint.
|
||||
#
|
||||
# Why this stack exists alongside the other TTS:
|
||||
# * Low-latency English with VOICE CLONING (Kokoro is fast but
|
||||
# fixed-voice; this fills the speed-AND-cloning slot).
|
||||
# * Native paralinguistic tags inline in text:
|
||||
# [laugh] [cough] [sigh] [gasp] [whisper] [breath]
|
||||
# Different shape from IndexTTS-2's emotion vector — cleaner for
|
||||
# "say it like this" markup directly in the prompt.
|
||||
# * MIT weights + code; ~2.5 GB VRAM at fp16; ~75 ms latency.
|
||||
# * Mandatory PerTh watermark on outputs (Resemble policy, can't
|
||||
# be disabled). Non-issue for internal use.
|
||||
#
|
||||
# Image is built locally from the upstream Dockerfile via docker
|
||||
# buildx git-context (no source vendored on the host).
|
||||
#
|
||||
# All tunables live in .env — edit that, not this file.
|
||||
|
||||
services:
|
||||
chatterbox:
|
||||
image: local/chatterbox:${CHATTERBOX_TAG}
|
||||
build:
|
||||
context: https://github.com/devnen/Chatterbox-TTS-Server.git#${CHATTERBOX_SHA}
|
||||
dockerfile: docker/Dockerfile.gpu
|
||||
container_name: chatterbox
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${CHATTERBOX_BIND:-0.0.0.0}:${CHATTERBOX_PORT}:8004"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${CHATTERBOX_GPU_DEVICES:-0}
|
||||
# Switch to ResembleAI/chatterbox-turbo (default) or the base
|
||||
# ResembleAI/chatterbox / ResembleAI/chatterbox-multilingual
|
||||
# via the wrapper's config hot-swap.
|
||||
- CHATTERBOX_MODEL_REPO=${CHATTERBOX_MODEL_REPO:-ResembleAI/chatterbox-turbo}
|
||||
- HF_HOME=/app/hf_cache
|
||||
volumes:
|
||||
- ${CHATTERBOX_REFERENCE_DIR}:/app/reference_audio
|
||||
- ${CHATTERBOX_CACHE_DIR}:/app/hf_cache
|
||||
# Optional: mount config.yaml as a host file for hot-edit. Default
|
||||
# is to use the in-image config + env var overrides.
|
||||
# - ${CHATTERBOX_CONFIG}:/app/config.yaml:ro
|
||||
healthcheck:
|
||||
# Devnen wrapper exposes /health; the OpenAPI/docs path is /docs.
|
||||
test: ["CMD-SHELL", "curl -fsS -o /dev/null http://localhost:8004/health || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# First boot pulls Chatterbox-Turbo (~6 GB total HF assets) and
|
||||
# warms torch.compile — give it a generous deadline.
|
||||
start_period: 600s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=Chatterbox Turbo
|
||||
- homepage.icon=mdi-account-music-outline
|
||||
- homepage.description=Low-latency English TTS w/ voice cloning + paralinguistics (irv-ml1)
|
||||
- homepage.href=http://10.100.79.3:${CHATTERBOX_PORT}
|
||||
Reference in New Issue
Block a user