stacks: add Kokoro, VibeVoice 1.5B, Chatterbox Turbo (TTS slate fill-in)
Three TTS additions to round out coverage on irv-ml1, each filling a
distinct niche the existing slate doesn't own.
Final coverage matrix (all on irv-ml1):
Kokoro — low-latency English, fixed voice library, ~300ms TTFA
Chatterbox Turbo — low-latency English w/ voice cloning + paralinguistic tags
IndexTTS-2 — English voice cloning + emotion vector / text control
Qwen3-TTS-1.7B-Base — high-quality English voice cloning
CosyVoice 3 — multilingual (Chinese-leaning)
VibeVoice 1.5B — long-form / multi-speaker dialogue
stacks/kokoro:
- port 8193, GPU device 0 (3090)
- pulls ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master (no Dockerfile,
no first-run model download — models baked in)
- 60+ built-in voices, OpenAI-compat with stream=true over chunked HTTP
- Apache-2.0 weights + code, ~1 GB VRAM
stacks/vibevoice:
- port 8194, GPU device 1 (A6000 — for 7B headroom)
- builds groxaxo/VibeVoice-FastAPI1 (more current fork of ncoder-ai)
pinned to 7614c469a145
- default model microsoft/VibeVoice-1.5B (~7 GB bf16 VRAM); env var
swap to rsxdalv/VibeVoice-Large (7B) or FabioSarracino/VibeVoice-Large-Q8
- multi-speaker dialogue via /v1/vibevoice/generate with Speaker N: format
- long-form niche only — not low-latency
stacks/chatterbox:
- port 8196, GPU device 0 (3090)
- builds devnen/Chatterbox-TTS-Server (most active Turbo-supporting wrapper)
- default model ResembleAI/chatterbox-turbo (~2.5 GB fp16, ~75ms latency)
- paralinguistic tags inline ([laugh] [whisper] etc) — different shape
from IndexTTS-2's emotion vector; fills the speed+cloning niche
Kokoro/IndexTTS don't cover together
- mandatory PerTh watermark on outputs (Resemble policy)
Three matching playbooks under playbooks/deploy-{kokoro,vibevoice,
chatterbox}.yaml. All idempotent, creates-/when-gated.
Cold-deploy disk on /worktank/: ~7 GB Kokoro + ~19 GB VibeVoice 1.5B
+ ~12 GB Chatterbox = ~38 GB total. VRAM concurrent: ~10-11 GB across
both GPUs.
Skipped from the original four-stack proposal: VibeVoice Realtime
(overlaps Kokoro's niche; Kokoro wins on latency, license, and not
needing a build).
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
# Kokoro-FastAPI stack tunables. Copy to `.env` on irv-ml1 before deploying.
|
||||
|
||||
# ── image pin ────────────────────────────────────────────────────────
|
||||
# Tagged release on GHCR. Avoid `latest` — upstream warns it can move
|
||||
# without notice. v0.2.4-master = 2025-12-13 release with Kokoro-82M v1.0
|
||||
# baked in (commit 9901c2b).
|
||||
KOKORO_TAG=v0.2.4-master
|
||||
|
||||
# ── network ──────────────────────────────────────────────────────────
|
||||
# Host port. Container listens on 8880 internally.
|
||||
# Reserved on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS,
|
||||
# 8192 IndexTTS-2, 8765 Parakeet. 8193 picked here.
|
||||
KOKORO_PORT=8193
|
||||
|
||||
# Bind address. 0.0.0.0 exposes on all interfaces (incl. WG tunnel
|
||||
# interface 10.100.79.3); 127.0.0.1 restricts to local-only.
|
||||
KOKORO_BIND=0.0.0.0
|
||||
|
||||
# ── runtime / GPU ────────────────────────────────────────────────────
|
||||
# Devices visible inside the container. "0" pins to the RTX 3090
|
||||
# (Kokoro is tiny — ~1 GB VRAM — and doesn't need the A6000). Use
|
||||
# "all" if you want the model swap to either GPU.
|
||||
KOKORO_GPU_DEVICES=0
|
||||
|
||||
# Logging level for the FastAPI app. INFO is the upstream default.
|
||||
KOKORO_LOG_LEVEL=INFO
|
||||
|
||||
# ── persistent storage on the host ───────────────────────────────────
|
||||
# Voicepacks dir — bind-mount target IF the (commented-out) override
|
||||
# is enabled in compose.yaml. Default: leave empty and use the
|
||||
# in-image voicepacks.
|
||||
KOKORO_VOICES_DIR=/worktank/kokoro/voices
|
||||
|
||||
# User-voices dir — a parallel directory the wrapper *also* scans for
|
||||
# voicepacks alongside the in-image ones. Always mounted (cheap, empty
|
||||
# by default). Drop your own .pt files here if you train Kokoro voices.
|
||||
KOKORO_USER_VOICES_DIR=/worktank/kokoro/user_voices
|
||||
@@ -0,0 +1,80 @@
|
||||
# Kokoro
|
||||
|
||||
[hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) served
|
||||
via [remsky/Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI).
|
||||
|
||||
## Why this stack exists
|
||||
|
||||
Lowest-latency English TTS in the fleet by a wide margin — ~300 ms
|
||||
time-to-first-audio on GPU, 35-100x realtime, ~1 GB VRAM at fp16.
|
||||
Native streaming (Kokoro's `KPipeline.__call__` is a per-phrase
|
||||
generator) and the wrapper exposes it via OpenAI-compat `stream=true`
|
||||
over chunked HTTP — drop-in for any OpenAI SDK client.
|
||||
|
||||
Complementary to the rest of the TTS slate:
|
||||
|
||||
| | use case |
|
||||
|---|---|
|
||||
| **Kokoro** | low-latency English, fixed voice library |
|
||||
| **Chatterbox Turbo** | low-latency English w/ voice cloning + paralinguistic tags |
|
||||
| **IndexTTS-2** | English voice cloning + emotion vector / text control |
|
||||
| **Qwen3-TTS-1.7B-Base** | high-quality English voice cloning |
|
||||
| **CosyVoice 3** | multilingual (Chinese-leaning) |
|
||||
| **VibeVoice 1.5B** | long-form podcast / multi-speaker dialogue |
|
||||
|
||||
## API
|
||||
|
||||
OpenAI-compat at `http://10.100.79.3:8193`:
|
||||
|
||||
```bash
|
||||
# List built-in voices (~60 of them, named like af_bella, am_adam, jf_*, zf_*).
|
||||
curl http://10.100.79.3:8193/v1/audio/voices
|
||||
|
||||
# Single-shot synthesis.
|
||||
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"kokoro","input":"Hello there.","voice":"af_bella","response_format":"wav"}' \
|
||||
> out.wav
|
||||
|
||||
# Streaming — pipe straight into a player.
|
||||
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"kokoro","input":"long passage here…","voice":"af_bella","stream":true}' \
|
||||
| mpv --no-cache -
|
||||
|
||||
# Voice mixing — sum voicepacks with weights.
|
||||
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"kokoro","input":"hello","voice":"af_bella(2)+af_heart(1)","response_format":"wav"}' \
|
||||
> mix.wav
|
||||
```
|
||||
|
||||
Web UI at `/web` (browse voices + try in-place); OpenAPI at `/docs`.
|
||||
|
||||
Supported `response_format`: `mp3 | wav | opus | flac | pcm`.
|
||||
|
||||
## Voices
|
||||
|
||||
- **Built-in**: 60+ in 8 languages (en-US, en-GB, ja, zh, es, fr, hi, it).
|
||||
Discoverable via `GET /v1/audio/voices` — naming convention is
|
||||
`<lang_code><gender_letter>_<name>` (e.g. `af_bella`, `am_adam`,
|
||||
`jf_alpha`, `zf_xiaobei`).
|
||||
- **Custom**: drop `.pt` voicepacks into `/worktank/kokoro/user_voices/`
|
||||
on the host. Wrapper auto-discovers them on next request (no
|
||||
restart). Training Kokoro voices is non-trivial — consult the
|
||||
hexgrad community for how-to.
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
scripts/elway irv-ml1 --playbook playbooks/deploy-kokoro.yaml
|
||||
```
|
||||
|
||||
First deploy: ~6.5 GB image pull from GHCR (~2-5 min on a fast link).
|
||||
No model download on first run — Kokoro-82M weights are baked in.
|
||||
Subsequent starts: a few seconds.
|
||||
|
||||
## License
|
||||
|
||||
Apache-2.0 for both the wrapper code (remsky/Kokoro-FastAPI) and the
|
||||
Kokoro-82M weights (hexgrad).
|
||||
@@ -0,0 +1,52 @@
|
||||
# Kokoro-82M served via remsky/Kokoro-FastAPI — the de-facto OpenAI-
|
||||
# compatible wrapper for hexgrad's Kokoro-82M TTS.
|
||||
#
|
||||
# Why this stack exists alongside the other TTS:
|
||||
# * Lowest-latency English in the fleet — ~300 ms TTFA on GPU,
|
||||
# RTF 35-100x on a 4060 Ti class card.
|
||||
# * Native streaming via OpenAI-compat `stream=true` over HTTP
|
||||
# chunked transfer (Kokoro's KPipeline is a per-phrase generator).
|
||||
# * Apache-2.0 weights + code; ~1 GB VRAM at fp16.
|
||||
# * 60+ built-in voices (no cloning — for that use IndexTTS-2 or
|
||||
# Chatterbox Turbo). Voices combinable via "voice(weight)+..." syntax.
|
||||
#
|
||||
# Image is a published GHCR build; no Dockerfile to maintain. Models
|
||||
# baked into the image, no first-run download. Deploy is a pull + up.
|
||||
#
|
||||
# All tunables live in .env — edit that, not this file.
|
||||
|
||||
services:
|
||||
kokoro:
|
||||
image: ghcr.io/remsky/kokoro-fastapi-gpu:${KOKORO_TAG}
|
||||
container_name: kokoro
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${KOKORO_BIND:-0.0.0.0}:${KOKORO_PORT}:8880"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${KOKORO_GPU_DEVICES:-0}
|
||||
- USE_GPU=true
|
||||
- API_LOG_LEVEL=${KOKORO_LOG_LEVEL:-INFO}
|
||||
volumes:
|
||||
# Optional voice-overlay mount — drop a custom <name>.pt into the
|
||||
# host dir to make it available alongside the 60+ built-ins. The
|
||||
# image already ships voicepacks at this path, so the bind mount
|
||||
# SHADOWS them — only do this if you actually want to manage the
|
||||
# full voice library yourself. For most deploys, leave the mount
|
||||
# commented out and use the in-image voices.
|
||||
# - ${KOKORO_VOICES_DIR}:/app/api/src/voices/v1_0
|
||||
- ${KOKORO_USER_VOICES_DIR}:/app/user_voices
|
||||
healthcheck:
|
||||
# The image is python-based with curl available. /v1/audio/voices
|
||||
# is a no-arg GET that exercises the full API path.
|
||||
test: ["CMD-SHELL", "curl -fsS -o /dev/null http://localhost:8880/v1/audio/voices || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 90s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=Kokoro
|
||||
- homepage.icon=mdi-microphone-message
|
||||
- homepage.description=Low-latency English TTS w/ streaming (irv-ml1)
|
||||
- homepage.href=http://10.100.79.3:${KOKORO_PORT}
|
||||
Reference in New Issue
Block a user