Files

T

vh 4549d241a7 stacks: add Kokoro, VibeVoice 1.5B, Chatterbox Turbo (TTS slate fill-in)

Three TTS additions to round out coverage on irv-ml1, each filling a
distinct niche the existing slate doesn't own.

Final coverage matrix (all on irv-ml1):
  Kokoro              — low-latency English, fixed voice library, ~300ms TTFA
  Chatterbox Turbo    — low-latency English w/ voice cloning + paralinguistic tags
  IndexTTS-2          — English voice cloning + emotion vector / text control
  Qwen3-TTS-1.7B-Base — high-quality English voice cloning
  CosyVoice 3         — multilingual (Chinese-leaning)
  VibeVoice 1.5B      — long-form / multi-speaker dialogue

stacks/kokoro:
  - port 8193, GPU device 0 (3090)
  - pulls ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master (no Dockerfile,
    no first-run model download — models baked in)
  - 60+ built-in voices, OpenAI-compat with stream=true over chunked HTTP
  - Apache-2.0 weights + code, ~1 GB VRAM

stacks/vibevoice:
  - port 8194, GPU device 1 (A6000 — for 7B headroom)
  - builds groxaxo/VibeVoice-FastAPI1 (more current fork of ncoder-ai)
    pinned to 7614c469a145
  - default model microsoft/VibeVoice-1.5B (~7 GB bf16 VRAM); env var
    swap to rsxdalv/VibeVoice-Large (7B) or FabioSarracino/VibeVoice-Large-Q8
  - multi-speaker dialogue via /v1/vibevoice/generate with Speaker N: format
  - long-form niche only — not low-latency

stacks/chatterbox:
  - port 8196, GPU device 0 (3090)
  - builds devnen/Chatterbox-TTS-Server (most active Turbo-supporting wrapper)
  - default model ResembleAI/chatterbox-turbo (~2.5 GB fp16, ~75ms latency)
  - paralinguistic tags inline ([laugh] [whisper] etc) — different shape
    from IndexTTS-2's emotion vector; fills the speed+cloning niche
    Kokoro/IndexTTS don't cover together
  - mandatory PerTh watermark on outputs (Resemble policy)

Three matching playbooks under playbooks/deploy-{kokoro,vibevoice,
chatterbox}.yaml. All idempotent, creates-/when-gated.

Cold-deploy disk on /worktank/: ~7 GB Kokoro + ~19 GB VibeVoice 1.5B
+ ~12 GB Chatterbox = ~38 GB total. VRAM concurrent: ~10-11 GB across
both GPUs.

Skipped from the original four-stack proposal: VibeVoice Realtime
(overlaps Kokoro's niche; Kokoro wins on latency, license, and not
needing a build).

2026-04-25 16:18:37 -07:00

2.8 KiB

Raw Blame History

Kokoro

hexgrad/Kokoro-82M served via remsky/Kokoro-FastAPI.

Why this stack exists

Lowest-latency English TTS in the fleet by a wide margin — ~300 ms time-to-first-audio on GPU, 35-100x realtime, ~1 GB VRAM at fp16. Native streaming (Kokoro's KPipeline.__call__ is a per-phrase generator) and the wrapper exposes it via OpenAI-compat stream=true over chunked HTTP — drop-in for any OpenAI SDK client.

Complementary to the rest of the TTS slate:

	use case
Kokoro	low-latency English, fixed voice library
Chatterbox Turbo	low-latency English w/ voice cloning + paralinguistic tags
IndexTTS-2	English voice cloning + emotion vector / text control
Qwen3-TTS-1.7B-Base	high-quality English voice cloning
CosyVoice 3	multilingual (Chinese-leaning)
VibeVoice 1.5B	long-form podcast / multi-speaker dialogue

API

OpenAI-compat at http://10.100.79.3:8193:

# List built-in voices (~60 of them, named like af_bella, am_adam, jf_*, zf_*).
curl http://10.100.79.3:8193/v1/audio/voices

# Single-shot synthesis.
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"Hello there.","voice":"af_bella","response_format":"wav"}' \
  > out.wav

# Streaming — pipe straight into a player.
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"long passage here…","voice":"af_bella","stream":true}' \
  | mpv --no-cache -

# Voice mixing — sum voicepacks with weights.
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"hello","voice":"af_bella(2)+af_heart(1)","response_format":"wav"}' \
  > mix.wav

Web UI at /web (browse voices + try in-place); OpenAPI at /docs.

Supported response_format: mp3 | wav | opus | flac | pcm.

Voices

Built-in: 60+ in 8 languages (en-US, en-GB, ja, zh, es, fr, hi, it). Discoverable via GET /v1/audio/voices — naming convention is <lang_code><gender_letter>_<name> (e.g. af_bella, am_adam, jf_alpha, zf_xiaobei).
Custom: drop .pt voicepacks into /worktank/kokoro/user_voices/ on the host. Wrapper auto-discovers them on next request (no restart). Training Kokoro voices is non-trivial — consult the hexgrad community for how-to.

Deploy

scripts/elway irv-ml1 --playbook playbooks/deploy-kokoro.yaml

First deploy: ~6.5 GB image pull from GHCR (~2-5 min on a fast link). No model download on first run — Kokoro-82M weights are baked in. Subsequent starts: a few seconds.

License

Apache-2.0 for both the wrapper code (remsky/Kokoro-FastAPI) and the Kokoro-82M weights (hexgrad).

2.8 KiB Raw Blame History