esh-pfi-infrastructure/stacks/kokoro/README.md

Kokoro

hexgrad/Kokoro-82M served via remsky/Kokoro-FastAPI.

Why this stack exists

Lowest-latency English TTS in the fleet by a wide margin — ~300 ms time-to-first-audio on GPU, 35-100x realtime, ~1 GB VRAM at fp16. Native streaming (Kokoro's KPipeline.__call__ is a per-phrase generator) and the wrapper exposes it via OpenAI-compat stream=true over chunked HTTP — drop-in for any OpenAI SDK client.

Complementary to the rest of the TTS slate:

	use case
Kokoro	low-latency English, fixed voice library
Chatterbox Turbo	low-latency English w/ voice cloning + paralinguistic tags
IndexTTS-2	English voice cloning + emotion vector / text control
Qwen3-TTS-1.7B-Base	high-quality English voice cloning
CosyVoice 3	multilingual (Chinese-leaning)
VibeVoice 1.5B	long-form podcast / multi-speaker dialogue

API

OpenAI-compat at http://10.100.79.3:8193:

# List built-in voices (~60 of them, named like af_bella, am_adam, jf_*, zf_*).
curl http://10.100.79.3:8193/v1/audio/voices

# Single-shot synthesis.
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"Hello there.","voice":"af_bella","response_format":"wav"}' \
  > out.wav

# Streaming — pipe straight into a player.
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"long passage here…","voice":"af_bella","stream":true}' \
  | mpv --no-cache -

# Voice mixing — sum voicepacks with weights.
curl -fsS -X POST http://10.100.79.3:8193/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"hello","voice":"af_bella(2)+af_heart(1)","response_format":"wav"}' \
  > mix.wav

Web UI at /web (browse voices + try in-place); OpenAPI at /docs.

Supported response_format: mp3 | wav | opus | flac | pcm.

Voices

• Built-in: 60+ in 8 languages (en-US, en-GB, ja, zh, es, fr, hi, it). Discoverable via GET /v1/audio/voices — naming convention is <lang_code><gender_letter>_<name> (e.g. af_bella, am_adam, jf_alpha, zf_xiaobei).

• Custom blends: weighted-average existing voicepacks into a new named voice using playbooks/blend-kokoro-voice.yaml (script: scripts/blend_kokoro_voice.py). Mirrors the wrapper's request-time voice="a(w)+b(w)" math but persists the result so it shows up in /v1/audio/voices. Example shipped: athena = af_bella(2)+af_aoede(1) normalized.

  scripts/elway irv-ml1 --playbook playbooks/blend-kokoro-voice.yaml \
    --var 'recipe=af_bella(1)+am_adam(1)' --var out_name=androgyne
  # add --var force=1 to overwrite an existing voice
  

  Persistence design: the wrapper enumerates exactly one directory (VOICES_DIR, default in-image /app/api/src/voices/v1_0). compose.yaml overrides VOICES_DIR=/app/user_voices (host bind mount /worktank/kokoro/user_voices) and adds a command: shim that cp -r's the in-image built-ins into that dir on every container start. Result:

  • built-in voicepacks re-seed on each start, so image upgrades that add/change built-ins propagate automatically
  • custom blends written by the script live on the host bind mount — survive docker restart, up --force-recreate, image upgrade, and host reboot
  • one-time prereq: the host dir must be chowned to 1001:1001 (uid of appuser inside the container) so the shim's cp can write — handled by playbooks/deploy-kokoro.yaml

• Cloned voices: training a Kokoro voice from samples is non- trivial — consult the hexgrad community for how-to. For voice cloning use Chatterbox Turbo or IndexTTS-2 instead.

Deploy

scripts/elway irv-ml1 --playbook playbooks/deploy-kokoro.yaml

First deploy: ~6.5 GB image pull from GHCR (~2-5 min on a fast link). No model download on first run — Kokoro-82M weights are baked in. Subsequent starts: a few seconds.

License

Apache-2.0 for both the wrapper code (remsky/Kokoro-FastAPI) and the Kokoro-82M weights (hexgrad).