vh
83e5e941d8
Two fixes from the failed first deploy on irv-ml1:
1. CPU/GPU variant. Kokoro's GPU image needs CUDA >= 12.9; irv-ml1's
driver 570.124.06 caps at 12.8 so the gpu variant fails with
"nvidia-container-cli: requirement error: unsatisfied condition:
cuda>=12.9". Make the variant a knob:
KOKORO_VARIANT=cpu (default — works anywhere)
KOKORO_VARIANT=gpu (after driver bump)
KOKORO_USE_GPU=false|true (matches the variant)
Kokoro is tiny (82M params) so CPU is workable: TTFA ~1s vs ~300ms
on GPU. Acceptable while the driver bump gets scheduled. compose.yaml
no longer hard-codes `runtime: nvidia` — relies on the daemon's
default-runtime + NVIDIA_VISIBLE_DEVICES gating, same as how the
wrapper's USE_GPU flag selects the inference path inside the
container. Toggling between variants is now a `.env` edit + restart.
2. Tighter pull-log filter. --quiet on `docker compose pull` only
suppresses the pull command's stdout; the docker daemon still
emits per-layer extraction events on stderr ("ffbfd7a09415
Extracting 64.06MB" repeated dozens of times per layer). Drop those
too via grep on the SHA-prefixed pattern. set -o pipefail keeps a
real pull failure visible.
For existing deployments: removing /opt/docker/compose/kokoro/.env
on the host and rerunning the playbook re-seeds with the new schema.
57 lines
2.6 KiB
YAML
57 lines
2.6 KiB
YAML
# Kokoro-82M served via remsky/Kokoro-FastAPI — the de-facto OpenAI-
|
|
# compatible wrapper for hexgrad's Kokoro-82M TTS.
|
|
#
|
|
# Why this stack exists alongside the other TTS:
|
|
# * Lowest-latency English in the fleet — ~300 ms TTFA on GPU,
|
|
# RTF 35-100x on a 4060 Ti class card.
|
|
# * Native streaming via OpenAI-compat `stream=true` over HTTP
|
|
# chunked transfer (Kokoro's KPipeline is a per-phrase generator).
|
|
# * Apache-2.0 weights + code; ~1 GB VRAM at fp16.
|
|
# * 60+ built-in voices (no cloning — for that use IndexTTS-2 or
|
|
# Chatterbox Turbo). Voices combinable via "voice(weight)+..." syntax.
|
|
#
|
|
# Image is a published GHCR build; no Dockerfile to maintain. Models
|
|
# baked into the image, no first-run download. Deploy is a pull + up.
|
|
#
|
|
# All tunables live in .env — edit that, not this file.
|
|
|
|
services:
|
|
kokoro:
|
|
image: ghcr.io/remsky/kokoro-fastapi-${KOKORO_VARIANT:-cpu}:${KOKORO_TAG}
|
|
container_name: kokoro
|
|
restart: unless-stopped
|
|
# Only request GPU runtime when running the GPU variant. Toggling
|
|
# `runtime: nvidia` from a YAML knob isn't possible directly; we
|
|
# accomplish it by routing nvidia-only fields through the
|
|
# NVIDIA_VISIBLE_DEVICES env var instead. The cpu variant ignores
|
|
# that env var harmlessly; the gpu variant honors it.
|
|
ports:
|
|
- "${KOKORO_BIND:-0.0.0.0}:${KOKORO_PORT}:8880"
|
|
environment:
|
|
- NVIDIA_VISIBLE_DEVICES=${KOKORO_GPU_DEVICES:-}
|
|
- USE_GPU=${KOKORO_USE_GPU:-false}
|
|
- API_LOG_LEVEL=${KOKORO_LOG_LEVEL:-INFO}
|
|
volumes:
|
|
# Optional voice-overlay mount — drop a custom <name>.pt into the
|
|
# host dir to make it available alongside the 60+ built-ins. The
|
|
# image already ships voicepacks at this path, so the bind mount
|
|
# SHADOWS them — only do this if you actually want to manage the
|
|
# full voice library yourself. For most deploys, leave the mount
|
|
# commented out and use the in-image voices.
|
|
# - ${KOKORO_VOICES_DIR}:/app/api/src/voices/v1_0
|
|
- ${KOKORO_USER_VOICES_DIR}:/app/user_voices
|
|
healthcheck:
|
|
# The image is python-based with curl available. /v1/audio/voices
|
|
# is a no-arg GET that exercises the full API path.
|
|
test: ["CMD-SHELL", "curl -fsS -o /dev/null http://localhost:8880/v1/audio/voices || exit 1"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 90s
|
|
labels:
|
|
- homepage.group=AI Systems
|
|
- homepage.name=Kokoro
|
|
- homepage.icon=mdi-microphone-message
|
|
- homepage.description=Low-latency English TTS w/ streaming (irv-ml1)
|
|
- homepage.href=http://10.100.79.3:${KOKORO_PORT}
|