stacks/kokoro: cpu/gpu variant toggle + tighter pull-log filter
Two fixes from the failed first deploy on irv-ml1:
1. CPU/GPU variant. Kokoro's GPU image needs CUDA >= 12.9; irv-ml1's
driver 570.124.06 caps at 12.8 so the gpu variant fails with
"nvidia-container-cli: requirement error: unsatisfied condition:
cuda>=12.9". Make the variant a knob:
KOKORO_VARIANT=cpu (default — works anywhere)
KOKORO_VARIANT=gpu (after driver bump)
KOKORO_USE_GPU=false|true (matches the variant)
Kokoro is tiny (82M params) so CPU is workable: TTFA ~1s vs ~300ms
on GPU. Acceptable while the driver bump gets scheduled. compose.yaml
no longer hard-codes `runtime: nvidia` — relies on the daemon's
default-runtime + NVIDIA_VISIBLE_DEVICES gating, same as how the
wrapper's USE_GPU flag selects the inference path inside the
container. Toggling between variants is now a `.env` edit + restart.
2. Tighter pull-log filter. --quiet on `docker compose pull` only
suppresses the pull command's stdout; the docker daemon still
emits per-layer extraction events on stderr ("ffbfd7a09415
Extracting 64.06MB" repeated dozens of times per layer). Drop those
too via grep on the SHA-prefixed pattern. set -o pipefail keeps a
real pull failure visible.
For existing deployments: removing /opt/docker/compose/kokoro/.env
on the host and rerunning the playbook re-seeds with the new schema.
This commit is contained in:
@@ -17,15 +17,19 @@
|
||||
|
||||
services:
|
||||
kokoro:
|
||||
image: ghcr.io/remsky/kokoro-fastapi-gpu:${KOKORO_TAG}
|
||||
image: ghcr.io/remsky/kokoro-fastapi-${KOKORO_VARIANT:-cpu}:${KOKORO_TAG}
|
||||
container_name: kokoro
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
# Only request GPU runtime when running the GPU variant. Toggling
|
||||
# `runtime: nvidia` from a YAML knob isn't possible directly; we
|
||||
# accomplish it by routing nvidia-only fields through the
|
||||
# NVIDIA_VISIBLE_DEVICES env var instead. The cpu variant ignores
|
||||
# that env var harmlessly; the gpu variant honors it.
|
||||
ports:
|
||||
- "${KOKORO_BIND:-0.0.0.0}:${KOKORO_PORT}:8880"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${KOKORO_GPU_DEVICES:-0}
|
||||
- USE_GPU=true
|
||||
- NVIDIA_VISIBLE_DEVICES=${KOKORO_GPU_DEVICES:-}
|
||||
- USE_GPU=${KOKORO_USE_GPU:-false}
|
||||
- API_LOG_LEVEL=${KOKORO_LOG_LEVEL:-INFO}
|
||||
volumes:
|
||||
# Optional voice-overlay mount — drop a custom <name>.pt into the
|
||||
|
||||
Reference in New Issue
Block a user