vh
83e5e941d8
Two fixes from the failed first deploy on irv-ml1:
1. CPU/GPU variant. Kokoro's GPU image needs CUDA >= 12.9; irv-ml1's
driver 570.124.06 caps at 12.8 so the gpu variant fails with
"nvidia-container-cli: requirement error: unsatisfied condition:
cuda>=12.9". Make the variant a knob:
KOKORO_VARIANT=cpu (default — works anywhere)
KOKORO_VARIANT=gpu (after driver bump)
KOKORO_USE_GPU=false|true (matches the variant)
Kokoro is tiny (82M params) so CPU is workable: TTFA ~1s vs ~300ms
on GPU. Acceptable while the driver bump gets scheduled. compose.yaml
no longer hard-codes `runtime: nvidia` — relies on the daemon's
default-runtime + NVIDIA_VISIBLE_DEVICES gating, same as how the
wrapper's USE_GPU flag selects the inference path inside the
container. Toggling between variants is now a `.env` edit + restart.
2. Tighter pull-log filter. --quiet on `docker compose pull` only
suppresses the pull command's stdout; the docker daemon still
emits per-layer extraction events on stderr ("ffbfd7a09415
Extracting 64.06MB" repeated dozens of times per layer). Drop those
too via grep on the SHA-prefixed pattern. set -o pipefail keeps a
real pull failure visible.
For existing deployments: removing /opt/docker/compose/kokoro/.env
on the host and rerunning the playbook re-seeds with the new schema.
95 lines
3.5 KiB
YAML
95 lines
3.5 KiB
YAML
# Deploy Kokoro-FastAPI to irv-ml1.
|
|
#
|
|
# Image is published on GHCR — no Dockerfile to maintain, no first-run
|
|
# model download (Kokoro-82M weights are baked in). Stage compose +
|
|
# .env, pull, bring up. ~6.5 GB pull on cold cache, ~2-5 min.
|
|
#
|
|
# Usage:
|
|
# scripts/elway irv-ml1 --playbook playbooks/deploy-kokoro.yaml
|
|
#
|
|
# Idempotent — every step is creates-/when-gated; rerun is safe.
|
|
|
|
vars:
|
|
compose_dir: /opt/docker/compose/kokoro
|
|
user_voices_dir: /worktank/kokoro/user_voices
|
|
voices_dir: /worktank/kokoro/voices
|
|
host_port: "8193"
|
|
|
|
steps:
|
|
# ── host-side dirs ──────────────────────────────────────────────────
|
|
|
|
- name: Ensure /worktank/kokoro root exists (one-time, sudo)
|
|
shell: mkdir -p /worktank/kokoro
|
|
sudo: true
|
|
creates: /worktank/kokoro
|
|
|
|
- name: Chown /worktank/kokoro to lkraven
|
|
shell: chown lkraven:lkraven /worktank/kokoro
|
|
sudo: true
|
|
when: '[ "$(stat -c %U /worktank/kokoro)" != lkraven ]'
|
|
|
|
- name: Ensure user-voices dir exists
|
|
shell: mkdir -p {{ user_voices_dir }}
|
|
creates: "{{ user_voices_dir }}"
|
|
|
|
- name: Ensure voices dir exists (used only if compose mount is enabled)
|
|
shell: mkdir -p {{ voices_dir }}
|
|
creates: "{{ voices_dir }}"
|
|
|
|
- name: Ensure compose dir exists
|
|
shell: mkdir -p {{ compose_dir }}
|
|
creates: "{{ compose_dir }}"
|
|
|
|
# ── deploy compose files ────────────────────────────────────────────
|
|
|
|
- name: Upload compose.yaml
|
|
upload:
|
|
src: stacks/kokoro/compose.yaml
|
|
dest: "{{ compose_dir }}/compose.yaml"
|
|
mode: "0644"
|
|
|
|
- name: Seed .env from template (only if absent)
|
|
upload:
|
|
src: stacks/kokoro/.env.example
|
|
dest: "{{ compose_dir }}/.env"
|
|
mode: "0644"
|
|
when: "[ ! -f {{ compose_dir }}/.env ]"
|
|
|
|
# ── pull + bring up ─────────────────────────────────────────────────
|
|
|
|
- name: "docker compose pull (first run: ~6.5 GB from GHCR)"
|
|
# --quiet suppresses pull-command progress, but the docker daemon
|
|
# still emits its own per-layer extraction progress events
|
|
# ("ffbfd7a09415 Extracting 64.06MB" etc) on stderr — these are
|
|
# what flooded the log on the previous run. Drop them with grep.
|
|
# set -o pipefail so a real pull failure isn't swallowed.
|
|
shell: |
|
|
set -o pipefail
|
|
cd {{ compose_dir }} && docker compose pull --quiet 2>&1 \
|
|
| grep -vE '^\s*[0-9a-f]{8,16}\s+(Extracting|Pull complete|Pulling fs layer|Verifying Checksum|Download complete|Waiting|Already exists|Downloading|Pulling from)'
|
|
|
|
- name: docker compose up -d
|
|
shell: cd {{ compose_dir }} && docker compose up -d
|
|
|
|
- name: Wait for /v1/audio/voices to respond
|
|
shell: |
|
|
for i in $(seq 1 60); do
|
|
curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/v1/audio/voices && exit 0
|
|
sleep 5
|
|
done
|
|
exit 1
|
|
changed_when: "false"
|
|
|
|
verify:
|
|
- name: /v1/audio/voices returns 200
|
|
shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/v1/audio/voices
|
|
changed_when: "false"
|
|
|
|
- name: /v1/audio/voices includes at least one built-in (af_bella)
|
|
shell: curl -sf http://localhost:{{ host_port }}/v1/audio/voices | grep -q af_bella
|
|
changed_when: "false"
|
|
|
|
- name: Container is running
|
|
shell: docker inspect kokoro --format '{{.State.Status}}' | grep -q running
|
|
changed_when: "false"
|