diff --git a/stacks/zonos-engine/.env.example b/stacks/zonos-engine/.env.example new file mode 100644 index 0000000..05bd610 --- /dev/null +++ b/stacks/zonos-engine/.env.example @@ -0,0 +1,23 @@ +# zonos-engine tunables — ZONOS2 native TTS engine (:1920, irv-ml1 3090). +# Copy to .env on the host. These feed both zonos2-server.sh (native) and the +# eventual container CMD/compose. Values captured from the live process 2026-07-16. + +# Upstream repo (github.com/Zyphra/Zonos2) commit to pin — reproducibility. +ZONOS_COMMIT=194c0a3 +ZONOS_MODEL=Zyphra/ZONOS2 + +# Endpoint +ZONOS_PORT=1920 + +# GPU — 3090 (irv-ml1 device 0). Operator 2026-07-16: keep OFF the A6000 (it OOMs +# under ComfyUI load). Container: NVIDIA_VISIBLE_DEVICES=0; native: the 3090 UUID. +ZONOS_ENGINE_GPU_ID=0 + +# Serving/perf flags (paged KV + cuda graphs; tuned to co-exist on the packed 3090). +ZONOS_CUDA_GRAPH_MAX_BS=1 +ZONOS_NUM_PAGES=16384 +ZONOS_MAX_RUNNING_REQUESTS=2 +ZONOS_MEMORY_RATIO=0.3 + +# HF weights cache (~15 GB, models--Zyphra--ZONOS2) — MOUNT, do not bake into the image. +HF_HOME=/home/lkraven/.cache/huggingface diff --git a/stacks/zonos-engine/README.md b/stacks/zonos-engine/README.md new file mode 100644 index 0000000..fe497f6 --- /dev/null +++ b/stacks/zonos-engine/README.md @@ -0,0 +1,59 @@ +# zonos-engine — ZONOS2 native TTS engine (`:1920`, irv-ml1 3090) + +The production Zonos TTS **engine** that `zonos-gateway` (`stacks/zonos`, `:8890`) +fronts via `ZONOS_URL=http://host.docker.internal:1920`, feeding asset-engine + +gateway-chat. This is the ENGINE; `stacks/zonos/` is the OpenAI-compatible ADAPTER. + +## Status (2026-07-16): NATIVE, config was live-only — now captured here + +The engine currently runs as a **bare native process** on irv-ml1 (launched +detached, reparented to init — **no systemd, won't survive a reboot**). This dir +captures its real config so it isn't lost, ahead of containerizing it (below). + +⚠️ **The old `~/tts-audition/harness/zonos_server.sh` on irv-ml1 is STALE** — it +says A6000 / port **1919** / no perf flags, but the live process runs on the +**3090** / port **1920** with the perf flags below. `zonos2-server.sh` here is the +CORRECTED, canonical invocation (verified against the live process's argv). + +## The engine + +- **Upstream (stock):** `github.com/Zyphra/Zonos2.git` @ commit **`194c0a3`**, cloned + at `~/tts-audition/models/zonos2` on irv-ml1 (uv-managed `.venv`). No custom PFI + server code — the `zonos2` package ships its own server (`python -m zonos2`). The + only "custom" part is the launch flags + this stack. +- **Stack:** torch **2.9.1+cu128** (CUDA 12.8); heavy compiled-CUDA deps + (flashinfer / sgl_kernel / cutlass-dsl / apache-tvm-ffi / pynini). `uv.lock` in the + repo pins the whole env deterministically. +- **GPU:** **3090** (irv-ml1 device 0, `GPU-12c15b3c-…`). Kept on the 3090 by operator + decision 2026-07-16 — the A6000 already OOMs under ComfyUI load, so it can't host + Zonos too. Footprint ~17 GB (the 3090 runs Zonos + chatterbox-fast + parakeet, ~1.4 GB free). +- **Model weights:** `Zyphra/ZONOS2`, ~15 GB HF cache at + `~/.cache/huggingface/hub/models--Zyphra--ZONOS2` — MOUNT this into the container, + do NOT bake it. +- **Endpoint:** `:1920` (`/tts/generate`), reused across prompts; emotion directions + auto-load from the repo's `emotion_directions/`. + +## Live invocation (source of truth) + +``` +uv run python -m zonos2 \ + --model-path Zyphra/ZONOS2 \ + --host 0.0.0.0 --port 1920 \ + --tts-default-voices-dir ./default_voices/ \ + --cuda-graph-max-bs 1 \ + --num-pages 16384 \ + --max-running-requests 2 \ + --memory-ratio 0.3 +``` + +## Containerization plan (pending build) + +Target shape = a self-contained image (fleet-standard, like `stacks/zonos` + +comfyui): `FROM` a CUDA 12.8 base → `uv sync` against the committed `uv.lock` (env +baked, reproducible) → mount the 15 GB HF weights + `default_voices/` → pin the 3090 +(`NVIDIA_VISIBLE_DEVICES=0`) → `restart: unless-stopped` → CMD = the invocation above. +Cutover = stop the native process (frees ~17 GB on the 3090) → `docker compose up -d` +(re-allocates ~17 GB, same footprint) → repoint `zonos-gateway` `ZONOS_URL` at the +container (or keep the `:1920` host-port publish). Brief prod-TTS blip at cutover. +Build risk = getting the compiled-CUDA deps to install in the base image (mostly +prebuilt wheels + the `uv.lock`, so likely a couple of build iterations). diff --git a/stacks/zonos-engine/zonos2-server.sh b/stacks/zonos-engine/zonos2-server.sh new file mode 100644 index 0000000..d7e92a4 --- /dev/null +++ b/stacks/zonos-engine/zonos2-server.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# CANONICAL launch invocation for the production ZONOS2 TTS engine (:1920, irv-ml1 3090). +# Captured 2026-07-16 from the LIVE process argv — supersedes the stale +# ~/tts-audition/harness/zonos_server.sh (which wrongly said A6000 / port 1919). +# This is the reference invocation the container CMD encodes; it also works as a +# direct native launcher until the container lands. +# +# Usage: +# zonos2-server.sh # foreground +# zonos2-server.sh --bg # detached, log to $ROOT/logs/zonos2_server.log +set -euo pipefail + +ROOT="${ZONOS_ROOT:-/home/lkraven/tts-audition}" +REPO="${ZONOS_REPO:-$ROOT/models/zonos2}" # github.com/Zyphra/Zonos2 @ 194c0a3 +GPU_3090_UUID="GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72" + +export CUDA_HOME="${CUDA_HOME:-/usr/local/cuda}" +export PATH="$CUDA_HOME/bin:$PATH" +export CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES:-$GPU_3090_UUID}" # 3090 (operator: keep off the A6000) + +cd "$REPO" +CMD=(uv run python -m zonos2 + --model-path "${ZONOS_MODEL:-Zyphra/ZONOS2}" + --host 0.0.0.0 --port "${ZONOS_PORT:-1920}" + --tts-default-voices-dir ./default_voices/ + --cuda-graph-max-bs "${ZONOS_CUDA_GRAPH_MAX_BS:-1}" + --num-pages "${ZONOS_NUM_PAGES:-16384}" + --max-running-requests "${ZONOS_MAX_RUNNING_REQUESTS:-2}" + --memory-ratio "${ZONOS_MEMORY_RATIO:-0.3}") + +if [[ "${1:-}" == "--bg" ]]; then + mkdir -p "$ROOT/logs" + nohup "${CMD[@]}" > "$ROOT/logs/zonos2_server.log" 2>&1 & + echo "zonos2 engine started (bg) pid $! -> $ROOT/logs/zonos2_server.log" +else + exec "${CMD[@]}" +fi