vh
14a0004a47
The Zonos TTS engine that zonos-gateway fronts (irv-ml1 3090, feeds asset-engine + gateway-chat) ran as a bare native process with its real invocation existing ONLY in the running process argv — the committed harness/zonos_server.sh on irv-ml1 was STALE (said A6000/:1919, no perf flags; live is 3090/:1920 with cuda-graph/num-pages/ max-running-requests/memory-ratio). Captured the corrected canonical invocation + tunables + the containerization plan here so the config survives a process death. Engine = stock Zyphra/Zonos2 @ 194c0a3 (no custom PFI server code); torch 2.9.1+cu128; 15 GB HF weights. Next: containerize in-place on the 3090 (operator: keep off the A6000, it OOMs under ComfyUI). Not yet built — this commit is the config capture only.
60 lines
3.1 KiB
Markdown
60 lines
3.1 KiB
Markdown
# zonos-engine — ZONOS2 native TTS engine (`:1920`, irv-ml1 3090)
|
|
|
|
The production Zonos TTS **engine** that `zonos-gateway` (`stacks/zonos`, `:8890`)
|
|
fronts via `ZONOS_URL=http://host.docker.internal:1920`, feeding asset-engine +
|
|
gateway-chat. This is the ENGINE; `stacks/zonos/` is the OpenAI-compatible ADAPTER.
|
|
|
|
## Status (2026-07-16): NATIVE, config was live-only — now captured here
|
|
|
|
The engine currently runs as a **bare native process** on irv-ml1 (launched
|
|
detached, reparented to init — **no systemd, won't survive a reboot**). This dir
|
|
captures its real config so it isn't lost, ahead of containerizing it (below).
|
|
|
|
⚠️ **The old `~/tts-audition/harness/zonos_server.sh` on irv-ml1 is STALE** — it
|
|
says A6000 / port **1919** / no perf flags, but the live process runs on the
|
|
**3090** / port **1920** with the perf flags below. `zonos2-server.sh` here is the
|
|
CORRECTED, canonical invocation (verified against the live process's argv).
|
|
|
|
## The engine
|
|
|
|
- **Upstream (stock):** `github.com/Zyphra/Zonos2.git` @ commit **`194c0a3`**, cloned
|
|
at `~/tts-audition/models/zonos2` on irv-ml1 (uv-managed `.venv`). No custom PFI
|
|
server code — the `zonos2` package ships its own server (`python -m zonos2`). The
|
|
only "custom" part is the launch flags + this stack.
|
|
- **Stack:** torch **2.9.1+cu128** (CUDA 12.8); heavy compiled-CUDA deps
|
|
(flashinfer / sgl_kernel / cutlass-dsl / apache-tvm-ffi / pynini). `uv.lock` in the
|
|
repo pins the whole env deterministically.
|
|
- **GPU:** **3090** (irv-ml1 device 0, `GPU-12c15b3c-…`). Kept on the 3090 by operator
|
|
decision 2026-07-16 — the A6000 already OOMs under ComfyUI load, so it can't host
|
|
Zonos too. Footprint ~17 GB (the 3090 runs Zonos + chatterbox-fast + parakeet, ~1.4 GB free).
|
|
- **Model weights:** `Zyphra/ZONOS2`, ~15 GB HF cache at
|
|
`~/.cache/huggingface/hub/models--Zyphra--ZONOS2` — MOUNT this into the container,
|
|
do NOT bake it.
|
|
- **Endpoint:** `:1920` (`/tts/generate`), reused across prompts; emotion directions
|
|
auto-load from the repo's `emotion_directions/`.
|
|
|
|
## Live invocation (source of truth)
|
|
|
|
```
|
|
uv run python -m zonos2 \
|
|
--model-path Zyphra/ZONOS2 \
|
|
--host 0.0.0.0 --port 1920 \
|
|
--tts-default-voices-dir ./default_voices/ \
|
|
--cuda-graph-max-bs 1 \
|
|
--num-pages 16384 \
|
|
--max-running-requests 2 \
|
|
--memory-ratio 0.3
|
|
```
|
|
|
|
## Containerization plan (pending build)
|
|
|
|
Target shape = a self-contained image (fleet-standard, like `stacks/zonos` +
|
|
comfyui): `FROM` a CUDA 12.8 base → `uv sync` against the committed `uv.lock` (env
|
|
baked, reproducible) → mount the 15 GB HF weights + `default_voices/` → pin the 3090
|
|
(`NVIDIA_VISIBLE_DEVICES=0`) → `restart: unless-stopped` → CMD = the invocation above.
|
|
Cutover = stop the native process (frees ~17 GB on the 3090) → `docker compose up -d`
|
|
(re-allocates ~17 GB, same footprint) → repoint `zonos-gateway` `ZONOS_URL` at the
|
|
container (or keep the `:1920` host-port publish). Brief prod-TTS blip at cutover.
|
|
Build risk = getting the compiled-CUDA deps to install in the base image (mostly
|
|
prebuilt wheels + the `uv.lock`, so likely a couple of build iterations).
|