From ba0ec64ac3f87c029f6f8cb062037a954ca0c447 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 17 Jul 2026 15:56:11 -0700 Subject: [PATCH] docs(zonos-gateway): capture the live :8890 gateway stack + emotion-preset system MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Captures the production Zonos TTS path that the repo was blind to: stock ZONOS2 :1920 engine (stacks/zonos-engine) fronted by zonos-gateway:0.2.0 :8890, reached via the LiteLLM ext-tts alias. Documents the emotion-preset system (neutral/warm/ excited/sad/intense/whisper, the simple preset: caller path), the API, and the measured real-time cost (calibrated steering free at RTF~0.52, cfg1.5 ~0.625 — still realtime). Flags the gateway source (~/zonos-gateway on irv-ml1) as not yet in gitea. Marks stacks/zonos (v0.1 Gradio) dead/superseded. Records the emotion- lever finding (text-priming flat -> native steering works) in persistent memory. --- persistent-memory.md | 2 + stacks/zonos-gateway/.env.example | 18 ++++++ stacks/zonos-gateway/README.md | 101 ++++++++++++++++++++++++++++++ stacks/zonos-gateway/compose.yaml | 29 +++++++++ stacks/zonos/README.md | 7 +++ 5 files changed, 157 insertions(+) create mode 100644 stacks/zonos-gateway/.env.example create mode 100644 stacks/zonos-gateway/README.md create mode 100644 stacks/zonos-gateway/compose.yaml diff --git a/persistent-memory.md b/persistent-memory.md index 51562ef..9399f9f 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -130,6 +130,8 @@ _As of 2026-07-17 — active work is CONTAINERIZING the Zonos2 TTS engine + prot ## Recent decisions +- `[2026-07-17]` **Zonos emotion levers RESOLVED: text-priming is FLAT → the working lever is ZONOS2's native emotion-steering, which the gateway ALREADY exposes as presets.** The prosody-priming A/B (prime→generate→excise, silence-gap cut, parakeet-validated) was operator-judged FLAT on this checkpoint — text doesn't move it. Native `emotion_directions/` (happy/sad/angry/surprised + valence/arousal axes, per-speaker calibrated for AmericanFemale/Male/British) clearly WORKS (sad→slow/quiet, excited→fast/bright, etc.). **`zonos-gateway:0.2.0` (:8890) already wires it**: simplest caller path = `POST /v1/audio/speech {preset:"…"}` — presets neutral/warm/excited/sad/intense/whisper (defined in `~/zonos-gateway/src/zonos_gateway/dials.py`), reached via the **LiteLLM `ext-tts` alias** (engine-neutral swap point; consumers never call the gateway by name). RTF measured on 3090: cfg1.0 steering = FREE (~0.52 = neutral, additive vectors), cfg1.5 amplified ~0.625 (~+20%, still realtime). Captured the live gateway stack → `stacks/zonos-gateway/` (compose+env+README); ⚠️ gateway SOURCE at `~/zonos-gateway` on irv-ml1 is NOT in gitea (backup gap, follow-up); `stacks/zonos` (v0.1 Gradio) marked DEAD/superseded. Whisper is a composed preset (no whisper *direction*; escalation for hard affects = custom directions via `scripts/build_emotion_directions.py` or emotional-ref cloning `speaker_audio_base64`). Harnesses in scratchpad (not yet landed). [[reference_zonos_tts_stack]] + - `[2026-07-17]` **Zonos2 :1920 → self-contained container (stays on 3090); prosody-priming is adapter-level, engine stays stock.** Config captured (14a0004, unpushed); build = cu128 base + `uv sync` vs the lock + weights mount; priming = prime→generate-one-utterance→parakeet-clip→deliver in the gateway adapter. Crux = does AR prosody carry the sentence boundary (A/B the join). → `persistent-memory.d/2026-07-17-zonos2-containerize-prosody-priming.md` - `[2026-07-16]` **GPU re-org: char-rp→GPU1 + both cards re-optimized for max context.** Moved char-rp (Magidonia-24B) GPU0→GPU1, then maxed context: char-rp-reasoning 150K→256K (util 0.46, 1.56x), gen→256K + seqs 16→32 (util 0.42, 5.43x), granite 64K→**128K full-chapter** (util 0.27, 1.50x). FINAL: GPU0 ~14 G reserve (both seats 256K native), GPU1 ~6.7 G headroom. All healthy. LESSON: KV must hold ≥1× max-len (util-floor crashes) + per-model KV cost varies ~8× (MoE cheap, dense pricey) → tune util empirically. See Current state for the full layout + backups. diff --git a/stacks/zonos-gateway/.env.example b/stacks/zonos-gateway/.env.example new file mode 100644 index 0000000..02a8770 --- /dev/null +++ b/stacks/zonos-gateway/.env.example @@ -0,0 +1,18 @@ +# zonos-gateway tunables — copy to .env on the host and override as needed. +# All values have inline compose defaults (${VAR:-default}); the live deploy +# currently runs with NO .env (pure defaults). This file documents the knobs. + +# Where the stock Zonos2 engine's native API lives. Default = same-host :1920 +# (host.docker.internal). For a non-colocated engine, point at its IP: +# ZONOS_URL=http://10.100.79.3:1920 +ZONOS_URL=http://host.docker.internal:1920 + +# Gateway listen port (host + container). +PORT=8890 + +# Default speaker when a request omits `voice`. Must exist in the engine's +# speaker registry (GET :8890/v1/voices). Live default = Cora. +DEFAULT_VOICE=Cora + +# Upstream request timeout (seconds) — generous for long inputs. +REQUEST_TIMEOUT_S=600 diff --git a/stacks/zonos-gateway/README.md b/stacks/zonos-gateway/README.md new file mode 100644 index 0000000..84c4f31 --- /dev/null +++ b/stacks/zonos-gateway/README.md @@ -0,0 +1,101 @@ +# zonos-gateway — OpenAI-compatible TTS gateway over stock ZONOS2 (`:8890`, irv-ml1) + +The production entry point for Zonos TTS. A thin FastAPI service that exposes a +clean OpenAI `POST /v1/audio/speech` surface and drives the **stock ZONOS2 +engine's** correct native `POST /tts/generate` path — while adding the +**emotion-preset system** and voice defaults. Fronts `stacks/zonos-engine` +(the ZONOS2 engine at `:1920`); consumers reach it through a LiteLLM alias. + +- **Server:** irv-ml1 (Irvine, WireGuard-only), 3090. +- **Deploy:** `/opt/docker/compose/zonos-gateway/compose.yaml` (fleet convention). +- **Image:** `zonos-gateway:0.2.0`, `build: .` from the source tree. +- **⚠️ Source:** lives at `~/zonos-gateway` on irv-ml1 (`src/zonos_gateway/`, + its own `pyproject.toml` + `ROADMAP.md` + `persistent-memory.md`) and is **NOT + yet in gitea** — a real version-control/backup gap. This eshpfi dir mirrors + the *stack* (compose + env + docs), not the source; the source needs its own + gitea repo (follow-up). + +## Why it exists + +ZONOS2 ships its own OpenAI `/v1/audio/speech`, but that path has two bugs: + +1. **Repetition-penalty off by default** → short prompts pad **~70 s of silence** + instead of stopping at end-of-speech. +2. **The `voice` field is ignored** → a random speaker every call. + +ZONOS2's **native `POST /tts/generate` is correct** (stock rep-penalty window +50 / penalty 1.2 / codebooks 8 stop the model at end-of-speech; speaker honored; +streams float32 PCM @ 44.1 kHz). This gateway drives that good native path and +leaves the vendored engine **100% stock** — no upstream patch to carry. + +## Architecture — the `ext-tts` alias is the swap point + +``` +consumers ──▶ LiteLLM alias ext-tts ──▶ zonos-gateway ──▶ ZONOS2 /tts/generate (stock) + (generic, engine-neutral) (:8890, this stack) (:1920, stacks/zonos-engine) +``` + +Consumers **never** call this service by name — they call the generic LiteLLM +alias **`ext-tts`**, so the backing engine can be hot-swapped later with a +one-line alias repoint, invisible to every consumer. This repo stays +Zonos-specific (voice registry, correct native defaults, presets); the swap +point is the alias, not the code. + +## Emotion control — the simple caller path (presets) + +Emotion is exposed as **named presets** that bundle the engine's steering knobs +(`emotion_sliders` / `emotion_valence` / `emotion_arousal` / `emotion_strength` +/ `emotion_cfg_scale` + `speaking_rate`), defined in `src/zonos_gateway/dials.py`. +The simplest caller request: + +```bash +curl -s http://:8890/v1/audio/speech -H 'Content-Type: application/json' \ + -d '{"input":"All the world'\''s a stage.","voice":"Cora","preset":"excited"}' +``` + +- **Presets (2026-07-17):** `neutral`, `warm`, `excited`, `sad`, `intense`, `whisper`. +- **Discovery:** `GET /v1/dials` self-describes voices / emotions + (`happy` `sad` `angry` `surprised`) / axes (`valence` `arousal`) / presets. +- **Full manual control** is also accepted on `/v1/audio/speech` + (`emotion_enabled`, `emotion_sliders`, `emotion_valence/arousal`, + `emotion_strength`, `emotion_cfg_scale`, `accurate_mode`, `speaking_rate*`, + `seed`, samplers) for fine-tuning or new presets. + +### Real-time cost (measured on the 3090, 2026-07-17) + +| Mode | RTF | Note | +|---|---|---| +| neutral (no steering) | ~0.52 | baseline | +| calibrated steering (`emotion_cfg_scale` 1.0) | ~0.52 | **free** — additive direction vectors, no extra forward pass | +| amplified (`emotion_cfg_scale` 1.5) | ~0.625 | ~+20% wall (CFG doubles the *decode* pass only), **still realtime** | + +Emotion is effectively free at cfg 1.0 and cheap (~+20%, RTF still ~0.6) at +cfg 1.5–1.6 (where `warm`/`excited`/`intense` sit). No loss of real-time; the +only cost is ~20% less aggregate throughput under heavy concurrency at high cfg. + +## API + +| Method + path | Purpose | +|---|---| +| `POST /v1/audio/speech` | OpenAI-style TTS; accepts `preset` or manual emotion/prosody knobs | +| `GET /v1/dials` | Self-describe voices / emotions / axes / presets | +| `GET /v1/voices` | Available speaker voices | +| `GET /health` | Liveness | + +## Relationship to the other Zonos stacks + +- **`stacks/zonos-engine`** — the ZONOS2 engine (`:1920`) this gateway fronts. +- **`stacks/zonos`** — **DEAD / superseded.** That stack is Zonos **v0.1** + (Gradio UI + a v0.1 SDK adapter); it is no longer deployed. The live path is + ZONOS2 `:1920` (engine) + this gateway `:8890`. + +## Emotion research (2026-07-17) + +Text-based prosody-priming (prime → generate → excise) was **flat** on this +checkpoint. The **native emotion-steering subsystem** (this gateway's presets) +is the working lever — clearly moves delivery (sad → slow/quiet, excited → +fast/bright, etc.). `whisper` is a composed preset (no `whisper` *direction* +exists; the stock directions are happy/sad/angry/surprised + valence/arousal). +Escalation path for affects the 4 directions can't reach = custom directions +(`scripts/build_emotion_directions.py` in the engine repo) or emotional +reference-audio cloning (`speaker_audio_base64`). diff --git a/stacks/zonos-gateway/compose.yaml b/stacks/zonos-gateway/compose.yaml new file mode 100644 index 0000000..c5b448a --- /dev/null +++ b/stacks/zonos-gateway/compose.yaml @@ -0,0 +1,29 @@ +# zonos-gateway — deploys on irv-ml1 next to the Zonos server. +# Follows the fleet convention: /opt/docker/compose/zonos-gateway/compose.yaml, +# tunables in an adjacent .env (this file stays constant). +services: + zonos-gateway: + build: . + image: zonos-gateway:0.2.0 + container_name: zonos-gateway + restart: unless-stopped + # Colocated with Zonos: reach the host's :1920 from inside the container. + # host-gateway resolves host.docker.internal to the host's bridge IP. + extra_hosts: + - "host.docker.internal:host-gateway" + environment: + # Default targets the Zonos server on the same host. For a non-colocated + # or dev target, override in .env (e.g. ZONOS_URL=http://10.100.79.3:1920). + ZONOS_URL: ${ZONOS_URL:-http://host.docker.internal:1920} + PORT: ${PORT:-8890} + DEFAULT_VOICE: ${DEFAULT_VOICE:-Cora} + VOICES_DIR: /app/voices + REQUEST_TIMEOUT_S: ${REQUEST_TIMEOUT_S:-600} + ports: + - "${PORT:-8890}:8890" + healthcheck: + test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8890/health', timeout=5).status==200 else 1)"] + interval: 30s + timeout: 8s + retries: 3 + start_period: 15s diff --git a/stacks/zonos/README.md b/stacks/zonos/README.md index 577d815..c36e0f8 100644 --- a/stacks/zonos/README.md +++ b/stacks/zonos/README.md @@ -1,5 +1,12 @@ # Zonos +> **⚠️ SUPERSEDED / NOT DEPLOYED (2026-07-17).** This stack is Zonos **v0.1** +> (Gradio UI + a v0.1 SDK adapter) and is **no longer running** anywhere on the +> fleet. The live Zonos TTS path is **ZONOS2** — see `stacks/zonos-engine` +> (the `:1920` engine) + `stacks/zonos-gateway` (the `:8890` OpenAI-compatible +> gateway with the emotion-preset system, reached via the LiteLLM `ext-tts` +> alias). Kept for history; do not deploy. + Zyphra's expressive multilingual open-weight TTS ([Zyphra/Zonos](https://github.com/Zyphra/Zonos)) — 44 kHz output, zero-shot voice cloning, and explicit emotion/conditioning controls —