esh-pfi-infrastructure/stacks/zonos-gateway/README.md

# zonos-gateway — OpenAI-compatible TTS gateway over stock ZONOS2 (`:8890`, irv-ml1)

The production entry point for Zonos TTS. A thin FastAPI service that exposes a
clean OpenAI `POST /v1/audio/speech` surface and drives the **stock ZONOS2
engine's** correct native `POST /tts/generate` path — while adding the
**emotion-preset system** and voice defaults. Fronts `stacks/zonos-engine`
(the ZONOS2 engine at `:1920`); consumers reach it through a LiteLLM alias.

- **Server:** irv-ml1 (Irvine, WireGuard-only), 3090.
- **Deploy:** `/opt/docker/compose/zonos-gateway/compose.yaml` (fleet convention).
- **Image:** `zonos-gateway:0.2.0`, `build: .` from the source tree.
- **Source (versioned 2026-07-17):** [`vh/zonos-gateway`](http://10.250.50.70:3000/vh/zonos-gateway)
  on gitea (private; internal SSH `ssh://git@10.250.50.70:222/vh/zonos-gateway.git`).
  The dials-first emotion spec lives there at `docs/EMOTION-DIALS-SPEC.md`. This
  eshpfi dir mirrors the *stack* (compose + env + docs). **Note:** the deployed
  working tree on irv-ml1 (`~/zonos-gateway`) was imported as the initial commit
  but is not yet git-connected/CI-wired — future edits go through the repo, then
  deploy to irv-ml1 (CI + deploy key is the open follow-up, like the other
  sister services).

## Why it exists

ZONOS2 ships its own OpenAI `/v1/audio/speech`, but that path has two bugs:

1. **Repetition-penalty off by default** → short prompts pad **~70 s of silence**
   instead of stopping at end-of-speech.
2. **The `voice` field is ignored** → a random speaker every call.

ZONOS2's **native `POST /tts/generate` is correct** (stock rep-penalty window
50 / penalty 1.2 / codebooks 8 stop the model at end-of-speech; speaker honored;
streams float32 PCM @ 44.1 kHz). This gateway drives that good native path and
leaves the vendored engine **100% stock** — no upstream patch to carry.

## Architecture — the `ext-tts` alias is the swap point

```
consumers ──▶  LiteLLM alias  ext-tts  ──▶  zonos-gateway  ──▶  ZONOS2 /tts/generate (stock)
               (generic, engine-neutral)     (:8890, this stack)   (:1920, stacks/zonos-engine)
```

Consumers **never** call this service by name — they call the generic LiteLLM
alias **`ext-tts`**, so the backing engine can be hot-swapped later with a
one-line alias repoint, invisible to every consumer. This repo stays
Zonos-specific (voice registry, correct native defaults, presets); the swap
point is the alias, not the code.

## Emotion control

> **Direction (operator ruling 2026-07-17): dials-first.** Emotion is set by
> twisting the raw dials per-utterance; presets are demoted to optional
> copy-and-tweak examples, not the primary interface. See
> [`EMOTION-DIALS-SPEC.md`](EMOTION-DIALS-SPEC.md) for the dial vocabulary, the
> `emotion_cfg_scale` "deaf by 1.5" rule (no cap, documented ceiling), the
> LLM-client system-prompt snippet, and reference-voice guidance. The
> preset-centric description below is retained as background.

### (background) the preset path

Emotion is exposed as **named presets** that bundle the engine's steering knobs
(`emotion_sliders` / `emotion_valence` / `emotion_arousal` / `emotion_strength`
/ `emotion_cfg_scale` + `speaking_rate`), defined in `src/zonos_gateway/dials.py`.
The simplest caller request:

```bash
curl -s http://<host>:8890/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"input":"All the world'\''s a stage.","voice":"Cora","preset":"excited"}'
```

- **Presets (2026-07-17):** `neutral`, `warm`, `excited`, `sad`, `intense`, `whisper`.
- **Discovery:** `GET /v1/dials` self-describes voices / emotions
  (`happy` `sad` `angry` `surprised`) / axes (`valence` `arousal`) / presets.
- **Full manual control** is also accepted on `/v1/audio/speech`
  (`emotion_enabled`, `emotion_sliders`, `emotion_valence/arousal`,
  `emotion_strength`, `emotion_cfg_scale`, `accurate_mode`, `speaking_rate*`,
  `seed`, samplers) for fine-tuning or new presets.

### Real-time cost (measured on the 3090, 2026-07-17)

| Mode | RTF | Note |
|---|---|---|
| neutral (no steering) | ~0.52 | baseline |
| calibrated steering (`emotion_cfg_scale` 1.0) | ~0.52 | **free** — additive direction vectors, no extra forward pass |
| amplified (`emotion_cfg_scale` 1.5) | ~0.625 | ~+20% wall (CFG doubles the *decode* pass only), **still realtime** |

Emotion is effectively free at cfg 1.0 and cheap (~+20%, RTF still ~0.6) at
cfg 1.5–1.6 (where `warm`/`excited`/`intense` sit). No loss of real-time; the
only cost is ~20% less aggregate throughput under heavy concurrency at high cfg.

## API

| Method + path | Purpose |
|---|---|
| `POST /v1/audio/speech` | OpenAI-style TTS; accepts `preset` or manual emotion/prosody knobs |
| `GET /v1/dials` | Self-describe voices / emotions / axes / presets |
| `GET /v1/voices` | Available speaker voices |
| `GET /health` | Liveness |

## Relationship to the other Zonos stacks

- **`stacks/zonos-engine`** — the ZONOS2 engine (`:1920`) this gateway fronts.
- **`stacks/zonos`** — **DEAD / superseded.** That stack is Zonos **v0.1**
  (Gradio UI + a v0.1 SDK adapter); it is no longer deployed. The live path is
  ZONOS2 `:1920` (engine) + this gateway `:8890`.

## Emotion research (2026-07-17)

Text-based prosody-priming (prime → generate → excise) was **flat** on this
checkpoint. The **native emotion-steering subsystem** (this gateway's presets)
is the working lever — clearly moves delivery (sad → slow/quiet, excited →
fast/bright, etc.). `whisper` is a composed preset (no `whisper` *direction*
exists; the stock directions are happy/sad/angry/surprised + valence/arousal).
Escalation path for affects the 4 directions can't reach = custom directions
(`scripts/build_emotion_directions.py` in the engine repo) or emotional
reference-audio cloning (`speaker_audio_base64`).