# Zonos Emotion Control — Dials-First Spec

**Status:** canonical direction (operator ruling 2026-07-17) + **empirically-
measured per-voice emotion presets baked provisional (2026-07-18).** **Applies
to:** `zonos-gateway` (`:8890`) + its consumers (asset-engine, gateway-chat, any
future client). **Supersedes:** the preset-centric usage pattern.

> **Two layers, not a contradiction.** The philosophy is dials-first (§1): twist
> the raw dials per utterance. The 2026-07-18 presets (§5) are the *measured
> answer* for the emotions naive twisting can't hit — anger (whose named
> direction misfires) and surprise (dead as a class). They are empirically-tuned
> dial-sets exposed under a name, per voice; you can still twist your own.

## 1. Philosophy — twist the dials, don't pick a mood

Emotion is set **per-utterance by twisting the raw dials**, not by choosing from
a fixed menu of named presets. The dials ARE the engine's canonical interface
(Zyphra/ZONOS2 exposes exactly these; presets were a PFI-only convenience layer,
not upstream). Rationale:

- **Expressiveness.** A scene needs "wry, tired, a little fond" — not `sad`. An
  LLM client picks fine-grained affect per line far better than N fixed moods.
- **Explicit over implicit.** A dial-set states the actual emotion in the call;
  a preset name hides it behind an opaque label we alone define.
- **No curation burden.** Presets are a layer we'd have to keep tuning; they
  already drifted overwrought (`excited`/`intense` cfg 1.6–1.8 chipmunked with
  nobody re-tuning them). Dials have no drift and no maintenance surface.

**The flavor presets (`warm`/`excited`/`intense`/`whisper`) stay demoted to
optional copy-and-tweak examples** (§6). The *emotion* presets in §5 are a
different thing — an empirical result, not a hand-tuned convenience.

## 2. The dials (`POST /v1/audio/speech`)

All emotion dials require `emotion_enabled: true` — it is the master switch;
without it every `emotion_*` field is inert.

| Dial | Range | Default | What it does | Guidance |
|---|---|---|---|---|
| `emotion_enabled` | bool | `false` | Master switch for emotion conditioning. | Set `true` to use any dial below. |
| `emotion_sliders` | `{happy\|sad\|angry\|surprised: w}` | `null` | Per-emotion weights (the 4 named directions). Blend by setting several. | Weights ~`0.4–0.8`. **`surprised` raises pitch — the chipmunk driver; use sparingly or omit.** Named `angry` is WEAK/misfires — prefer the axes preset (§5). |
| `emotion_valence` | `−1.0 … +1.0` | `0.0` | Pleasant ↔ unpleasant axis. | Continuous affect for states between the 4 named emotions. The lever behind §5. |
| `emotion_arousal` | `−1.0 … +1.0` | `0.0` | Calm ↔ energized axis. | `+` = excited/anxious, `−` = tired/subdued. High arousal + `surprised` = chipmunk. |
| `emotion_strength` | float | `1.0` | Multiplier on the **per-speaker-calibrated** direction. `1.0` = calibrated; `>1` exaggerates. | Keep `~0.7–1.1`. Above `~1.3` strains most voices; NOT a smooth knob (§5 caveat 4). |
| `emotion_cfg_scale` | `≥ 1.0` | `1.0` | Classifier-free guidance on emotion. `1.0` = off. `>1` amplifies. | **See §3 — the "deaf by 1.5" dial.** |
| `accurate_mode` | bool | `true` | `true` = faithful to the reference voice; `false` = looser / more expressive. | `false` is REQUIRED for emotion to land (accurate mode suppresses it); the cost is some identity drift. |
| `speaking_rate_enabled` | bool | `false` | Master switch for rate conditioning. | Needed for `speaking_rate` / `speaking_rate_bucket` / `speed` to bite. |
| `speaking_rate_bucket` | int `0–7` | `null` | Exact rate bucket (`0`=slowest … `7`=fastest; wps bands from `/tts/capabilities`). | Fast = excited/anxious; slow = sad/tired. Pair with the emotion, don't overdrive. |

Discovery is live: `GET /v1/dials` self-describes emotions, axes, rate buckets,
per-dial ranges, **and the per-voice emotion presets** (the `voice_emotion_presets`
block).

## 3. `emotion_cfg_scale` — the knob goes to 30, but you're deaf by 1.5

**There is deliberately NO enforced cap.** The dial will take large values — set
it to 30 if you want. You will regret it. Emotional realism tops out around
**1.5**; past ~1.4 the voice is overwrought and distorted, and by 1.5 you're
"deaf" — pushing harder only adds artifact, not feeling.

- **Usable range: `1.0`–`1.5`.** Recommended ceiling **`1.4`**. **Never above 1.5.**
- `1.0` = off: the additive emotion directions alone (already calibrated). Plenty
  expressive for most lines.
- `~1.25–1.4` = the sweet spot when you want a line to land harder.
- `> 1.5` = do not. It's the amp that goes to 11 — the number is bigger, the
  sound is worse.

The §5 emotion presets use cfg `1.5` deliberately — expressive mode needs the
push for the axes emotion to land; it's the top of the usable band, not past it.

## 4. Cost / real-time — a second reason to stay ≤ 1.5

Measured on the 3090 (irv-ml1):

| `emotion_cfg_scale` | RTF | Note |
|---|---|---|
| `1.0` (off) | ~0.52 | **free** — same as neutral; additive vectors, no extra forward pass |
| `1.5` | ~0.625 | ~+20% wall (CFG doubles the *decode* pass), still comfortably real-time |
| `> 1.5` | worse | more compute AND worse sound — strictly dominated |

## 5. Empirically-measured per-voice emotion presets (PROVISIONAL, 2026-07-18)

An empirical sweep (valence×arousal grid, scored by an emotion classifier +
speaker-identity retention) established that the four named emotions do **not**
respond uniformly to naive dial-twisting:

- **happy / sad** — work via the named sliders (already usable).
- **angry** — the *named* `angry` slider is weak-to-broken (on BritishFemale it
  misfires as **fear**). Driving the **valence/arousal axes** instead (negative
  valence + high arousal) rescues it.
- **surprised** — **dead as an emotion class** on this engine (never activates,
  any dial). The usable stand-in is **startled-happy** (high arousal + positive
  valence), which reads as bright surprise.

The winning cell is **different per voice**, so a single global preset is unsafe.
The gateway therefore exposes emotion presets that **resolve against the voice**:

```
POST /v1/audio/speech  { "input": "...", "voice": "BritishFemale", "preset": "angry" }
```

### Preset names

| preset | resolves to | aliases |
|---|---|---|
| `angry` | axes anger cell for the voice | — |
| `happy` | axes happy cell (better identity than the named happy slider) | — |
| `startled_happy` | the "surprised" product stand-in | `surprised`, `startled` |
| `sad` | named-slider preset (unchanged; not axes-tuned yet) | — |
| `neutral`/`warm`/`excited`/`intense`/`whisper` | voice-independent flavor presets (§6) | — |

### Calibrated cells (the 3 default voices)

All expressive (`accurate_mode:false`), `emotion_cfg_scale:1.5`, pure-axes (no
sliders). `emo` = emotion-classifier target prob, `id` = speaker-identity
retention (floor 0.65; neutral ~0.85).

| Voice | preset | valence | arousal | strength | emo | id |
|---|---|---|---|---|---|---|
| AmericanFemale | `angry` | −0.4 | +1.0 | 1.0 | 0.53 | 0.69 |
| AmericanFemale | `happy` / `startled_happy` | +0.6 | +0.8 | 1.0 | 1.0 | 0.80 |
| AmericanMale | `angry` (drama) | −0.6 | +0.8 | 1.2 | 1.0 | 0.62 |
| AmericanMale | `happy` / `startled_happy` | +0.3 | +1.0 | 1.0 | 0.98 | 0.76 |
| BritishFemale | `angry` | −0.4 | +0.8 | 1.0 | 0.99 | 0.73 |
| BritishFemale | `happy` / `startled_happy` | +0.6 | +1.0 | 1.0 | 1.0 | 0.74 |

**Uncalibrated voices** (Cora + cloned characters) fall back to a mid-region
default (`angry` = v−0.5/a+0.9/str1.1; `happy`/`startled_happy` = v+0.5/a+0.9)
until they earn a measured row. The live table is in `GET /v1/dials` →
`voice_emotion_presets`.

### Caveats (why "provisional")

1. **Non-monotonic surface** — the exact `(valence, arousal, strength)` is
   pinned; do NOT interpolate a preset's neighborhood in a UI.
2. **Anger costs identity** — retention ~0.62–0.73 vs ~0.85 neutral. Acceptable
   for a drama beat, not for identity-critical dialogue.
3. **Surprised-as-class is dead** — never expose a "surprised" *slider* promise;
   map user-intent surprised/startled/shocked to the `startled_happy` preset.
4. **`emotion_strength` is not a smooth knob** (esp. AmericanMale: 1.0 = mild,
   1.2 = full anger, higher flips to *disgust*). The `angry` preset pins the
   voice's tuned strength; send an explicit `emotion_strength` to shift tier
   (e.g. `1.0` on AmericanMale for a softer anger).
5. **Two-regime policy** — identity-critical lines: `accurate_mode:true`,
   emotion off or soft. Tagged drama beats: these presets. Don't stack a high
   named slider AND a high axes preset (single-condition wins).

Baked provisional pending confident ear-validation on emotion-congruent text
(neutral-sentence auditing was inconclusive — the dials clearly differ, but a
flat line understates landing). Re-tune when we revisit.

## 6. Starting-point flavor dial-sets (examples — copy and tweak)

Tuned by ear on BritishFemale. Departure points, not a menu. (Distinct from §5:
these are hand-tuned flavors, not measured emotion cells.)

```jsonc
// warm  — friendly, unhurried
{ "emotion_enabled": true, "emotion_sliders": {"happy": 0.4},
  "emotion_valence": 0.4, "emotion_strength": 0.8, "emotion_cfg_scale": 1.3 }

// sad — slow, quiet
{ "emotion_enabled": true, "emotion_sliders": {"sad": 0.8},
  "emotion_valence": -0.6, "emotion_arousal": -0.5, "emotion_strength": 1.0,
  "emotion_cfg_scale": 1.5, "speaking_rate_enabled": true, "speaking_rate_bucket": 1 }

// whisper — hushed, breathy
{ "emotion_enabled": true, "emotion_sliders": {"sad": 0.2},
  "emotion_arousal": -0.7, "emotion_strength": 0.9, "emotion_cfg_scale": 1.4,
  "speaking_rate_enabled": true, "speaking_rate_bucket": 2 }
```

## 7. LLM client system-prompt snippet

Paste into an LLM client that drives TTS, so it emits a dial-set per utterance:

> You control speech emotion per line. For **anger** or **surprise/startle**, use
> a preset: `{"preset": "angry"}` or `{"preset": "startled_happy"}` (the named
> `angry`/`surprised` sliders misfire — don't use them). For everything else,
> emit a dial-set: set `emotion_enabled: true`, then shape delivery with
> `emotion_sliders` (happy/sad ~0–0.8), `emotion_valence`/`emotion_arousal`
> (−1..+1), `emotion_strength` (~0.7–1.1), and `emotion_cfg_scale`
> (**1.0–1.4, never above 1.5**). Add `speaking_rate_enabled: true` +
> `speaking_rate_bucket` (0 slow … 7 fast) for pacing. Subtle by default, bold
> only when the moment earns it.

## 8. Voice cloning (reference audio)

Cloning is text-independent (Qwen3 speaker embedding) — **no transcript needed.**
Give it **~15–24s of clean, single-speaker audio** (Zyphra's own default voices
run 8.7–23.9s). Shorter clips (≤5s) under-condition the embedding and audibly
degrade, worse under emotion steering. A cloned voice has **no calibrated
emotion row** yet — it uses the §5 fallback; add a measured row (or map it to the
nearest default) when its emotions matter.

## 9. Follow-ups (not done here)

- **Ear-validation on congruent text** — the §5 cells were baked provisional off
  an inconclusive neutral-text audition; confirm on emotion-appropriate lines and
  promote from provisional (or re-tune).
- **Sad axes pass** — sad was not in the axes sweep; still the named-slider
  preset. Owes a per-voice text/strength pass on the 3 defaults.
- **AmericanMale angry sweet-spot** — no single cell clears both the emo and
  identity floors; it ships as drama (str 1.2) with a documented soft override
  (str 1.0). A finer strength ladder could find a middle tier.
- **Clone-character rows** — Emmie/Penny/Natalie/Miranda use the fallback; run a
  1-row confirm at the baked cell per voice, or map to the nearest default.
- **CI / deploy wiring** — the deployed irv-ml1 tree is a hand-updated build
  context (not git / not CI-deployed). Git-connect + build-on-push with a deploy
  key, like the other sister services.