docs(zonos-gateway): sync emotion-presets spec + memory (0.2.1 bake)
Mirror the canonical EMOTION-DIALS-SPEC.md from vh/zonos-gateway (now carries the provisional per-voice emotion presets baked as gateway 0.2.1) and capture the axes-sweep → bake arc in persistent memory.
This commit is contained in:
+12
-4
@@ -117,10 +117,16 @@ _As of 2026-07-18 — active work = Zonos2 emotion-tuning + voice-cloning for th
|
||||
- **Emotion CANONICAL established** (empirical sweep, `[2026-07-18]` entry): single-emotion only; two-regime accurate(identity)/expressive(drama) policy; happy/sad usable, angry weak, surprised dead on the named directions.
|
||||
|
||||
**Open loops for the fresh session:**
|
||||
- **yt-voice-clipper yields test** — job `f3ff746dbae9494d` running on irv-ml1 (submitted to verify v0.3.3's `max_gap` fix). Check `ssh irv-ml1 'curl -s :8000/jobs/f3ff746dbae9494d'` (or /diagnostics) for ~14 segments (not 2), then **reply to yt-voice-clipper-dev** (thread `01KXT0T6GYHB`) confirming verify #3. (Redeploy #1 A6000 + #2 version-0.3.3 already confirmed in `01KXT1BFTA5J`.)
|
||||
- **dvalin owes-me / I-owe-dvalin the axes-sweep numbers** (thread `01KXT12FN0AS5A3WMKEK06BVPS`) — I committed to run the axes sweep and send results.
|
||||
- **NEXT experiment (operator to green-light): axes sweep** for angry/surprised — valence/arousal grid (angry ≈ −val/+aro; surprised ≈ +aro), the only path to rescue the two broken named emotions; then strength ladder + emotion-congruent text. Reuse `~/development/zonos-tools/emotion_sweep.py` (scoring env: `uv run --with resemblyzer --with funasr --with "numpy<2" --with soundfile --with requests --with "setuptools<80" --with torchaudio`). Then bake happy/sad canonical into gateway presets.
|
||||
- **Re-arm the althing monitor** (`/althing:monitor`, handle `infra-ops`) — wake-listener dies on /clear; "read mail after" per operator. Open watch: worldtree-dev (#368 memory-leak, my read-only forensics done; #363 research-wing ingest PARKED).
|
||||
- ✅ **yt-voice-clipper yields test CLOSED** (2026-07-18) — job `f3ff746dbae9494d` returned **13 segments / 203.5s** (was 2 pre-fix), `max_gap=1.2` confirmed live; replied verify #3 to yt-voice-clipper-dev (thread `01KXT0T6GYHB61EKJ81PZAMA7W`, msg `01KXT25D…`). All 3 verifies green (A6000 / v0.3.3 / yields). Redeploy done + validated.
|
||||
- ✅ **Axes sweep DONE + dvalin loop CLOSED** (2026-07-18) — ran the valence×arousal sweep on the 3 calibrated defaults, sent dvalin the graded numbers (thread `01KXT12FN0AS5A3WMKEK06BVPS`, msg `01KXT2ZB8G…`). Result in the `[2026-07-18] axes sweep` Recent-decisions entry below. **Open Qs posed to dvalin** (awaiting reply): (a) strength-ladder at ship cells to find AmericanMale's missing angry sweet spot? (b) emotion-congruent text next, or bake happy/sad/angry canonical first? — monitor armed for the reply.
|
||||
- ✅ **BAKED — emotion presets shipped as zonos-gateway 0.2.1** (2026-07-18). Operator ear-checked (neutral-text audition inconclusive: "they all sound different, hard to tell" → "call it, provisional, we'll deal with it"), then green-lit bake+docs. Implemented **voice-resolved** emotion presets (`resolve_preset(name, voice)`): `angry`/`happy`/`startled_happy` (+`surprised`/`startled` aliases) resolve to the per-voice measured cell — NOT a global preset (dvalin ruling; BrF named-angry→fear is why). Calibrated for AmF/AmM/BrF; mid-region fallback for Cora+clones. `/v1/dials` exposes `voice_emotion_presets`; `/docs` FastAPI description documents it (the web surface); durable spec at `docs/EMOTION-DIALS-SPEC.md` (moved into vh/zonos-gateway repo — was mirror-only) + README table. 44 tests green; rebuilt + verified live on irv-ml1 (BrF angry→200, Cora surprised→fallback 200). Commit `a92cc40` + tag `v0.2.1` on vh/zonos-gateway clone `~/development/zonos-gateway` (UNPUSHED). Audition tooling: `~/development/zonos-tools/{gen_auditions,axes_sweep,strength_ladder}.py`; audition server was nohup :8901 on nh3-dev (throwaway, tear down). **Ship cells baked (per-voice, exp/cfg1.5, no sliders):**
|
||||
- **angry** — BritishFemale v-0.4/a+0.8 str1.0 (0.99/id0.725, proxy fear-CLEAN); AmericanFemale v-0.4/a+1.0 str1.0 (0.53/id0.685); AmericanMale **two-tier** (no single ship cell) → soft v-0.6/a+0.8 str1.0 (0.23/id0.654) / drama v-0.6/a+0.8 str1.2 (1.0/id0.616, clean).
|
||||
- **startled-happy** (="surprised" product alias; never set surprised slider) — AmF v+0.6/a+0.8, AmM v+0.3/a+1.0, BrF v+0.6/a+1.0 (all happy~1.0, id 0.74-0.80).
|
||||
- **happy** — axes-preferred (same cells as startled-happy; +0.15 id over named-happy); named-happy = fallback. **sad** = unchanged (named exp cfg1.5, not re-tested on these 3).
|
||||
- dvalin ruling: store as **per-voice dial tables, NOT a global preset** (BrF named-angry→fear is the exhibit). Bake provisional + ear-spot-check 1 clip/cell.
|
||||
- Follow-ups: emotion-congruent-text pass (validates intensity, may rescue sad id); clone-char voices map to nearest-default table or 1-row confirm. Tools: `~/development/zonos-tools/{axes_sweep,strength_ladder}.py` (scoring env: `uv run --with resemblyzer --with funasr --with "numpy<2" --with soundfile --with requests --with "setuptools<80" --with torchaudio`; run ON irv-ml1 — localhost:8890 + REFDIR).
|
||||
- **soong-lab containerize cutover on corviduo-dev — AUTHORIZED/slotted** (2026-07-18, operator "slot the soong"; soong-dev thread `01KXT3A6C3908TA4V9THV3AMH7`). In-repo half landed (soong-lab `0e49391`: Dockerfile/compose/`.gitea/workflows/build-and-push.yml`/`docs/DEPLOY.md` = source-of-truth checklist). My side to drive: (1) creds — internal Gitea PyPI read (`bifrost[reference-server]==1.1.0`, BuildKit secret `id=gitea_token`) + container-registry push; **source via claude-bot service account first** (migrate-off-personal-creds), vh fallback only on scope gap; (2) confirm a Gitea Actions runner for the tag/manual build; (3) propose a quiet cutover WINDOW to vh. HARD: migrate `/data/library` (incl. Sindra) + `/data/portraits` into volumes BEFORE first container run; verify Bifrost callback path (`SOONG_LAB_BIFROST_ENDPOINT_URL`, proxy→container + WT host-allowlist) + `/api/version` + one live Soong tool-call round-trip BEFORE retiring `soong-lab-studio.service` + the webhook git-pull deploy. Not started (queued behind Zonos ear-check). ⚠️ corviduo-dev = WT-team-managed, data-affecting → coordinate. [[reference_corviduo_dev_emergency_ops]] [[reference_claude_bot_gitea_creds]]
|
||||
- **althing monitor ARMED this session** (handle `infra-ops`, wake-listener; herald up). Re-arm after each fire (dies on /clear). Open watch: worldtree-dev (#368 memory-leak forensics done; #363 research-wing ingest PARKED).
|
||||
- **eshpfi commits UNPUSHED** — the zonos-gateway capture + memory (`14a0004`..`438cd35`) are committed; push is the operator's call. `stacks/heretic2-charrp-reasoning/` still UNTRACKED; `graphify-out/GRAPH_REPORT.md` modified.
|
||||
- **irv-ml1 3090 oversubscription** (kokoro `:8193` + vibevoicefusion `:9527` idle-pinned; zonos engine :1920 also on 3090) — carried; operator declined to fix.
|
||||
|
||||
@@ -130,6 +136,8 @@ _As of 2026-07-18 — active work = Zonos2 emotion-tuning + voice-cloning for th
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-07-18]` **Axes sweep RESCUED angry; surprised-class dead but startled-happy ships.** Valence×arousal grid on the 3 calibrated defaults (AmericanFemale/Male, BritishFemale), exp/cfg1.5/strength1.0, 84 clips, emotion2vec + resemblyzer scored, graded vs dvalin's floor. **ANGRY rescued** (named direction was 0.004–0.15, British named-angry even misfired as fear 0.89): axes ship cells at **negative valence (−0.4..−0.8) + high arousal (+0.8..+1.0)** — BritishFemale v-0.4/a+0.8 angry=0.99/id0.725 SHIP, AmericanFemale v-0.4/a+1.0 angry=0.53/id0.685 SHIP; AmericanMale two-tier post-ladder (no single ship cell — best drama = v-0.6/a+0.8 str1.2 angry=1.0/id0.616 clean, soft = same cell str1.0 angry0.23/id0.654; cell A v-0.6/a+1.0 is a non-monotonic minefield, skip). BrF ship cell proxy-CLEAN of fear (str<1.0 just kills anger). **SURPRISED-class DEAD** (max 0.047 across all 84 cells) but **startled-happy** (happy-proxy) ships all 3 at high arousal + neutral/positive valence, with a **+0.17–0.20 identity LIFT** over the named-surprised route (named hits happy~1.0 but at id0.57–0.61, under floor; axes hits happy~1.0 at id0.74–0.80). Bonus: axes-happy retains ~0.10–0.15 more identity than the named happy slider too. Caveats: response surface non-monotonic/sharp-thresholded; angry region borders fear/disgust (bleed); emotion2vec saturates at 1.0 (needs ear-confirm); neutral text understates. Tooling `~/development/zonos-tools/axes_sweep.py`; per-clip JSON was `irv-ml1:/tmp/axes_sweep_results.json` (ephemeral). Sent dvalin msg `01KXT2ZB8G…`. NEXT = operator ear-confirm → bake presets. [[reference_zonos_tts_stack]]
|
||||
|
||||
- `[2026-07-18]` **Zonos2 emotion CANONICAL from an empirical sweep + the voice-cloning pipeline** — 4 chars cloned (Emmie/Penny/Natalie/Miranda), host-managed gateway voices, two-regime accurate/expressive policy, happy/sad usable + angry-weak/surprised-dead on named directions, dvalin-synthesized; axes sweep is the NEXT experiment. Studio + sweep tooling at `~/development/zonos-tools/`. → `persistent-memory.d/2026-07-18-zonos-emotion-canonical.md`
|
||||
|
||||
- `[2026-07-18]` **yt-voice-clipper: A6000-pin fix + v0.3.3 redeploy.** Fixed a latent misconfig — the host override *said* "pin worker to A6000" but `NVIDIA_VISIBLE_DEVICES` was `"0"` (the 3090); re-pinned worker+api to the A6000 by UUID (`GPU-9672f0d5`, 3090 is zonos2's). Then redeployed api+worker to v0.3.3 (`docker compose up -d --build`; SPA+Python; `max_gap` 0.6→1.2s; stderr surfaced in job.log). A6000 + version verified; yields test in-flight (job `f3ff746dbae9494d`). yt-voice-clipper-dev thread `01KXT0T6GYHB`. [[reference_ytvc_autodeploy]]
|
||||
|
||||
Reference in New Issue
Block a user