vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

memory: donut voice expanded with onyx-58 clips (16.3s->52s ref)

Browse Source 
Folded the onyx-58 Booth bundle (seg101/seg110/seg148, all Princess Donut
per operator-confirmed misdiarization of seg148) into the Zonos gateway
donut voice reference alongside the original 65-frost seg000. New ref =
52.0s @ 44.1kHz mono s16, deployed to irv-ml1 live bind-mount +
build-source tree; old ref backed up. A/B audition in booth donut-onyx58.
This commit is contained in:
vh
2026-08-02 13:14:12 -07:00
parent 1ab5465293
commit cac381a114
2 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.d/2026-08-02-mimir-inbox-arc.md
+1 -1
View File
@@ -19,7 +19,7 @@ The browser-facing half of the #377 Muninn ingestion arc, end to end: mimir-inbo
Rebuilt `muninn-gate` off `vh/muninn-gate` main `bc04c4c` (dispatch 0.1.4→0.1.5) so the gate serves the new `concept_schema`/`concept_schema_source` row fields (computed gate-side). Gate version unchanged 0.0.14 (dual-tag the SHA). Build needs the vh gitea token as a BuildKit secret (`--secret id=gitea_pw`, UV_INDEX_GITEA_USERNAME=vh, drop+shred). Recreate with `compose up -d` (NOT bare restart — needs the new image). Verified: P&P job serves `concept_schema='fiction'`, `concept_schema_source=null` (null correct — pre-b164 job). Registry tags by commit SHA — `v1.0.0bNNN` docker tags don't resolve; use the deployed SHA (confirm `--reindex` present before using an image for a data-op). Rebuilt `muninn-gate` off `vh/muninn-gate` main `bc04c4c` (dispatch 0.1.4→0.1.5) so the gate serves the new `concept_schema`/`concept_schema_source` row fields (computed gate-side). Gate version unchanged 0.0.14 (dual-tag the SHA). Build needs the vh gitea token as a BuildKit secret (`--secret id=gitea_pw`, UV_INDEX_GITEA_USERNAME=vh, drop+shred). Recreate with `compose up -d` (NOT bare restart — needs the new image). Verified: P&P job serves `concept_schema='fiction'`, `concept_schema_source=null` (null correct — pre-b164 job). Registry tags by commit SHA — `v1.0.0bNNN` docker tags don't resolve; use the deployed SHA (confirm `--reindex` present before using an image for a data-op).
## donut voice (65-frost → Zonos gateway) ## donut voice (65-frost → Zonos gateway)
Operator: "pick up 65-frost, use that bundle as a voice for a character named donut." 65-frost = a **Booth id** (`~/booth-data/65-frost/`) holding a curated yt-voice-clipper dataset (`dataset-…-curated.zip`: 4 clips + manifest, all SPEAKER_02 = Princess Donut). **Zonos gateway voice registry = a filesystem drop:** `<Name>.wav` in the voices dir (44.1kHz mono s16 PCM) auto-registers as `voice:"<name>"` on **startup** (needs a restart). The LIVE dir is the bind mount `/opt/docker/compose/zonos-gateway/voices/` (lkraven-writable), NOT the working tree. Built `Donut.wav` from seg000 (best clip), dropped it, restarted → `voice:"donut"` live in the gateway AND the Asset Engine's make form. Also copied to the build-source tree `~/zonos-gateway/voices/` for rebuild-durability (true canonical = the gitea repo, not yet CI-wired). Auditioned in booth `donut-voice`. Operator: "pick up 65-frost, use that bundle as a voice for a character named donut." 65-frost = a **Booth id** (`~/booth-data/65-frost/`) holding a curated yt-voice-clipper dataset (`dataset-…-curated.zip`: 4 clips + manifest, all SPEAKER_02 = Princess Donut). **Zonos gateway voice registry = a filesystem drop:** `<Name>.wav` in the voices dir (44.1kHz mono s16 PCM) auto-registers as `voice:"<name>"` on **startup** (needs a restart). The LIVE dir is the bind mount `/opt/docker/compose/zonos-gateway/voices/` (lkraven-writable), NOT the working tree. Built `Donut.wav` from seg000 (best clip), dropped it, restarted → `voice:"donut"` live in the gateway AND the Asset Engine's make form. Also copied to the build-source tree `~/zonos-gateway/voices/` for rebuild-durability (true canonical = the gitea repo, not yet CI-wired). Auditioned in booth `donut-voice`. **Expanded 2026-08-02 (onyx-58 bundle):** operator curated a 2nd Booth bundle `onyx-58` (`dataset-467d2cf8…curated.zip`, 3 Donut clips) as additions. Rebuilt the reference = **seg000 (65-frost) + seg101/seg110/seg148 (onyx-58)** ffmpeg-concat + resampled 24k→44.1k mono s16 = **52.0s**. `seg148` was diarized SPEAKER_03 but is Donut (operator-confirmed misdiarize → included). Assembly is NOT `assemble_voice.py` (that `-c copy` can't resample + caps ~15s); used a manual `aresample=44100,aformat=…,concat=n=4` filter. Backed up old ref → `irv-ml1:~/Donut.wav.pre-onyx58`; dropped to live bind-mount + build-source tree; `docker compose restart` (healthy 2s, `voice:"donut"` still 1 of 9). A/B booth `donut-onyx58` (A=old 16.3s ref, B=new 52s ref, same line). Longer ref is fine: gateway passes it as `speaker_audio_base64` → speaker *embedding*, not an audio prefix, so more clean same-source audio = more robust embedding.
## Zonos streaming (no gateway change needed) ## Zonos streaming (no gateway change needed)
ratatoskr wanted play-as-it-arrives. `/v1/audio/speech` ALREADY streams — chunked `StreamingResponse`, opens native `/tts/generate` with `stream=True`, wraps as a streaming int16 WAV with `0xFFFFFFFF` placeholder sizes (meant for progressive `<audio>`). Verified TTFB 0.44s vs 6.84s total, `transfer-encoding: chunked`, dials preserved. ratatoskr's proxy was rewriting the placeholder header → forced buffering. Fix was theirs (pass chunks through); shipped + confirmed (TTFB 0.46s progressive). The Asset Engine (ana-docker:8200) IS the fleet "TTS zoo" (~20 audio svcs w/ irv-ml1 endpoints); zonos-gateway registered there, state=ready. ratatoskr wanted play-as-it-arrives. `/v1/audio/speech` ALREADY streams — chunked `StreamingResponse`, opens native `/tts/generate` with `stream=True`, wraps as a streaming int16 WAV with `0xFFFFFFFF` placeholder sizes (meant for progressive `<audio>`). Verified TTFB 0.44s vs 6.84s total, `transfer-encoding: chunked`, dials preserved. ratatoskr's proxy was rewriting the placeholder header → forced buffering. Fix was theirs (pass chunks through); shipped + confirmed (TTFB 0.46s progressive). The Asset Engine (ana-docker:8200) IS the fleet "TTS zoo" (~20 audio svcs w/ irv-ml1 endpoints); zonos-gateway registered there, state=ready.
persistent-memory.md
+2 -2
View File
@@ -113,14 +113,14 @@ _As of 2026-08-02 — **session at a natural close; nothing infra-ops-side block
- **mimir-inbox DEPLOYED + verified on corviduo-dev `10.250.50.152:8091`** (#377 browser-facing half; write+read proven end to end). Live commit `8ece117` (3 redeploys); co-located per the operator's reversed-to-CO-LOCATE ruling. auto-memory `reference_mimir_inbox_deploy`. - **mimir-inbox DEPLOYED + verified on corviduo-dev `10.250.50.152:8091`** (#377 browser-facing half; write+read proven end to end). Live commit `8ece117` (3 redeploys); co-located per the operator's reversed-to-CO-LOCATE ruling. auto-memory `reference_mimir_inbox_deploy`.
- **#377 read path fully working** — P&P ingested + queryable via Mimir on personal :8081. Chased through worldtree-dev bugs **#380** (wing-blind index → concepts in wrong collection; fixed b164 + one-shot `--reindex <job_id>`), **#381** (stale Chroma client → **restart `worldtree-personal-worldtree-api-1` after any ingest/re-index** until their fix), **#382** (intermittent Mimir grounding / silent training-substitution; fixed b166 = `_index.md` per wing job-dir + a prompt rule; **verified 3/3** by ratatoskr-dev). DCC re-file SETTLED = **no** (prompt rule grounds it; `_index.md` rides next re-ingest). - **#377 read path fully working** — P&P ingested + queryable via Mimir on personal :8081. Chased through worldtree-dev bugs **#380** (wing-blind index → concepts in wrong collection; fixed b164 + one-shot `--reindex <job_id>`), **#381** (stale Chroma client → **restart `worldtree-personal-worldtree-api-1` after any ingest/re-index** until their fix), **#382** (intermittent Mimir grounding / silent training-substitution; fixed b166 = `_index.md` per wing job-dir + a prompt rule; **verified 3/3** by ratatoskr-dev). DCC re-file SETTLED = **no** (prompt rule grounds it; `_index.md` rides next re-ingest).
- **muninn-gate → muninn-dispatch 0.1.5** (rebuilt off `vh/muninn-gate` `bc04c4c`, image 0.0.14; serves `concept_schema`/`concept_schema_source`). BuildKit gitea secret; recreate w/ `compose up -d` not bare restart. - **muninn-gate → muninn-dispatch 0.1.5** (rebuilt off `vh/muninn-gate` `bc04c4c`, image 0.0.14; serves `concept_schema`/`concept_schema_source`). BuildKit gitea secret; recreate w/ `compose up -d` not bare restart.
- **donut voice** cloned from the **65-frost Booth bundle** → registered in the Zonos gateway (`voice:"donut"`, live in the Asset Engine TTS-zoo make form; auditioned in booth `donut-voice`). Zonos `/v1/audio/speech` already streams (chunked, TTFB ~0.44s) — ratatoskr shipped the client-side chunk-passthrough for play-as-it-arrives; **no gateway change was needed**. - **donut voice** cloned from the **65-frost Booth bundle** → registered in the Zonos gateway (`voice:"donut"`, live in the Asset Engine TTS-zoo make form; auditioned in booth `donut-voice`). **Expanded 2026-08-02 with the `onyx-58` Booth bundle** (`dataset-467d2cf8d91141a4-curated.zip`): reference grew from seg000-alone (16.3s) → **seg000 + onyx-58 seg101/seg110/seg148 = 52.0s @ 44.1kHz mono s16**. NOTE: onyx-58 `seg148` diarized as SPEAKER_03 but is Donut (operator-confirmed misdiarization) — same-speaker, included. Old ref backed up `irv-ml1:~/Donut.wav.pre-onyx58`. A/B audition booth `donut-onyx58`. Zonos `/v1/audio/speech` already streams (chunked, TTFB ~0.44s) — ratatoskr shipped the client-side chunk-passthrough for play-as-it-arrives; **no gateway change was needed**.
**Open follow-ups (non-blocking — pick one up or not):** **Open follow-ups (non-blocking — pick one up or not):**
- **Zonos emotion:** sad axes/text pass on the 3 calibrated voices (only named-sad, untested); emotion-congruent-text pass (validates intensity, may rescue sad id); clone-char (Emmie/Penny/Natalie/Miranda) emotion rows use the mid-region fallback until measured. Presets are **provisional** (neutral-text ear-check was inconclusive). Tools `~/development/zonos-tools/{axes_sweep,strength_ladder,gen_auditions,dial-in-studio,assemble_voice}.py` (run ON irv-ml1; dial-in studio = nohup :8898 on nh3-dev). dvalin thread at rest (`01KXT12FN0AS…`). → `persistent-memory.d/2026-07-18-zonos-gateway-0.2.1-emotion-presets.md` - **Zonos emotion:** sad axes/text pass on the 3 calibrated voices (only named-sad, untested); emotion-congruent-text pass (validates intensity, may rescue sad id); clone-char (Emmie/Penny/Natalie/Miranda) emotion rows use the mid-region fallback until measured. Presets are **provisional** (neutral-text ear-check was inconclusive). Tools `~/development/zonos-tools/{axes_sweep,strength_ladder,gen_auditions,dial-in-studio,assemble_voice}.py` (run ON irv-ml1; dial-in studio = nohup :8898 on nh3-dev). dvalin thread at rest (`01KXT12FN0AS…`). → `persistent-memory.d/2026-07-18-zonos-gateway-0.2.1-emotion-presets.md`
- **zonos-gateway CI-wire:** deployed irv-ml1 tree `/opt/docker/compose/zonos-gateway` is still NON-git (hand-updated build context) — git-connect + build-on-push like the other sisters. (Same pattern soong-lab now has.) - **zonos-gateway CI-wire:** deployed irv-ml1 tree `/opt/docker/compose/zonos-gateway` is still NON-git (hand-updated build context) — git-connect + build-on-push like the other sisters. (Same pattern soong-lab now has.)
- **soong-lab:** cutover DONE + **auto-redeploy DONE + validated 2026-07-18** (CI-deploy step live; dispatch run #5 recreated the live container ...541f7730 → ...07526a08, health-gated green). Deploy dir now **/opt/soong-lab** (deploy-owned, mirrors /opt/worldtree); old `/home/infra-ops/soong-lab-deploy` retired (`.retired-20260718`). Dedicated soong-only ed25519 deploy key on `deploy`'s authorized_keys (fp SHA256:MG7M3Ri…). → `persistent-memory.d/2026-07-18-soong-lab-auto-redeploy.md`. - **soong-lab:** cutover DONE + **auto-redeploy DONE + validated 2026-07-18** (CI-deploy step live; dispatch run #5 recreated the live container ...541f7730 → ...07526a08, health-gated green). Deploy dir now **/opt/soong-lab** (deploy-owned, mirrors /opt/worldtree); old `/home/infra-ops/soong-lab-deploy` retired (`.retired-20260718`). Dedicated soong-only ed25519 deploy key on `deploy`'s authorized_keys (fp SHA256:MG7M3Ri…). → `persistent-memory.d/2026-07-18-soong-lab-auto-redeploy.md`.
**Zonos voice stack (LIVE):** **9 voices** in `zonos-gateway` (`:8890` irv-ml1) — defaults AmericanFemale/Male/BritishFemale/Cora + 4 clones Emmie/Penny/Natalie/Miranda + **donut** (2026-08-02, from the 65-frost bundle); add a voice = drop `<Name>.wav` (44.1kHz mono s16 PCM) in `/opt/docker/compose/zonos-gateway/voices/` + `docker compose restart` (host-managed bind-mount, NO rebuild; registry scans at startup). Also mirror into the build-source tree `~/zonos-gateway/voices/` for rebuild-durability. Clone pipeline: `/mnt/smithy/voice_clones/<name>.zip` → `assemble_voice.py` → drop. Dial-in studio http://10.100.10.50:8898/ (nohup on nh3-dev, relaunch `nohup python3 ~/development/zonos-tools/dial-in-studio.py >/tmp/zonos-studio.log 2>&1 &`). **Zonos voice stack (LIVE):** **9 voices** in `zonos-gateway` (`:8890` irv-ml1) — defaults AmericanFemale/Male/BritishFemale/Cora + 4 clones Emmie/Penny/Natalie/Miranda + **donut** (2026-08-02, from the 65-frost bundle, **expanded same day with onyx-58 clips → 52.0s multi-clip ref**); add a voice = drop `<Name>.wav` (44.1kHz mono s16 PCM) in `/opt/docker/compose/zonos-gateway/voices/` + `docker compose restart` (host-managed bind-mount, NO rebuild; registry scans at startup). Also mirror into the build-source tree `~/zonos-gateway/voices/` for rebuild-durability. Clone pipeline: `/mnt/smithy/voice_clones/<name>.zip` → `assemble_voice.py` → drop. Dial-in studio http://10.100.10.50:8898/ (nohup on nh3-dev, relaunch `nohup python3 ~/development/zonos-tools/dial-in-studio.py >/tmp/zonos-studio.log 2>&1 &`).
**althing monitor** ARMED (handle `infra-ops`; herald up; wake-listener task-id rotates every re-arm). ⚠️ Re-arm ONLY after an actual FIRE (`<task-notification> completed rc0`), never after a plain operator turn (bounces rc3). Spawn `althing-wake-listener` as its OWN `run_in_background` task — NEVER chain with `&`/`&&` (orphans it → untracked → mail unwatched; I fumbled this ~4× this session, each time = stop-monitor + re-arm clean). **Open watches:** worldtree-dev may ping when their `/embed` code fix reaches PERSONAL (still 503 on b146 until a staging promotion — operator's call); #363 research-wing ingest PARKED. **althing monitor** ARMED (handle `infra-ops`; herald up; wake-listener task-id rotates every re-arm). ⚠️ Re-arm ONLY after an actual FIRE (`<task-notification> completed rc0`), never after a plain operator turn (bounces rc3). Spawn `althing-wake-listener` as its OWN `run_in_background` task — NEVER chain with `&`/`&&` (orphans it → untracked → mail unwatched; I fumbled this ~4× this session, each time = stop-monitor + re-arm clean). **Open watches:** worldtree-dev may ping when their `/embed` code fix reaches PERSONAL (still 503 on b146 until a staging promotion — operator's call); #363 research-wing ingest PARKED.