memory: snapshot — chatterbox-fast streaming build (adaptive buffer-ratchet chunking); native-turbo-streaming abandoned; fish reference_id fix + glados transcript + NFS/smithy + references-togglable; add design doc

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — eshpfi-management
 
-_Last updated: 2026-06-01_
+_Last updated: 2026-06-02_
 
 ## Repo purpose
 
@@ -87,8 +87,34 @@ Sister repos (separate gitea repos, deployed by playbooks here):
 
 ## Current state / in-flight
 
-_As of 2026-06-01:_
+_As of 2026-06-02:_
 
+- **PRIMARY FOCUS — building `chatterbox-fast`, a custom streaming TTS
+  container; Chatterbox is becoming our MAIN TTS engine.** Operator-authorized
+  high-effort build (incl. custom container from source). **Plan-of-attack:
+  `/tmp/chatterbox-fast-plan.md`** (write/refresh before /clear — it carries
+  the full executable detail). Design doc: `docs/design/chatterbox-fast.md`.
+  - **Goal:** cut time-to-first-audio from ~2.5s → sub-second via streaming,
+    keep turbo quality. Workload = single-stream interactive.
+  - **Chosen approach = adaptive buffer-ratchet chunking** (operator's idea):
+    generate the FIRST sentence alone for instant response (~0.66s first-audio
+    measured), then while it plays, generate the largest sentence-aligned chunk
+    that fits in ~80% of the bought playback time; chunks grow ~3× each (driven
+    by Chatterbox's ~3.8× realtime headroom), so after 2-3 chunks the rest of
+    the paragraph is one big chunk with near-full context. Context loss confined
+    to 2-3 joins at natural sentence-pause boundaries. THIS is "sentence-level
+    done right" and supersedes naive per-sentence splitting (which loses
+    cross-sentence prosodic context = real quality loss, operator-corrected).
+    Only works because RTF>1 (Fish at <1× realtime starves — can't use this).
+  - **Native frame-level streaming on turbo = ABANDONED** (see Tried/abandoned).
+  - **Also to build for A/B:** base-chatterbox + davidbrowne17 streaming fork
+    (true frame-stream but base-model quality) — not yet installed.
+  - **A/B samples** (GLaDOS voice) on nh3-dev `~/chatterbox-ab/`: sentence-level
+    turbo, chunked-native (artifacty), chunked-oneshot.
+  - **Dev/test pattern:** one-off GPU-1 container from `local/chatterbox:v1`,
+    mount `/worktank/chatterbox/{cache,reference_audio}` + the test script;
+    `docker exec -i chatterbox python -` for lib introspection. lkraven is in
+    the `docker` group on irv-ml1 — NO sudo needed for docker.
 - **TTS fleet on irv-ml1 reworked this session; asset-engine catalog now
   17 services** (CSM removed). Canonical `docs/asset-engine/services.yaml`
   + vendored `vh/asset-engine` both pushed (eshpfi `38d9e3b`; asset-engine
@@ -109,15 +135,16 @@ _As of 2026-06-01:_
     API). Research/non-commercial license.
   - **zonos (:8203)** down (adapter built, not deployed). **ComfyUI :8188**
     catalog-deferred link.
-- **BLOCKED on asset-engine-dev — P1 "undefined" select bug** (default voice
-  picks send the literal string `"undefined"` → Chatterbox 404s, Fish falls to
-  default voice). ROOT-CAUSED: the form's Kokoro voice-blend widget
-  (`templates/partials/_fields.html`) runs `recompute()` at page load, reading
-  the Shoelace `<sl-select>.value` BEFORE hydration → writes "undefined" into
-  the submit field. Escalated with full root cause + 3-part fix (althing msg
-  `01KT2K2SY9N7AY69R9V0B4RXSW` → asset-engine-dev). **Workaround until fixed:
-  click the dropdown (fires change) or hit the API directly.** My half
-  (`blendable: false` catalog flag) is queued — see Recent decisions.
+- **"undefined" select bug — RESOLVED.** asset-engine-dev shipped form-select
+  hardening (v0.1.9/.10) + a durable **per-field enable-toggle** (`togglable`,
+  v0.1.14/.16) — the real fix for the "form submits untouched fields" family.
+  My catalog half: opted fish-s2 `references` into `togglable: true`
+  (`catalog_version 1→2`, schema gained the `togglable` boolean; `c98a12b`,
+  pushed to origin). The earlier `blendable` idea was superseded by `togglable`.
+  dia2/chatterbox clone fields deliberately NOT toggled (dia2 defaults clone-on
+  as its stable voice). asset-engine CI was briefly red because the commit was
+  local-only until pushed — lesson: push catalog commits promptly (their CI
+  drift-checks against the remote).
 - **Fish "not British" had TWO independent root causes — BOTH now fixed.**
   The ECAPA-TDNN re-test (2026-06-01) showed Fish's cloning *engine* works: an
   Imogen reference scores **~0.79 cosine vs real `Imogen.wav`** vs **~0.10 vs
@@ -147,9 +174,19 @@ _As of 2026-06-01:_
   (CC BY 4.0, accent-tagged speaker IDs), Unmute voice-donations (CC0), EARS +
   expresso (CC BY-NC). Source for future clone voices. British-female Southern
   England speakers p225/p228/p229 staged into Fish as Imogen/Eleanor/Beatrice.
-- **Parakeet ASR (:8765) is DOWN** — no container, but catalog says
-  `status: ready` (drift). Use Whisper-in-a-container (`faster-whisper`) for
-  transcripts instead; it's self-contained (CTranslate2, no torch).
+- **Parakeet ASR (:8765) is now UP** (brought online 2026-06-02 to transcribe
+  the glados clip; operator wants it kept online). CPU-only (`gpu_device_id:
+  null`), `restart=unless-stopped`, healthy. API: POST multipart `file` to
+  `/transcribe` → `{"text":...}`. Image `local/parakeet:sherpa-onnx-v2`
+  pre-built; `docker compose up -d` in `/opt/docker/compose/parakeet`.
+- **NFS share for Smithy — DONE.** `/volume1/smithy` on nh3-nas → `/mnt/smithy`
+  on nh3-dev (single export, RW, map-to-admin, scoped to nh3-dev only, fstab
+  `_netdev,nofail,x-systemd.automount`). Subdirs `datasets/{raw,manifest-store,
+  derived,holdout,quarantine}`; rename-atomicity verified. Smithy wired
+  storage-roots.yaml + E2E-verified. **Backup DEFERRED** per operator (datasets
+  regenerable; raw + manifest-store are the irreplaceable surface for when it's
+  wired — durable tier should land OFF nh3-nas/cross-site, since the NH3 restic
+  repo lives on the same NAS as the source).
 - **Worldtree healthy v0.29.13** (last-known); **Skaldsong v0.32.2**
   (ana-docker:8300, Kokoro SSE streaming); **artemis-31b-v1i** live on
   llama-swap + worldtree personal; **ttyd fleet driver seat** on nh3-dev
@@ -168,6 +205,29 @@ _As of 2026-06-01:_
 
 ## Recent decisions
 
+- `[2026-06-02]` **Chatterbox → main TTS engine; build custom `chatterbox-fast`
+  streaming container.** Workload = single-stream interactive. **GPU placement:
+  3090 (device 0) if it fits else A6000 (device 1)** — shared dev stack, 20.5 GB
+  3090-idle is expected residency, not a blocker. **Cutover: parallel catalog
+  entry**, burn in beside live `chatterbox`, then flip. **Streaming approach:
+  adaptive buffer-ratchet chunking** (see in-flight). Native frame-streaming
+  abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast.md` +
+  `/tmp/chatterbox-fast-plan.md`.
+
+- `[2026-06-02]` **Sentence-splitting loses quality (operator-corrected).** I
+  claimed naive sentence-level streaming has "zero quality loss" — WRONG. The
+  T3 AR backbone conditions prosody on the WHOLE text; splitting loses
+  cross-sentence prosodic context (contextual delivery, declination, affect
+  continuity) even though voice timbre stays (reference-conditioned). No
+  *artifacts* ≠ no *quality loss*. Hence the adaptive-chunk design (maximize
+  context per chunk subject to latency budget), not fixed per-sentence splits.
+
+- `[2026-06-01]` **Fish reference_id empty-dir fix shipped** (`c5bbb90`) — see
+  in-flight + Tried/abandoned. Populated `references/<name>/<name>.wav`+`.lab`
+  for all 32 voices; playbook gained normalize-step + A/B smoke gate. glados got
+  a real transcript (ASR'd via Parakeet): the Portal "Welcome to test chamber 4"
+  lines.
+
 - `[2026-06-01]` **Fish cloning VERIFIED competent (ECAPA-TDNN)** — retracting
   the earlier "weak cloner" call. Isolated test: Imogen-referenced clone ~0.79
   cosine to the real `Imogen.wav` vs ~0.10 for the no-reference default;
@@ -262,6 +322,32 @@ _25 older entries archived to archival-memory.md._
 
 ## Tried and abandoned
 
+- `[2026-06-02]` **Native frame-level streaming on Chatterbox-TURBO — ABANDONED
+  (turbo isn't built for streaming).** Long R&D arc; record so it's not
+  re-derived. (1) The model's flow is CosyVoice2-derived but `S3GenStreamer` is
+  referenced-in-docstring-only (not implemented). (2) The lib's
+  `flow_inference(finalize=False)` is BUGGY: the lookahead trim removes
+  `pre_lookahead_len(3)*token_mel_ratio(2)=6` frames from `h` but NOT from
+  `h_masks`/conds → decoder shape mismatch (e.g. 656 vs 662). A 1-line patch
+  (`h_masks = h_masks[:, :, :-pre*ratio]` after the `h` trim) + sizing the
+  meanflow noise to the trimmed length makes finalize=False RUN. (3) BUT the
+  flow encoder uses FULL-context attention (`static_chunk_size=0`), so
+  incremental/cumulative decode is **prefix-unstable** — adding tokens
+  re-attends and shifts earlier mel (maxdiff ~0.30-0.39 vs one-shot,
+  irrespective of fixed-noise slicing or emit-margin). (4) Forcing
+  `static_chunk_size>0` on the 2 modules that carry the attr did NOT stabilize
+  it (decoding_chunk_size is a forward-arg, not settable via attribute). Verdict:
+  true sub-second frame-streaming on turbo needs deep model-attention surgery
+  with quality risk — not worth it. Matches research ("turbo+streaming
+  unsolved"; vLLM-turbo outputs noise; davidbrowne17 streaming fork is
+  BASE-only). → Use adaptive-chunking instead.
+
+- `[2026-06-02]` **Naive cumulative re-decode for streaming** — decode the
+  growing token prefix each chunk + emit the delta, assuming the causal flow
+  gives a stable prefix. It does NOT (full-context attention, see above);
+  maxdiff 0.30. Also the high-level `s3gen.inference` re-applies `trim_fade` at
+  the start every call. Don't go this way.
+
 - `[2026-06-01]` **CSM bring-up** — upstream `phildougherty/sesame_csm_openai`
   Dockerfile pins NO `huggingface_hub` version, so it now resolves to 1.17.0
   where `huggingface-cli` is fully removed (replaced by `hf`) → the build dies