docs(chatterbox-fast): add executable plan-of-attack (durable, survives reboot vs /tmp)

Self-contained build plan for the chatterbox-fast streaming engine: the
adaptive buffer-ratchet chunking design, validated turbo API + facts, the
GPU-1 dev/test container pattern, 4 build phases, the base-fork A/B, and
watch-outs (incl. native-turbo-streaming is abandoned). Intended for a
fresh-context session to execute at full strength.

docs/design/chatterbox-fast-plan.md

@@ -0,0 +1,162 @@
+# Plan of Attack — `chatterbox-fast` streaming TTS engine
+
+_Authored 2026-06-02 for a fresh-context build session. Self-contained: you should
+not need the prior conversation. Cross-refs: `docs/design/chatterbox-fast.md`
+(design), `persistent-memory.md` (durable state + the abandoned native-streaming
+arc), repo `~/development/eshpfi-management` on host **nh3-dev**._
+
+---
+
+## 0. Mission
+
+Chatterbox(-Turbo) is becoming our **main TTS engine**. Build `chatterbox-fast`:
+a custom streaming server + container that delivers **sub-second time-to-first-
+audio** while keeping **turbo's full quality**. Workload = **single-stream
+interactive**. Operator authorized high effort incl. building the container from
+source. Deploy as a **parallel** stack beside the live `chatterbox` (:8196), burn
+in, then flip the catalog route.
+
+## 1. THE design — adaptive buffer-ratchet chunking (operator's idea; chosen)
+
+**Why not the alternatives** (settled this session, don't relitigate):
+- **Whole-paragraph one-shot** = best quality but ~2.5s+ TTFB (no streaming).
+- **Naive per-sentence split** = fast but **loses cross-sentence prosodic
+  context** → real quality loss (the T3 AR backbone conditions prosody on the
+  WHOLE text: contextual delivery, declination, affect continuity). "No
+  artifacts" ≠ "no quality loss." Operator corrected this; don't claim otherwise.
+- **Native frame-level streaming on turbo** = ABANDONED (turbo's flow uses
+  full-context attention, `static_chunk_size=0` → prefix-unstable; see
+  persistent-memory Tried/abandoned for the full dead-end map). Do NOT re-attempt
+  without explicit operator direction.
+
+**The adaptive-chunk algorithm:**
+1. Split text into **sentences** (and fall back to clause/comma split for a very
+   long FIRST sentence only, to protect first-audio latency).
+2. **Chunk 1 = first sentence** — generate alone, emit immediately (~0.66s
+   first-audio measured for a short sentence). Latency-critical.
+3. **While chunk N plays, generate chunk N+1** = greedily accumulate WHOLE
+   sentences until the *next* sentence would exceed the gen-time budget
+   `margin × audio_buffered_remaining`. Never split mid-sentence (keeps each
+   chunk prosodically self-coherent; joins land at natural sentence pauses).
+4. Chunks grow ~**3× each** (Chatterbox runs ~3.8× realtime; each chunk's
+   playback buys wall-clock for a ~3× bigger next chunk). So after 2-3 chunks,
+   the rest of the paragraph is ONE big chunk with near-full context. Context
+   loss confined to 2-3 joins at sentence boundaries.
+5. **Drive off MEASURED realtime factor**, not a constant — track actual
+   gen-speed live and self-correct. Start `margin=0.8`; be more conservative on
+   the **first** transition (smallest buffer = highest starvation risk) — ~0.6-0.7
+   there, then relax.
+6. **Optional context-priming at joins (quality-max):** prepend the previous
+   sentence as context to a chunk, generate, discard its audio → the chunk's
+   first sentence gets backward context. Cheap on early small chunks; skip once
+   chunks are large. Add this in Phase 2, measure if it's audibly worth it.
+
+**Critical enabling fact:** this only works because **RTF > 1**. Fish (<1×
+realtime) would starve no matter the chunking — that's why this is the
+chatterbox-specific answer.
+
+## 2. Validated API + facts (don't re-derive)
+
+- Model: `from chatterbox.tts_turbo import ChatterboxTurboTTS`
+  - `m = ChatterboxTurboTTS.from_pretrained(device="cuda")` (loads from HF cache)
+  - `m.prepare_conditionals(wav_path, exaggeration=0.5, norm_loudness=True)`
+  - `wav = m.generate(text, repetition_penalty=1.2, top_p=0.95, temperature=0.8,
+    top_k=1000)` → returns **watermarked** wav tensor shape `[1, T]`, `m.sr=24000`.
+    (CFG/exaggeration/min_p are ignored by turbo — warns but harmless.)
+  - Paralinguistic tags work inline (`[laugh] [whispers] [sigh]` etc.).
+- Architecture: T3 AR Llama 350M → S3Gen flow (2-step meanflow) → HiFTGenerator.
+- Realtime: ~3.8× on A6000 (17.7s audio / 4.7s), ~3.4× on 3090.
+- First-sentence latency: ~0.66s (short sentence, warm).
+- Watermark (Resemble PerTh) is applied inside `m.generate` — mandatory, fine
+  for internal use.
+
+## 3. Dev/test pattern (host irv-ml1 = 10.100.79.3, ssh `lkraven@10.100.79.3`)
+
+- lkraven is in the `docker` group on irv-ml1 → **NO sudo for docker**.
+- Model weights cached at `/worktank/chatterbox/cache` (HF_HOME); reference wavs
+  at `/worktank/chatterbox/reference_audio` (has `glados_25s.wav`, `Imogen.wav`).
+- One-off GPU container (use **GPU 1 / A6000** for dev — 3090 is VRAM-tight):
+  ```
+  IMG=$(docker images --format '{{.Repository}}:{{.Tag}}' | grep -i chatterbox | grep -v '<none>' | head -1)  # local/chatterbox:v1
+  docker run --rm --gpus '"device=1"' -e NVIDIA_VISIBLE_DEVICES=1 -e HF_HOME=/app/hf_cache \
+    -v /worktank/chatterbox/cache:/app/hf_cache \
+    -v /worktank/chatterbox/reference_audio:/refs \
+    -v /tmp/yourscript.py:/test.py "$IMG" python /test.py
+  ```
+- Lib introspection: `docker exec -i chatterbox python - <<'PY' ... PY` against
+  the running server container.
+- Write A/B samples to `/refs/_*.wav`, then `scp lkraven@10.100.79.3:/worktank/
+  chatterbox/reference_audio/_*.wav ~/chatterbox-ab/` for the operator to hear.
+
+## 4. Build phases
+
+**Phase 1 — streaming server MVP (the scheduler is the meat):**
+- `stacks/chatterbox-fast/app.py` — FastAPI server:
+  - Load model once at startup, warm it (one throwaway `generate`).
+  - `POST /tts` → `StreamingResponse` of audio chunks. Body: text, voice
+    (predefined name or clone ref), format (raw pcm s16le default for lowest
+    latency; offer wav/opus), the sampling knobs.
+  - The **adaptive-chunk scheduler** (§1): sentence-split → gen first sentence →
+    emit → loop {measure RTF, accumulate sentences to budget, generate, emit}.
+    Track `audio_emitted_seconds` and wall-clock to estimate buffer drain.
+  - `GET /health`.
+- Validate: first-audio latency, that the stream never starves (sim a player
+  consuming at 1× realtime), and produce a sample for the operator vs the
+  whole-paragraph one-shot.
+
+**Phase 2 — parity + perf:**
+- Predefined voices (dir of wavs) + clone refs (`prepare_conditionals`).
+- bf16 (`TTS_BF16`-style, or set model dtype), TF32
+  (`torch.backends.cuda.matmul.allow_tf32=True`), SDPA/flash backend.
+- Optional context-priming at joins (§1.6) — measure if audibly worth it.
+- torch.compile: DEFER (research flags batch-1 regression; bench separately).
+
+**Phase 3 — containerize + deploy:**
+- `stacks/chatterbox-fast/` : `compose.yaml`, `Dockerfile` (FROM the chatterbox
+  base image / vendored chatterbox + our `app.py`), `.env.example`, `README.md`.
+- Follow repo conventions (CLAUDE.md): `traefik-net`/`tnet`, named volumes,
+  `restart: unless-stopped`, healthcheck, homepage labels, GPU pin via
+  `device_ids`. **Port 8197** (next free on irv-ml1; reserved list in
+  `stacks/chatterbox/.env.example`). **GPU: 3090 (device 0) if turbo fits in
+  free VRAM, else A6000 (device 1)** — try 3090, fall back on OOM.
+- `playbooks/deploy-chatterbox-fast.yaml` (model is HF-cached already; reuse
+  `/worktank/chatterbox/cache`). Add an A/B smoke gate (first-audio < target).
+- Deploy **alongside** the live `chatterbox` — do NOT disrupt :8196.
+
+**Phase 4 — A/B + cutover:**
+- Add a **parallel** `chatterbox-fast` catalog entry in
+  `docs/asset-engine/services.yaml` (NOT replace `chatterbox` yet). If it needs a
+  new schema field, that's a `catalog_version` bump — coordinate with
+  **asset-engine-dev** via althing (and PUSH the commit promptly; their CI
+  drift-checks against the remote — lesson learned this session).
+- Burn-in + operator ear-A/B vs whole-paragraph. Then flip the route.
+
+## 5. ALSO build for A/B (operator asked): base-chatterbox + streaming fork
+
+- Install `davidbrowne17/chatterbox-streaming` (a fork with `generate_stream()`,
+  measured first-chunk ~0.47s on a 4090) — **BASE chatterbox model, not turbo**.
+  True frame-level streaming but base-model quality. Stand it up (own container /
+  port), generate a sample with the SAME text + a comparable voice, drop in
+  `~/chatterbox-ab/` for the 3-way A/B: adaptive-chunk-turbo vs base-fork-stream
+  vs whole-paragraph-turbo. Operator judges by ear.
+
+## 6. Acceptance / A/B
+
+- **Latency:** first-audio < ~0.8s on the deployment GPU.
+- **No starvation:** stream stays ahead of 1× playback (assert in a sim).
+- **Quality:** operator ear-A/B the adaptive-chunk output vs whole-paragraph
+  one-shot — the join-context loss should be ~imperceptible for multi-sentence
+  text. Samples → `~/chatterbox-ab/`.
+
+## 7. Existing A/B samples (this session, GLaDOS voice) on nh3-dev `~/chatterbox-ab/`
+- `01_sentence_level_turbo.wav` — naive per-sentence (the baseline to BEAT).
+- `02_chunked_native_streamed.wav` — abandoned native attempt (artifacty).
+- `03_chunked_oneshot.wav` — chunked-attention one-shot.
+(The adaptive-chunk output and the base-fork output are still to be generated.)
+
+## 8. Watch-outs
+- Don't claim sentence-splitting is lossless (it isn't — prosodic context).
+- Don't re-attempt native turbo frame-streaming without operator say-so.
+- Push catalog commits to origin promptly (asset-engine CI).
+- Use `ssh -t` only when a remote needs sudo; docker on irv-ml1 needs no sudo.
+- The 3090 shows ~20.5 GB used at idle (shared dev stack) — expect tight fit.