diff --git a/persistent-memory.md b/persistent-memory.md index 9ccfeff..4b0f42e 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -95,8 +95,18 @@ _As of 2026-06-02:_ `docs/design/chatterbox-fast-plan.md`** (durable; carries the full executable detail). The old `docs/design/chatterbox-fast.md` was KILLED 2026-06-01 — its §5 windowed-token-streaming was the abandoned native arc; superseded by the - adaptive-chunk plan. Phase 1 built: `stacks/chatterbox-fast/` (scheduler + - app + sim tests). + adaptive-chunk plan. **DEPLOYED 2026-06-02** — Phases 1-3 done: + `stacks/chatterbox-fast/` (scheduler + app + sim tests + Dockerfile/compose), + live on **irv-ml1 :8197** (container `chatterbox-fast`, healthy) BESIDE the live + `chatterbox` :8196. Measured: **TTFB ~0.5s** (vs ~5.2s one-shot), no starvation, + RTF ~3.9-4.0× on A6000. **GPU = A6000 (device 1) ONLY** — turbo loads **fp32** + (NOT the ~2.5GB fp16 old notes claimed), measured **5.34 GB**, so the 3090's + ~3.8GB free does NOT fit. Perf: TF32/SDPA on (don't move TTFA — AR-decode-bound; + bf16 deferred, no clean fp32→bf16 cast path). Context-priming for join coherence + TRIED + REVERTED (discard-cut leaks prefix — see Tried/abandoned); coherence loss + at joins is an ACCEPTED limitation (operator: cold streaming "really good"). + Remaining: Phase 4 catalog entry + cutover; optional deploy-playbook smoke gate; + optional §5 base-fork 3-way A/B. - **Goal:** cut time-to-first-audio from ~2.5s → sub-second via streaming, keep turbo quality. Workload = single-stream interactive. - **Chosen approach = adaptive buffer-ratchet chunking** (operator's idea):