From fa56718ba0ac8aa4f9a013363abad7544b64a777 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 2 Jun 2026 08:59:17 -0700 Subject: [PATCH] memory: shelve fish-streaming (sub-realtime 0.80x); standing fp4+Blackwell revisit trigger --- persistent-memory.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/persistent-memory.md b/persistent-memory.md index 4b0f42e..30dbd6f 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -334,6 +334,9 @@ _25 older entries archived to archival-memory.md._ ## Tried and abandoned +- `[2026-06-02]` **Fish (fish-s2 / OpenAudio S1-mini) progressive streaming — SHELVED (sub-realtime).** Benched RTF on A6000: 0.72x (12w) / 0.82x (30w) / 0.86x (60w), **mean 0.80x = sub-realtime**, so client-side chunking would starve (same reason chatterbox-fast needs turbo's RTF>1). Root cause of the buffering (dvalin-smithy-dev deep research, verified in our code text2semantic/inference.py L600-607): Fish only chunks on `<|speaker:X|>` tags; **plain text -> batches=[whole text]** -> all semantic tokens generate before any audio (chunk_length inert). Plus a 2nd layer: kui/ASGI StreamResponse doesn't flush (header produced t=1s, delivered t=23s) -> fix = anti-buffering headers (X-Accel-Buffering:no / Transfer-Encoding:chunked) in tools/server/views.py (kept on file, not applied). A rebuild does NOT fix this (current main same logic). **STANDING REVISIT TRIGGER: when an RTX Blackwell Pro lands in the fleet -> bench fp4-quantized Fish; if RTF > ~1.5x, give it the chatterbox-fast treatment** (client-side adaptive buffer-ratchet chunker driving /v1/tts with small text pieces). Projection: fp4 (~1/4 weight bytes, memory-bound AR decode) + Blackwell (GDDR7 ~1.8TB/s vs A6000 0.77TB/s, native FP4 cores) ~ 2-3x RTF; validate fp4 voice quality (ear/ECAPA) before committing. For now Fish stays a buffered catalog entry (great for SAVED gens, not the live-audition lane). + + - `[2026-06-02]` **Context-priming at chunk joins (chatterbox-fast §1.6) — ABANDONED (discard-cut leaks the prefix).** To give a chunk backward prosodic context, prepend the prior sentence, generate `prefix+content` together, then