feat(chatterbox-fast): Phase 1 streaming server — adaptive-chunk scheduler

Build the streaming TTS server MVP per docs/design/chatterbox-fast-plan.md §4.

- scheduler.py: adaptive buffer-ratchet chunker (the meat) — GPU-free pure
  logic. First sentence emitted alone for low TTFA, then chunks ratchet ~3x by
  packing whole sentences to margin x buffered-audio; drives off measured RTF +
  sec/char (EMA). relieve_leader() clause-splits a too-big mid-stream sentence
  to avoid starvation (joins land on commas); a long comma-less sentence is the
  one honored-but-flagged limitation.
- test_scheduler.py: GPU-free simulation, 13 tests — asserts no-starvation
  (incl. overestimated RTF) and the ratchet.
- app.py: FastAPI model holder + POST /tts StreamingResponse (raw PCM s16le
  default, wav optional, stream/oneshot) + GET /health.
- bench.py: client — ground-truth TTFB + real 1x-consumer starvation check.

Live test on irv-ml1 (turbo, A6000, GLaDOS voice): streaming TTFB 499ms vs
oneshot 5230ms (~10x), stayed ahead of a 1x player (no starvation), ratchet
1.64->4.08->8.60->8.60s audio, measured RTF self-corrected 3.38->4.01.

Kill the superseded docs/design/chatterbox-fast.md — its §5 windowed-token
streaming was the abandoned native-frame-streaming arc; the adaptive-chunk plan
supersedes it. Repoint persistent-memory + README at the canonical plan.
This commit is contained in:
vh
2026-06-01 22:42:28 -07:00
parent 4b9bd109bf
commit 7cd39001b2
7 changed files with 957 additions and 141 deletions
+6 -4
View File
@@ -92,8 +92,11 @@ _As of 2026-06-02:_
- **PRIMARY FOCUS — building `chatterbox-fast`, a custom streaming TTS
container; Chatterbox is becoming our MAIN TTS engine.** Operator-authorized
high-effort build (incl. custom container from source). **Plan-of-attack:
`/tmp/chatterbox-fast-plan.md`** (write/refresh before /clear — it carries
the full executable detail). Design doc: `docs/design/chatterbox-fast.md`.
`docs/design/chatterbox-fast-plan.md`** (durable; carries the full executable
detail). The old `docs/design/chatterbox-fast.md` was KILLED 2026-06-01 — its
§5 windowed-token-streaming was the abandoned native arc; superseded by the
adaptive-chunk plan. Phase 1 built: `stacks/chatterbox-fast/` (scheduler +
app + sim tests).
- **Goal:** cut time-to-first-audio from ~2.5s → sub-second via streaming,
keep turbo quality. Workload = single-stream interactive.
- **Chosen approach = adaptive buffer-ratchet chunking** (operator's idea):
@@ -211,8 +214,7 @@ _As of 2026-06-02:_
3090-idle is expected residency, not a blocker. **Cutover: parallel catalog
entry**, burn in beside live `chatterbox`, then flip. **Streaming approach:
adaptive buffer-ratchet chunking** (see in-flight). Native frame-streaming
abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast.md` +
`/tmp/chatterbox-fast-plan.md`.
abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast-plan.md`.
- `[2026-06-02]` **Sentence-splitting loses quality (operator-corrected).** I
claimed naive sentence-level streaming has "zero quality loss" — WRONG. The