feat(chatterbox-fast): Phase 1 streaming server — adaptive-chunk scheduler
Build the streaming TTS server MVP per docs/design/chatterbox-fast-plan.md §4. - scheduler.py: adaptive buffer-ratchet chunker (the meat) — GPU-free pure logic. First sentence emitted alone for low TTFA, then chunks ratchet ~3x by packing whole sentences to margin x buffered-audio; drives off measured RTF + sec/char (EMA). relieve_leader() clause-splits a too-big mid-stream sentence to avoid starvation (joins land on commas); a long comma-less sentence is the one honored-but-flagged limitation. - test_scheduler.py: GPU-free simulation, 13 tests — asserts no-starvation (incl. overestimated RTF) and the ratchet. - app.py: FastAPI model holder + POST /tts StreamingResponse (raw PCM s16le default, wav optional, stream/oneshot) + GET /health. - bench.py: client — ground-truth TTFB + real 1x-consumer starvation check. Live test on irv-ml1 (turbo, A6000, GLaDOS voice): streaming TTFB 499ms vs oneshot 5230ms (~10x), stayed ahead of a 1x player (no starvation), ratchet 1.64->4.08->8.60->8.60s audio, measured RTF self-corrected 3.38->4.01. Kill the superseded docs/design/chatterbox-fast.md — its §5 windowed-token streaming was the abandoned native-frame-streaming arc; the adaptive-chunk plan supersedes it. Repoint persistent-memory + README at the canonical plan.
This commit is contained in:
@@ -92,8 +92,11 @@ _As of 2026-06-02:_
|
||||
- **PRIMARY FOCUS — building `chatterbox-fast`, a custom streaming TTS
|
||||
container; Chatterbox is becoming our MAIN TTS engine.** Operator-authorized
|
||||
high-effort build (incl. custom container from source). **Plan-of-attack:
|
||||
`/tmp/chatterbox-fast-plan.md`** (write/refresh before /clear — it carries
|
||||
the full executable detail). Design doc: `docs/design/chatterbox-fast.md`.
|
||||
`docs/design/chatterbox-fast-plan.md`** (durable; carries the full executable
|
||||
detail). The old `docs/design/chatterbox-fast.md` was KILLED 2026-06-01 — its
|
||||
§5 windowed-token-streaming was the abandoned native arc; superseded by the
|
||||
adaptive-chunk plan. Phase 1 built: `stacks/chatterbox-fast/` (scheduler +
|
||||
app + sim tests).
|
||||
- **Goal:** cut time-to-first-audio from ~2.5s → sub-second via streaming,
|
||||
keep turbo quality. Workload = single-stream interactive.
|
||||
- **Chosen approach = adaptive buffer-ratchet chunking** (operator's idea):
|
||||
@@ -211,8 +214,7 @@ _As of 2026-06-02:_
|
||||
3090-idle is expected residency, not a blocker. **Cutover: parallel catalog
|
||||
entry**, burn in beside live `chatterbox`, then flip. **Streaming approach:
|
||||
adaptive buffer-ratchet chunking** (see in-flight). Native frame-streaming
|
||||
abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast.md` +
|
||||
`/tmp/chatterbox-fast-plan.md`.
|
||||
abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast-plan.md`.
|
||||
|
||||
- `[2026-06-02]` **Sentence-splitting loses quality (operator-corrected).** I
|
||||
claimed naive sentence-level streaming has "zero quality loss" — WRONG. The
|
||||
|
||||
Reference in New Issue
Block a user