feat(chatterbox-fast): context-priming at joins (§1.6, opt-in)

Prime early joins by prepending the prior sentence as backward prosodic context,
generating context+content together, then discarding the context audio. The cut
snaps to the inter-sentence pause (energy-minimum search around the context's
solo duration) with a 5ms fade-in to kill any seam click (app: _cut_at_pause /
_fade_in / Engine.generate_primed). Opt-in via request `prime` (default off).

Scheduler: priming is AFFORDABILITY-GATED so it can never starve. A primed chunk
costs ~(2·context + content)/rtf (a 2nd context-solo pass); a chunk is only primed
when buffer ≥ prime_buffer_factor (1.5) × that cost, else it falls back to a cold
generate. Consequences proven in the GPU-free sim (17 tests):
  - fires on early joins for any GPU at/above rtf_prior (3.4 = 3090; A6000 ~3.8-4.0)
  - self-skips (degrades to cold) on a slower-than-fleet GPU rather than starving
  - never primes chunk 0 (latency-critical)
Also fixed a latent Phase-1 bug: margin_first was applied at chunk 0 (budget always
0 there) so it never did anything — now applied at chunk 1 (the first transition).

Live A/B on irv-ml1 (A6000, GLaDOS): TTFB unaffected (445 vs 467ms), no starvation;
priming fired on chunk 2 (gen 1.6s for the doubled pass). On typical text exactly
ONE early join safely primes — priming chunk 2 flattens the buffer so later/larger
chunks no longer clear the safety gate. Samples: ~/chatterbox-ab/_p2_{cold,primed}.wav.
This commit is contained in:
vh
2026-06-01 23:11:21 -07:00
parent 3a92fcd943
commit d707439041
5 changed files with 193 additions and 22 deletions
+48 -5
View File
@@ -45,13 +45,19 @@ class FakeClock:
def make_generator(clock: FakeClock, *, true_rtf: float, sec_per_char: float):
"""A fake generate() that costs realistic wall-clock and returns audio_sec.
Audio duration is proportional to text length; generation costs
``audio_sec / true_rtf`` of (simulated) wall-clock, advancing the clock.
Audio duration is proportional to content length; generation costs
``audio_sec / true_rtf`` of (simulated) wall-clock, advancing the clock. A
primed chunk (context given) costs extra: a context-solo pass plus the
context portion of the joint pass — modelling the ~2× cost the scheduler
must budget for.
"""
def generate(text: str):
audio_sec = len(text) * sec_per_char
clock.t += audio_sec / true_rtf
def generate(text: str, context: str | None):
audio_sec = len(text) * sec_per_char # content only (context discarded)
gen_audio = audio_sec
if context:
gen_audio += 2 * len(context) * sec_per_char # ctx-solo + ctx in joint
clock.t += gen_audio / true_rtf
return None, audio_sec
return generate
@@ -185,6 +191,43 @@ def test_full_text_reconstructed():
assert joined == " ".join(split_sentences(PARAGRAPH))
def test_priming_fires_but_never_on_chunk_zero():
"""With prime_first_n=2, priming is best-effort (affordability-gated): it fires
on at least one early join at fleet RTF, and NEVER on chunk 0 (latency-critical)."""
cfg = ChunkConfig(prime_first_n=2)
results = run(true_rtf=3.8, cfg=cfg)
assert results[0].primed is False
assert any(r.primed for r in results[1:]), "expected at least one primed join"
# Priming only ever lands on chunks 1..N.
assert all(not r.primed for r in results if r.index > cfg.prime_first_n)
def test_priming_never_starves_at_fleet_rtf():
"""Priming must keep the no-starvation guarantee for any GPU at/above the
rtf_prior floor (3090 ~3.4, A6000 ~3.8–4.0)."""
cfg = ChunkConfig(prime_first_n=2)
for rtf in (4.0, 3.8, 3.4):
results = run(true_rtf=rtf, cfg=cfg)
assert all(not r.starved for r in results), (
rtf, [(r.index, r.drained) for r in results if r.starved]
)
def test_priming_self_skips_on_slow_gpu_no_starvation():
"""On a slower-than-fleet GPU (RTF below the prior), priming gracefully
self-skips rather than starving — the guarantee holds, priming just stops."""
cfg = ChunkConfig(prime_first_n=2)
results = run(true_rtf=3.0, cfg=cfg)
assert all(not r.starved for r in results)
assert not any(r.primed for r in results) # degraded to cold
def test_priming_off_by_default():
"""Default config primes nothing (so context is never requested)."""
results = run(true_rtf=3.8)
assert all(not r.primed for r in results)
def _main():
results = run(true_rtf=3.8)
print(f"{'idx':>3} {'chars':>5} {'audio_s':>8} {'gen_s':>7} "