feat(chatterbox-fast): context-priming at joins (§1.6, opt-in)
Prime early joins by prepending the prior sentence as backward prosodic context,
generating context+content together, then discarding the context audio. The cut
snaps to the inter-sentence pause (energy-minimum search around the context's
solo duration) with a 5ms fade-in to kill any seam click (app: _cut_at_pause /
_fade_in / Engine.generate_primed). Opt-in via request `prime` (default off).
Scheduler: priming is AFFORDABILITY-GATED so it can never starve. A primed chunk
costs ~(2·context + content)/rtf (a 2nd context-solo pass); a chunk is only primed
when buffer ≥ prime_buffer_factor (1.5) × that cost, else it falls back to a cold
generate. Consequences proven in the GPU-free sim (17 tests):
- fires on early joins for any GPU at/above rtf_prior (3.4 = 3090; A6000 ~3.8-4.0)
- self-skips (degrades to cold) on a slower-than-fleet GPU rather than starving
- never primes chunk 0 (latency-critical)
Also fixed a latent Phase-1 bug: margin_first was applied at chunk 0 (budget always
0 there) so it never did anything — now applied at chunk 1 (the first transition).
Live A/B on irv-ml1 (A6000, GLaDOS): TTFB unaffected (445 vs 467ms), no starvation;
priming fired on chunk 2 (gen 1.6s for the doubled pass). On typical text exactly
ONE early join safely primes — priming chunk 2 flattens the buffer so later/larger
chunks no longer clear the safety gate. Samples: ~/chatterbox-ab/_p2_{cold,primed}.wav.
This commit is contained in:
@@ -45,13 +45,19 @@ class FakeClock:
|
||||
def make_generator(clock: FakeClock, *, true_rtf: float, sec_per_char: float):
|
||||
"""A fake generate() that costs realistic wall-clock and returns audio_sec.
|
||||
|
||||
Audio duration is proportional to text length; generation costs
|
||||
``audio_sec / true_rtf`` of (simulated) wall-clock, advancing the clock.
|
||||
Audio duration is proportional to content length; generation costs
|
||||
``audio_sec / true_rtf`` of (simulated) wall-clock, advancing the clock. A
|
||||
primed chunk (context given) costs extra: a context-solo pass plus the
|
||||
context portion of the joint pass — modelling the ~2× cost the scheduler
|
||||
must budget for.
|
||||
"""
|
||||
|
||||
def generate(text: str):
|
||||
audio_sec = len(text) * sec_per_char
|
||||
clock.t += audio_sec / true_rtf
|
||||
def generate(text: str, context: str | None):
|
||||
audio_sec = len(text) * sec_per_char # content only (context discarded)
|
||||
gen_audio = audio_sec
|
||||
if context:
|
||||
gen_audio += 2 * len(context) * sec_per_char # ctx-solo + ctx in joint
|
||||
clock.t += gen_audio / true_rtf
|
||||
return None, audio_sec
|
||||
|
||||
return generate
|
||||
@@ -185,6 +191,43 @@ def test_full_text_reconstructed():
|
||||
assert joined == " ".join(split_sentences(PARAGRAPH))
|
||||
|
||||
|
||||
def test_priming_fires_but_never_on_chunk_zero():
|
||||
"""With prime_first_n=2, priming is best-effort (affordability-gated): it fires
|
||||
on at least one early join at fleet RTF, and NEVER on chunk 0 (latency-critical)."""
|
||||
cfg = ChunkConfig(prime_first_n=2)
|
||||
results = run(true_rtf=3.8, cfg=cfg)
|
||||
assert results[0].primed is False
|
||||
assert any(r.primed for r in results[1:]), "expected at least one primed join"
|
||||
# Priming only ever lands on chunks 1..N.
|
||||
assert all(not r.primed for r in results if r.index > cfg.prime_first_n)
|
||||
|
||||
|
||||
def test_priming_never_starves_at_fleet_rtf():
|
||||
"""Priming must keep the no-starvation guarantee for any GPU at/above the
|
||||
rtf_prior floor (3090 ~3.4, A6000 ~3.8–4.0)."""
|
||||
cfg = ChunkConfig(prime_first_n=2)
|
||||
for rtf in (4.0, 3.8, 3.4):
|
||||
results = run(true_rtf=rtf, cfg=cfg)
|
||||
assert all(not r.starved for r in results), (
|
||||
rtf, [(r.index, r.drained) for r in results if r.starved]
|
||||
)
|
||||
|
||||
|
||||
def test_priming_self_skips_on_slow_gpu_no_starvation():
|
||||
"""On a slower-than-fleet GPU (RTF below the prior), priming gracefully
|
||||
self-skips rather than starving — the guarantee holds, priming just stops."""
|
||||
cfg = ChunkConfig(prime_first_n=2)
|
||||
results = run(true_rtf=3.0, cfg=cfg)
|
||||
assert all(not r.starved for r in results)
|
||||
assert not any(r.primed for r in results) # degraded to cold
|
||||
|
||||
|
||||
def test_priming_off_by_default():
|
||||
"""Default config primes nothing (so context is never requested)."""
|
||||
results = run(true_rtf=3.8)
|
||||
assert all(not r.primed for r in results)
|
||||
|
||||
|
||||
def _main():
|
||||
results = run(true_rtf=3.8)
|
||||
print(f"{'idx':>3} {'chars':>5} {'audio_s':>8} {'gen_s':>7} "
|
||||
|
||||
Reference in New Issue
Block a user