fix(tts): revert sampling knobs — real cause was Turbo AR-tail over-run, fixed server-side
The long-turn "swaps to German" garble was NOT a language leak (infra-ops's
initial framing) and NOT the sampling entropy my interim curb targeted. The real
cause, signal-measured by infra-ops: the Chatterbox Turbo model over-runs its
generation TAIL — a long single generation degrades into garble/dead-air in its
final ~2-3s (voiced-tail zero-crossing rate 1.58x the middle). The gateway's
unbounded chunk-size ratchet built 300-600 char mega-chunks that landed in that
zone, and streaming concatenated each bad tail.
My interim curb (top_k 1000->80, top_p 0.95->0.85, temp 0.8->0.5) made it WORSE:
tight sampling pulls the degradation onset to a shorter length (~200 chars vs
~300 at defaults), so it fights the server-side fix rather than helping.
Fixed server-side (infra-ops, chatterbox-fast image :v2): a max_chunk_chars=250
cap bounds each generation below the ~300-char onset -> clean prosodic chunks
(verified ZCR 1.58x -> 0.64x; operator ear-confirmed clean audio + clean joins).
Consumer side, this commit:
- Revert the sampling knobs: gateway_body back to {text, voice, format, stream},
send full text with the gateway's default sampling. The server chunks at 250.
- Keep the /api/tts empty-200 -> 503 guard as hygiene (DEC-9a; the shared-3090
OOM that produced empty 200s is also resolved — Zonos moved off the card).
Contract DEC-9 rewritten with the resolved root cause + the two wrong hypotheses;
DEC-9a marked kept-as-hygiene. 520 green.
This commit is contained in:
@@ -154,36 +154,37 @@ each independently shippable. Slice order is chosen for fastest visible result.
|
||||
`ratatoskr:donut → "donut"` directly. NOTE the case: chatterbox wants lowercase
|
||||
`"donut"` (Zonos used `"Donut"`). Non-interview agents fall to the chatterbox
|
||||
default `"glados_25s"` (was Zonos `"Cora"`, which does not exist on chatterbox).
|
||||
- **DEC-9 — hold English (amended 2026-08-07; the "nothing to drift" call was WRONG).**
|
||||
The Zonos `language:"en-us"` pin is dropped — chatterbox has no `language` field. BUT the
|
||||
initial "English-only, nothing to drift" rationale was FALSIFIED by an operator report the
|
||||
same day: the Turbo checkpoint drifts into German partway through a long turn. infra-ops's
|
||||
authoritative root-cause (source-read, thread 01KZEDMJ…): **Turbo has latent multilingual
|
||||
capacity that leaks**, and the drift is **length-driven** — the gateway's adaptive scheduler
|
||||
ratchets chunk size upward with NO cap, so a long turn collapses into essentially ONE long
|
||||
generation after the first 2-3 sentences, and the sampler wanders off English on that single
|
||||
long decode (generation state DOES reset per chunk — cross-chunk carry is not the mechanism;
|
||||
it's the unbounded per-chunk length). Two-layer response:
|
||||
- **Sampling curbs (shipped, no redeploy) — REDUCE drift probability, do NOT guarantee it.**
|
||||
`gateway_body` tightens below the gateway defaults: `_TTS_TOP_K = 80` (from 1000 — the
|
||||
highest-leverage knob; the huge default admits off-language tokens), `_TTS_TOP_P = 0.85`
|
||||
(from 0.95), `_TTS_TEMPERATURE = 0.5` (from 0.8). Escalation if still drifting: temp
|
||||
0.3-0.4, rep_penalty 1.2→1.3.
|
||||
- **Length-bounding (the ROBUST fix) — GUARANTEES English by keeping each generation short.**
|
||||
Two paths, operator's call: (a) return to short CLIENT-side chunking (~1-2 sentences per
|
||||
/tts call, each a fresh re-anchored generation; partially reverses DEC-10; works today, no
|
||||
redeploy; loses the gateway's seamless internal streaming); (b) infra-ops adds a SERVER-side
|
||||
max-chunk cap to the scheduler (keeps seamless streaming AND holds English; needs a
|
||||
chatterbox-fast redeploy + operator greenlight on the card-shared fleet service — then the
|
||||
consumer reverts to sending full text).
|
||||
The persona's dialogue-only rewrite in `docs/characters/donut.md` still stands (removes the
|
||||
asterisk-RP-voiced-verbatim vector regardless of engine). (Original Zonos DEC-9 below.)
|
||||
- **DEC-9 — hold English: RESOLVED SERVER-SIDE 2026-08-07 (client sends full text, default
|
||||
sampling).** The Zonos `language:"en-us"` pin is dropped — chatterbox has no `language` field.
|
||||
The long-turn garble ("swaps to German halfway through") went through two WRONG hypotheses
|
||||
before the real cause surfaced under the operator's clean-diagnosis push (thread 01KZEDMJ…):
|
||||
- WRONG-1: "English-only, nothing to drift" (infra-ops Q8) — falsified by the operator report.
|
||||
- WRONG-2: "multilingual leak, curb with tighter sampling" — I shipped `top_k 1000→80` /
|
||||
`top_p 0.95→0.85` / `temp 0.8→0.5`. This made it WORSE: tight sampling pulls the degradation
|
||||
onset to a SHORTER length (garble by ~200 chars vs ~300 at default knobs).
|
||||
- **REAL CAUSE (infra-ops, signal-measured): the Turbo model OVER-RUNS its generation TAIL** —
|
||||
a long single generation degrades into garble/dead-air in its final ~2-3s (voiced-tail
|
||||
zero-crossing rate 1.58x the middle; the lib itself filters OOV tokens + pads silence). The
|
||||
scheduler's unbounded buffer-ratchet built 300-600 char mega-chunks landing squarely in that
|
||||
zone, and streaming concatenated each bad tail. The "German" was that tail garble mis-heard
|
||||
(compounded by shared-3090 OOM garbage, now gone — Zonos was moved off the 3090).
|
||||
- **FIX (deployed server-side, image :v2): `max_chunk_chars=250`** on the gateway scheduler,
|
||||
bounding each generation below the ~300-char onset → clean 3-4 sentence chunks with a full
|
||||
prosodic arc (verified: voiced-tail ZCR 1.58x→0.64x; operator ear-confirmed clean audio AND
|
||||
clean paragraph joins — chatterbox's low emotiveness keeps the seams smooth). Per-request
|
||||
`max_chunk_chars` override available for per-call tuning.
|
||||
- **CONSUMER SIDE (this is what the code does): send FULL text with the gateway's DEFAULT
|
||||
sampling.** The WRONG-2 sampling curbs were REVERTED (they degrade audio earlier and fight
|
||||
the :v2 cap). `gateway_body` is back to `{text, voice, format, stream}` — no client chunking
|
||||
(DEC-10 stays retired; the server chunks at 250), no sampling overrides. The persona's
|
||||
dialogue-only rewrite in `docs/characters/donut.md` still stands. (Original Zonos DEC-9 below.)
|
||||
|
||||
- **DEC-9a — OOM on long single generations → empty 200 (infra-ops 2026-08-07).** chatterbox-fast
|
||||
shares the RTX 3090 with Zonos2 (~1 GB headroom). A long single generation can OOM the card;
|
||||
the gateway then returns HTTP **200 with a 0-byte body** (not a 5xx). `tts_endpoint` treats an
|
||||
empty 200 body as a synthesis failure → 503 (INV-TTS-4 visible skip), never a silent empty
|
||||
audio/wav stream. Length-bounding (DEC-9 above) fixes the OOM too — small chunks don't OOM.
|
||||
- **DEC-9a — empty-200 OOM guard: KEPT as hygiene (OOM itself resolved).** chatterbox-fast used
|
||||
to share the RTX 3090 with Zonos2 (~1 GB headroom); a long single generation could OOM the card
|
||||
and the gateway then returned HTTP **200 with a 0-byte body**. The OOM is resolved (Zonos moved
|
||||
off the 3090; the :v2 250-char cap keeps generations small), but `tts_endpoint` KEEPS treating an
|
||||
empty 200 body as a synthesis failure → 503 (INV-TTS-4 visible skip) — good hygiene regardless
|
||||
(infra-ops-endorsed), never a silent empty audio/wav stream.
|
||||
- **DEC-10 — long-form chunk-and-concatenate. RETIRED 2026-08-07 (chatterbox
|
||||
migration).** chatterbox-fast has NO per-synth token/duration cap (Zonos capped at
|
||||
6144 tok / 71.2s) and chunks arbitrary-length text INTERNALLY via its adaptive
|
||||
@@ -247,8 +248,9 @@ tts_stream(text, *, voice, client: httpx.AsyncClient, url=CHATTERBOX_TTS_URL) ->
|
||||
# YIELD wav chunks as they synthesize. Pass through verbatim — never buffer, never rewrite the placeholder
|
||||
# header. chatterbox chunks arbitrary-length text INTERNALLY (no per-synth cap, DEC-10 RETIRED), so this
|
||||
# SINGLE call voices a whole turn — no client-side chunk-and-concatenate wrapper.
|
||||
# gateway_body(text, voice) = {text, voice, format:"wav", stream:true, temperature:_TTS_TEMPERATURE}.
|
||||
# temperature < gateway-default 0.8 holds English across a long turn (DEC-9). NO dials, NO language (RETIRED).
|
||||
# gateway_body(text, voice) = {text, voice, format:"wav", stream:true}. Full text, DEFAULT sampling — the
|
||||
# :v2 server caps each generation at 250 chars, which fixes the long-turn tail garble (DEC-9). NO dials,
|
||||
# NO language, NO client sampling curbs (a curb was counterproductive — it pulled the garble onset earlier).
|
||||
precondition: text non-empty. Voice membership in GET /voices is GATEWAY-enforced, not client-asserted.
|
||||
postcondition: yields the gateway's chunked int16 streaming WAV bytes unmodified (0xFFFFFFFF placeholder
|
||||
sizes intact), one leading header then s16le PCM @ 24000 Hz to EOF.
|
||||
|
||||
Reference in New Issue
Block a user