vh/ratatoskr
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(tts): migrate off Zonos to chatterbox-fast; drop affect, hold English

Browse Source 
Repoint the TTS client from the Zonos gateway (:8890 /v1/audio/speech) to
chatterbox-fast (:8197 /tts — bespoke non-OpenAI {text,voice,format,stream}
schema, no auth, 24kHz, infra-ops-verified). tts.py stays the single swap seam.

Dropped, no backward-compat (pre-v1):
- Affect (DEC-7): the Turbo checkpoint has no emotion knob, so PadState,
  EmotionDials, pad_to_dials, the /api/tts p/a fields, and the browser pad
  argument are deleted. Voice is now flat.
- Client-side chunking (DEC-10): chatterbox has no per-synth cap and chunks
  internally, so chunk_text/tts_stream_long/_pcm_after_header are deleted; a
  single tts_stream call voices a whole turn, the mid-stream yielded_any degrade
  folded into it.
- Language pin (DEC-9): no language field; re-purposed to sampling curbs (below).

Fixed / added:
- Browser Web Audio sample rate 44100 -> 24000 (the chatterbox rate).
- Default voice Cora -> glados_25s; donut registered lowercase at /refs/donut.wav.
- English-drift curb: Turbo is multilingual-leaky and wanders off English on a
  long generation (the gateway scheduler ratchets chunk size unbounded). Tighten
  sampling in gateway_body: top_k 1000->80, top_p 0.95->0.85, temperature
  0.8->0.5. These reduce drift probability; the guaranteed fix is a server-side
  max-chunk cap (infra-ops, greenlit).
- OOM guard (DEC-9a): a long generation can OOM the shared 3090, returning 200
  with a 0-byte body; /api/tts surfaces an empty 200 as 503 rather than
  committing silent audio.

Contract donut_voiced_interview.contract.md amended: migration banner, DEC-1/3/8
amended, DEC-7/9/10 retired with historical notes, DEC-9a added.

Tests rewritten to the new wire; 520 green. Live-smoked against the gateway
(24kHz synth + endpoint proxy + web console). persistent-memory.md committed
alongside (commit-along).
This commit is contained in:
vh
2026-08-07 10:23:13 -07:00
parent 2cc670e4a1
commit 19b499ab50
8 changed files with 393 additions and 760 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/contracts/donut_voiced_interview.contract.md
+145 -66
View File
@@ -3,9 +3,10 @@ contract_version: "2.1"
module: "ratatoskr.web.tts_kb"
purpose: >
A voiced, corpus-grounded Tier-3 interview character in the ratatoskr web
console. Two capabilities plus one character: (a) auto-TTS via the Zonos
gateway, spoken on SSE `done`, with emotion driven by the character's LIVE
PAD affect; (b) a consumer-side KB-retrieval + `memory_context` pinning
console. Two capabilities plus one character: (a) auto-TTS via the
chatterbox-fast gateway, spoken on SSE `done` (migrated off Zonos 2026-08-07;
no affect modulation — chatterbox Turbo has no emotion knob); (b) a
consumer-side KB-retrieval + `memory_context` pinning
BRIDGE that grounds the character's recall in the ingested corpus while she
stays in-voice; (c) Princess Donut (Dungeon Crawler Carl) as the first
instance. The bridge is a deliberate, retire-ready workaround for a Worldtree
@@ -21,12 +22,12 @@ touches:
- src/ratatoskr/web/server.py # /api/tts route + the retrieval-pinning seam on the turn path
- src/ratatoskr/web/static/index.html # speak-on-done playback, 🔊 toggle, <audio> sink; turn POST carries agent_id
- src/ratatoskr/web/entrypoint.py # RATATOSKR_TTS_URL override (the tts swap seam)
- src/ratatoskr/tts.py # NEW — Zonos gateway client + PAD->emotion-dial mapping
- src/ratatoskr/tts.py # chatterbox-fast gateway client (was Zonos + PAD->dial; migrated 2026-08-07)
- src/ratatoskr/kb_bridge.py # NEW, RETIRE-READY — consumer-side retrieval + memory_context pinning
- src/ratatoskr/wt.py # stream_turn gains a memory_context passthrough (seam-review: the contract's original touch list undercounted this by one file; the param defaults None so the bridge's RETIREMENT stays inert — deleting kb_bridge.py + the one call-site leaves wt.stream_turn's SDK-parity param harmless)
- docs/characters/donut.md # NEW — Princess Donut persona (content; the tier3 define source)
depends_on:
- "Zonos gateway: POST http://10.100.79.3:8890/v1/audio/speech (infra-ops; WG-internal, no auth; wav; verified 2026-08-02)"
- "chatterbox-fast gateway: POST http://10.100.79.3:8197/tts (infra-ops; WG-internal, no auth; bespoke non-OpenAI schema {text,voice,format,stream}; streaming placeholder-header wav @ 24000 Hz; no per-synth cap; NO affect controls; verified 2026-08-07 against image local/chatterbox-fast:v1)"
- "Worldtree turn stream: memory_context[] passthrough (SDK stream_turn already forwards it verbatim)"
- "Worldtree agents.define (Tier-3) for Donut; Mimir (search_kb) for the out-of-band retrieval consult"
used_by:
@@ -40,6 +41,29 @@ confidence: 0.8
# Contract: Donut voiced interview (auto-TTS + KB-recall bridge)
> **⚠ TTS MIGRATED OFF ZONOS → chatterbox-fast 2026-08-07 (operator-directed).**
> Slice 2's synthesis backend moved from the Zonos gateway (:8890
> `/v1/audio/speech`) to chatterbox-fast (:8197 `/tts`). Three architecture deltas,
> all infra-ops-verified against image `local/chatterbox-fast:v1`:
> - **Affect dropped (DEC-7 RETIRED).** chatterbox serves the Chatterbox TURBO
> checkpoint, which has NO valence/arousal/emotion knob (exaggeration is exposed
> but inert on Turbo). The whole PAD→emotion-dial path — `PadState`,
> `EmotionDials`, `pad_to_dials`, and the browser `p`/`a` body fields — is
> deleted. Voice is now flat (high-quality but unmodulated). Operator's call: if
> live affect ever becomes load-bearing again, Zonos (:8890) remains the only
> fleet TTS with real emotion steering.
> - **Client-side chunking dropped (DEC-10 RETIRED).** chatterbox has no per-synth
> token/duration cap (Zonos capped at 6144 tok / 71.2s) and chunks arbitrary-length
> text internally, so `chunk_text` + `tts_stream_long` + `_pcm_after_header` are
> deleted; a single `tts_stream` call voices a whole turn. The mid-stream degrade
> policy (`yielded_any`) folds INTO `tts_stream`.
> - **Sample rate 44100 → 24000 Hz.** The browser Web Audio decode MUST use 24000
> or the voice plays ~1.8× too fast.
> `/api/tts` stays POST; the streaming placeholder-header WAV shape (DEC-2/DEC-3) and
> the browser Web-Audio PCM decode path (DEC-2) are UNCHANGED except the sample rate.
> The `tts.py` client remains the single swap seam (DEC-1). DEC-7/9/10 below are
> retained as historical record of the Zonos build.
> **⚠ SLICE 3 (KB-recall bridge) RETIRED 2026-08-02.** The `kb_bridge.py` module +
> its single `web/server.py` call-site were deleted per INV-KB-1 when Worldtree #383
> shipped native Tier-3 `reference_knowledge` (v1.0.0b167, live on :8081 + demo).
@@ -58,10 +82,14 @@ each independently shippable. Slice order is chosen for fastest visible result.
## Decisions (DEC)
- **DEC-1 — direct :8890 coupling.** Proxy straight to the Zonos gateway, not the
swappable `ext-tts` LiteLLM alias. Rationale: the emotion dials (the whole
point — affect-driven voice) don't pass through `ext-tts`. Accept the
Zonos coupling; the `tts.py` client is the single swap seam if we ever move.
- **DEC-1 — direct :8197 coupling (amended 2026-08-07).** Proxy straight to the
chatterbox-fast gateway. Original Zonos rationale was the emotion dials (which
`ext-tts` dropped); that rationale is retired with affect (DEC-7). The coupling
STANDS regardless: chatterbox-fast is a bespoke, non-OpenAI `/tts` schema
(`{text,voice,format,stream}`, verified with infra-ops), NOT reachable through the
OpenAI-shaped `ext-tts` LiteLLM alias — so `tts.py` remains the single swap seam,
now translating that bespoke schema. (Considered routing via the generic alias
once affect was dropped; the non-OpenAI wire ruled it out.)
- **DEC-2 — STREAMING, play-as-it-arrives (amended 2026-08-02, operator-directed).**
The gateway ALREADY streams: `POST /v1/audio/speech` relays a chunked int16 WAV
(transfer-encoding: chunked, placeholder 0xFFFFFFFF RIFF/data sizes) as it synthesizes
@@ -75,8 +103,14 @@ each independently shippable. Slice order is chosen for fastest visible result.
(Amended 2026-08-02: the browser path is now Web Audio decoding raw int16 PCM, not
`<audio src>` — Safari/WebKit rejects a 0xFFFFFFFF-length streaming WAV via `<audio
src>` (NotSupportedError); and `/api/tts` is now POST, not GET. See DEC-10.)
- **DEC-3 — wav only.** `response_format:"wav"` (streaming int16 RIFF/WAVE). `mp3`/`opus`
are accepted but silently return mislabeled PCM — never request them.
- **DEC-3 — wav streaming (amended 2026-08-07).** `format:"wav"`, `stream:true` →
a streaming int16 RIFF/WAVE with placeholder (0xFFFFFFFF) RIFF/data sizes, one
header, then s16le PCM to EOF — the shape the browser's one-header-strip decoder
expects. chatterbox also offers `format:"pcm"` (headerless raw s16le, leaner);
kept on `wav` so the streaming shape stays byte-identical to the Zonos path and the
endpoint's RIFF-sniff (non-WAV-200 → 503) and the browser header-strip both stand
unchanged. (pcm is a deferred lean-up — it would drop the header-strip + the
RIFF-sniff, net a few lines.)
- **DEC-4 — server-side proxy.** Browser → `/api/tts` (nh3-dev) → gateway. The
irv-ml1 host/URL never reaches the client (INV-TTS-1). No key exists, so
INV-003 is trivially satisfied, but the proxy still stands (browser can't
@@ -103,26 +137,64 @@ each independently shippable. Slice order is chosen for fastest visible result.
a wing-scope mismatch — tool scoped to the failed `fiction` wing — or a confidence
threshold rejecting the weak ~0.03 hits). Escalated to worldtree-dev. Until fixed,
Donut recalls from her own character knowledge (degrades in-voice per the persona).
- **DEC-7 — affect-driven emotion.** Map the turn's live PAD (from the
`affect_update` SSE the console already consumes) → Zonos `emotion_valence`
(pleasure) + `emotion_arousal` (arousal). This reframes the feature as voice
OBSERVABILITY (hear the affect the persona pane shows), not chat-app TTS.
- **DEC-8 — voice: custom "donut" is REGISTERED (superseded the preset-first plan).**
The original plan was a theatrical Zonos preset first (Miranda/Penny/Emmie), custom
"donut" later. But infra-ops registered + verified the custom `voice:"donut"` before
slice-2 build (GET /v1/voices returns Donut; case-folded), so `_TTS_VOICE_MAP` maps
`ratatoskr:donut → "donut"` directly — no interim preset. Non-interview agents still
fall to the gateway default (Cora). (Amended 2026-08-02 per heid-code-review: 3 arms
flagged the code as DEC-8 drift; a live gateway read INVERTED the remedy — the code is
correct, DEC-8's "preset now" was stale.)
- **DEC-9 — pin English (added 2026-08-02, operator-directed).** Zonos is multilingual;
with no `language` field it drifts into other-language phonemes / gibberish on names,
numerics, and long spans (operator report). `gateway_body` pins `language:"en-us"` on
every call (gateway accepts it, verified live). The persona's dialogue-only rewrite
(`docs/characters/donut.md`) removes the other gibberish vector — asterisk RP action
beats were being voiced verbatim.
- **DEC-10 — long-form chunk-and-concatenate (added 2026-08-02, operator-directed;
infra-ops recipe 01KZ1FKX…).** The Zonos model hard-caps ONE synthesis at
- **DEC-7 — affect-driven emotion. RETIRED 2026-08-07 (chatterbox migration).**
chatterbox Turbo has no valence/arousal/emotion control (infra-ops-verified:
exaggeration is exposed but inert; cfg_weight/min_p not exposed; only generic
sampling knobs move output, and those change timbre/variance not emotion). There
is no coupling point for a live-PAD driver, so the entire path is deleted —
`PadState`, `EmotionDials`, `pad_to_dials`, the `/api/tts` `p`/`a` body fields, and
the browser's `pad` argument. Voice is now flat. Retained below as historical
record of the Zonos build. (Original: map live PAD from the `affect_update` SSE →
Zonos `emotion_valence`/`emotion_arousal`, reframing the feature as voice
OBSERVABILITY. The observability framing dies with the knob.)
- **DEC-8 — voice: custom "donut" is REGISTERED (amended 2026-08-07 for chatterbox).**
chatterbox voices are `*.wav` reference clips in `/refs` (GET /voices lists the
stems). infra-ops registered `/refs/donut.wav` (the same reference clip behind the
Zonos Donut voice) at operator direction, so `_TTS_VOICE_MAP` maps
`ratatoskr:donut → "donut"` directly. NOTE the case: chatterbox wants lowercase
`"donut"` (Zonos used `"Donut"`). Non-interview agents fall to the chatterbox
default `"glados_25s"` (was Zonos `"Cora"`, which does not exist on chatterbox).
- **DEC-9 — hold English (amended 2026-08-07; the "nothing to drift" call was WRONG).**
The Zonos `language:"en-us"` pin is dropped — chatterbox has no `language` field. BUT the
initial "English-only, nothing to drift" rationale was FALSIFIED by an operator report the
same day: the Turbo checkpoint drifts into German partway through a long turn. infra-ops's
authoritative root-cause (source-read, thread 01KZEDMJ…): **Turbo has latent multilingual
capacity that leaks**, and the drift is **length-driven** — the gateway's adaptive scheduler
ratchets chunk size upward with NO cap, so a long turn collapses into essentially ONE long
generation after the first 2-3 sentences, and the sampler wanders off English on that single
long decode (generation state DOES reset per chunk — cross-chunk carry is not the mechanism;
it's the unbounded per-chunk length). Two-layer response:
- **Sampling curbs (shipped, no redeploy) — REDUCE drift probability, do NOT guarantee it.**
`gateway_body` tightens below the gateway defaults: `_TTS_TOP_K = 80` (from 1000 — the
highest-leverage knob; the huge default admits off-language tokens), `_TTS_TOP_P = 0.85`
(from 0.95), `_TTS_TEMPERATURE = 0.5` (from 0.8). Escalation if still drifting: temp
0.3-0.4, rep_penalty 1.2→1.3.
- **Length-bounding (the ROBUST fix) — GUARANTEES English by keeping each generation short.**
Two paths, operator's call: (a) return to short CLIENT-side chunking (~1-2 sentences per
/tts call, each a fresh re-anchored generation; partially reverses DEC-10; works today, no
redeploy; loses the gateway's seamless internal streaming); (b) infra-ops adds a SERVER-side
max-chunk cap to the scheduler (keeps seamless streaming AND holds English; needs a
chatterbox-fast redeploy + operator greenlight on the card-shared fleet service — then the
consumer reverts to sending full text).
The persona's dialogue-only rewrite in `docs/characters/donut.md` still stands (removes the
asterisk-RP-voiced-verbatim vector regardless of engine). (Original Zonos DEC-9 below.)
- **DEC-9a — OOM on long single generations → empty 200 (infra-ops 2026-08-07).** chatterbox-fast
shares the RTX 3090 with Zonos2 (~1 GB headroom). A long single generation can OOM the card;
the gateway then returns HTTP **200 with a 0-byte body** (not a 5xx). `tts_endpoint` treats an
empty 200 body as a synthesis failure → 503 (INV-TTS-4 visible skip), never a silent empty
audio/wav stream. Length-bounding (DEC-9 above) fixes the OOM too — small chunks don't OOM.
- **DEC-10 — long-form chunk-and-concatenate. RETIRED 2026-08-07 (chatterbox
migration).** chatterbox-fast has NO per-synth token/duration cap (Zonos capped at
6144 tok / 71.2s) and chunks arbitrary-length text INTERNALLY via its adaptive
scheduler, streaming seamlessly — so client-side chunk-and-concatenate is deleted:
`chunk_text`, `tts_stream_long`, `_pcm_after_header`, and the `_TTS_CHUNK_CHAR_BUDGET`
constants are gone; a single `tts_stream` call voices a whole turn. `/api/tts` STAYS
POST (DEC-10a) — a long turn still rides the body, not a length-capped URL — and the
8000-char outer cap stays as a shared-GPU hold ceiling (no longer a chunk-count
bound). The `yielded_any` mid-stream degrade that lived in `tts_stream_long` folds
into `tts_stream`. Retained below as historical record of the Zonos build. (Original:)
The Zonos model hard-caps ONE synthesis at
`max_tokens=6144` = 71.2s of audio (6144 / 86.3 Hz codec frame rate; the gateway 400s
above 6144 — an architectural sequence limit, unraisable). So a turn longer than ~71s
truncated mid-stream. Fix: split the text client-side and concatenate the PCM
@@ -140,8 +212,8 @@ each independently shippable. Slice order is chosen for fastest visible result.
## Invariants
- **INV-TTS-1 [hard]** — the Zonos gateway host/URL never reaches the browser;
all synthesis goes through `/api/tts`.
- **INV-TTS-1 [hard]** — the TTS gateway host/URL (chatterbox-fast :8197) never
reaches the browser; all synthesis goes through `/api/tts`.
- **INV-TTS-2 [hard]** — TTS is opt-in: a 🔊 toggle (default OFF), persisted to
localStorage (mirrors the theme/cot-toggle pattern). No speech without it.
- **INV-TTS-3 [hard]** — one synth in flight at a time; a new turn cancels the
@@ -168,22 +240,27 @@ each independently shippable. Slice order is chosen for fastest visible result.
## FN blocks
### FN tts_stream (the per-CHUNK primitive — DEC-2 streaming; wrapped by tts_stream_long)
### FN tts_stream (the sole synthesis primitive — DEC-2 streaming; amended 2026-08-07)
```
tts_stream(text, *, voice, dials, client: httpx.AsyncClient, url=ZONOS_TTS_URL) -> AsyncIterator[bytes]
# Open the gateway's CHUNKED stream (client.stream("POST", url, json=gateway_body(...))) and YIELD wav
# chunks as they synthesize. Pass through verbatim — never buffer, never rewrite the placeholder header.
# gateway_body(text, voice, dials) = {input, voice, response_format:"wav", language:"en-us" (DEC-9),
# **dials.to_body()}.
precondition: text non-empty. Voice membership in /v1/voices is GATEWAY-enforced, not client-asserted.
tts_stream(text, *, voice, client: httpx.AsyncClient, url=CHATTERBOX_TTS_URL) -> AsyncIterator[bytes]
# Open the gateway's CHUNKED stream (client.stream("POST", url, json=gateway_body(text, voice))) and
# YIELD wav chunks as they synthesize. Pass through verbatim — never buffer, never rewrite the placeholder
# header. chatterbox chunks arbitrary-length text INTERNALLY (no per-synth cap, DEC-10 RETIRED), so this
# SINGLE call voices a whole turn — no client-side chunk-and-concatenate wrapper.
# gateway_body(text, voice) = {text, voice, format:"wav", stream:true, temperature:_TTS_TEMPERATURE}.
# temperature < gateway-default 0.8 holds English across a long turn (DEC-9). NO dials, NO language (RETIRED).
precondition: text non-empty. Voice membership in GET /voices is GATEWAY-enforced, not client-asserted.
postcondition: yields the gateway's chunked int16 streaming WAV bytes unmodified (0xFFFFFFFF placeholder
sizes intact). This is ONE synthesis (<= 71.2s cap, DEC-10); tts_stream_long stitches many.
error: a non-200 OPEN or connect/transport failure -> TtsUnavailable BEFORE the first chunk (so the
endpoint can still return 503); a mid-stream drop just ends the generator.
invariant: response_format is ALWAYS "wav" (DEC-3); never mp3/opus. language ALWAYS "en-us" (DEC-9).
sizes intact), one leading header then s16le PCM @ 24000 Hz to EOF.
error (the yielded_any pivot, folded in from the retired tts_stream_long):
- a non-200 OPEN or a connect/transport failure BEFORE the first byte -> TtsUnavailable (so the endpoint
peek can still return 503; nothing committed yet).
- a transport drop AFTER >= 1 byte has streamed (the 200 is committed) -> DEGRADE: write a `tts_degrade`
stderr line, END the generator, keep what played. NEVER raise into the committed StreamingResponse.
invariant: format is ALWAYS "wav" (DEC-3); never mp3/opus/pcm from this seam.
```
### FN chunk_text (DEC-10 long-form splitting; pure)
### FN chunk_text (DEC-10 long-form splitting; pure) — RETIRED 2026-08-07 (chatterbox chunks internally; deleted). Historical:
```
chunk_text(text, budget=_TTS_CHUNK_CHAR_BUDGET) -> list[str]
# Split into synthesis chunks each <= budget chars. Paragraph-first (seams on blank lines), greedy pack.
@@ -196,7 +273,7 @@ chunk_text(text, budget=_TTS_CHUNK_CHAR_BUDGET) -> list[str]
input has no boundary at all. budget = 71.2s * 0.75 * 14 c/s ≈ 747 (75% of cap for prosody).
```
### FN tts_stream_long (DEC-10 orchestrator — concatenate per-chunk synthesis)
### FN tts_stream_long (DEC-10 orchestrator) — RETIRED 2026-08-07 (no per-synth cap; deleted, its yielded_any degrade folded into tts_stream). Historical:
```
tts_stream_long(text, *, voice, dials, client, url=ZONOS_TTS_URL, budget=_TTS_CHUNK_CHAR_BUDGET) -> AsyncIterator[bytes]
# chunk_text(text) then synth each chunk with tts_stream (identical voice+dials+language). Emit chunk 1
@@ -211,7 +288,7 @@ tts_stream_long(text, *, voice, dials, client, url=ZONOS_TTS_URL, budget=_TTS_CH
chunk-0 mid-stream drop from raising into the 200.
```
### FN pad_to_dials
### FN pad_to_dials — RETIRED 2026-08-07 (DEC-7 affect dropped; PadState/EmotionDials/pad_to_dials all deleted). Historical:
```
pad_to_dials(pad: PadState | None) -> EmotionDials
# Map live PAD -> Zonos emotion dials (DEC-7).
@@ -225,27 +302,28 @@ pad_to_dials(pad: PadState | None) -> EmotionDials
endpoint coerces via PadState.from_obj; not defended inside pad_to_dials.)
```
### FN tts_endpoint (server.py, POST /api/tts — DEC-10a)
### FN tts_endpoint (server.py, POST /api/tts — DEC-10a; amended 2026-08-07)
```
POST /api/tts {text, agent_id?, p?, a?} -> audio/wav (chunked StreamingResponse)
# POST (not GET) so an arbitrarily long turn rides the body, not a length-capped URL (DEC-10). The server
# chunk-and-concatenates under the 71.2s/call cap (tts_stream_long). text capped 8000 chars, word-boundary
# truncated (a shared-3090 hold bound; the transcript still shows the full text). ALL of text/agent_id/p/a
# are untrusted open-world body fields — each degrades, never 500s (INV-TTS-4).
POST /api/tts {text, agent_id?} -> audio/wav (chunked StreamingResponse)
# POST (not GET) so an arbitrarily long turn rides the body, not a length-capped URL. The gateway chunks
# arbitrary-length text internally (DEC-10 RETIRED — no client concat); a single tts_stream call proxies it.
# text capped 8000 chars, word-boundary truncated (a shared-GPU hold bound; the transcript still shows the
# full text). text/agent_id are untrusted open-world body fields — each degrades, never 500s (INV-TTS-4).
# (The `p`/`a` PAD body fields are GONE — DEC-7 affect retired.)
steps:
- bad JSON / non-str text -> 400. Scrub lone surrogates from text (else httpx's utf-8 encode of the gateway
body 500s); if the scrubbed text is blank after strip -> 400. word-boundary truncate to 8000 (with a
mid-word HARD-CUT fallback when the last space sits at index <= limit//2).
- resolve voice: per-character map -> "donut", default Cora; a NON-str agent_id (unhashable) -> default voice.
- dials = pad_to_dials(PadState.from_obj({pleasure:p, arousal:a})) — from_obj hardens the parse (a huge-int
OverflowError / non-numeric / missing axis -> neutral read), never a 500.
- acquire the serialize lock (DEC-5, one stream at a time on the shared 3090); open tts_stream_long and PEEK
the first byte: a bad gateway OPEN on chunk 1 -> 503, and a 200 whose first bytes are NOT a RIFF header
-> 503 too (a mislabeled non-WAV body would decode as garbage) — both BEFORE committing a 200 (INV-TTS-4).
Any OTHER escape during the peek (CancelledError, httpx.InvalidURL) releases the lock+client, then propagates.
- return StreamingResponse piping the concatenated chunks; the generator's finally releases the lock +
closes the client (incl. the browser-abort path: a new turn's fetch() drops the POST). httpx.Timeout is
connect=10 / read=120 / write=10 / pool=10 (read=120 per infra-ops: a near-cap chunk can render slowly).
- resolve voice: per-character map -> "donut", default "glados_25s"; a NON-str agent_id (unhashable) -> default.
- acquire the serialize lock (DEC-5, one stream at a time on the shared GPU); open tts_stream and PEEK the
first byte: a bad gateway OPEN -> 503; an EMPTY 200 body (no bytes — an OOM synth, DEC-9a) -> 503; and a
200 whose first bytes are NOT a RIFF header -> 503 too (a mislabeled non-WAV body would decode as
garbage) — all BEFORE committing a 200 (INV-TTS-4). Any OTHER escape during the peek (CancelledError,
httpx.InvalidURL) releases the lock+client, then propagates.
- return StreamingResponse piping tts_stream; the generator's finally releases the lock + closes the client
(incl. the browser-abort path: a new turn's fetch() drops the POST) and, on a committed mid-stream drop,
tts_stream degrades internally (ends the generator, no raise). httpx.Timeout is connect=10 / read=120 /
write=10 / pool=10 (read=120 per infra-ops: a long synth can render slowly).
```
### FN pin_kb_context (kb_bridge.py — RETIRE-READY, INV-KB-1)
@@ -278,14 +356,15 @@ pin_kb_context(question: str, agent_id: str | None, *, client) -> list[dict] #
searches in-voice natively.
```
### FN client: speakOnDone (index.html — Web Audio STREAMING, DEC-2)
### FN client: speakOnDone (index.html — Web Audio STREAMING, DEC-2; amended 2026-08-07)
```
on SSE `done`:
if !ttsEnabled(): return # INV-TTS-2
cancelTts() # INV-TTS-3: abort fetch + stop scheduled nodes
POST /api/tts {text (sliced to the 8000 cap), agent_id?, p?, a?} -> reader # DEC-10a: POST body, not a GET URL
POST /api/tts {text (sliced to the 8000 cap), agent_id?} -> reader # DEC-10a: POST body. NO p/a (DEC-7 retired).
loop: read chunk -> skip ONE WAV header up to the data chunk (bounded 64KiB) -> int16 LE PCM -> Float32 ->
AudioBuffer -> BufferSource.start(playAt) GAPLESSLY -> playAt += buf.duration # progressive, TTFA ~0.5s
AudioBuffer(sampleRate=24000) -> BufferSource.start(playAt) GAPLESSLY -> playAt += buf.duration
# SR = 24000 (chatterbox; was 44100 for Zonos — MUST match or the voice plays ~1.8x too fast). TTFA ~0.5s.
first scheduled node -> "▶ voiced". HARD failure (non-OK HTTP, or 64KiB with no WAV header) -> ticker + skip;
ABORT/cancel (INV-TTS-3 new-turn) + bare network error -> SILENT skip (INV-TTS-4, cancel is not a failure)