---
contract_version: "2.1"
module: "ratatoskr.web.tts_kb"
purpose: >
  A voiced, corpus-grounded Tier-3 interview character in the ratatoskr web
  console. Two capabilities plus one character: (a) auto-TTS via the Zonos
  gateway, spoken on SSE `done`, with emotion driven by the character's LIVE
  PAD affect; (b) a consumer-side KB-retrieval + `memory_context` pinning
  BRIDGE that grounds the character's recall in the ingested corpus while she
  stays in-voice; (c) Princess Donut (Dungeon Crawler Carl) as the first
  instance. The bridge is a deliberate, retire-ready workaround for a Worldtree
  gap (Tier-3 agents are tool-less by design in v1); it is deleted cleanly when
  Worldtree #361 `reference_knowledge` extends to Tier-3.
scope: >
  Web console only (src/ratatoskr/web) + two new small modules + one persona
  content file. The Bifrost provider planes and the ratatoskr.wt SDK-adapter
  core are UNTOUCHED. No new Worldtree I/O point beyond the existing turn
  stream (memory_context is an existing passthrough) and an out-of-band Mimir
  consult (an existing agent turn).
touches:
  - src/ratatoskr/web/server.py          # /api/tts route + the retrieval-pinning seam on the turn path
  - src/ratatoskr/web/static/index.html   # speak-on-done playback, 🔊 toggle, <audio> sink; turn POST carries agent_id
  - src/ratatoskr/web/entrypoint.py       # RATATOSKR_TTS_URL override (the tts swap seam)
  - src/ratatoskr/tts.py                  # NEW — Zonos gateway client + PAD->emotion-dial mapping
  - src/ratatoskr/kb_bridge.py            # NEW, RETIRE-READY — consumer-side retrieval + memory_context pinning
  - src/ratatoskr/wt.py                   # stream_turn gains a memory_context passthrough (seam-review: the contract's original touch list undercounted this by one file; the param defaults None so the bridge's RETIREMENT stays inert — deleting kb_bridge.py + the one call-site leaves wt.stream_turn's SDK-parity param harmless)
  - docs/characters/donut.md              # NEW — Princess Donut persona (content; the tier3 define source)
depends_on:
  - "Zonos gateway: POST http://10.100.79.3:8890/v1/audio/speech (infra-ops; WG-internal, no auth; wav; verified 2026-08-02)"
  - "Worldtree turn stream: memory_context[] passthrough (SDK stream_turn already forwards it verbatim)"
  - "Worldtree agents.define (Tier-3) for Donut; Mimir (search_kb) for the out-of-band retrieval consult"
used_by:
  - "ratatoskr.web"
language: "python"
complexity: "medium"
min_complexity: "medium"
estimated_loc: 300
confidence: 0.8
---

# Contract: Donut voiced interview (auto-TTS + KB-recall bridge)

> **⚠ SLICE 3 (KB-recall bridge) RETIRED 2026-08-02.** The `kb_bridge.py` module +
> its single `web/server.py` call-site were deleted per INV-KB-1 when Worldtree #383
> shipped native Tier-3 `reference_knowledge` (v1.0.0b167, live on :8081 + demo).
> Donut now searches the fiction wing (DCC corpus) natively, in-turn, with evidence
> packets (note_id + path provenance, confidence bucket) and a server-side grounding
> rule — strictly better than the consumer-side memory_context pinning it replaced
> (no separate consult round-trip, not gated by our out-of-band prompt). Retirement
> live-verified: Donut called `reference_knowledge` and grounded in DCC in-voice
> before deletion. `wt.stream_turn`'s `memory_context` param was KEPT (inert SDK
> parity). The DEC-6 / INV-KB-* / FN pin_kb_context sections below are retained as
> historical record of what was built and why it retired. **Slices 1 (persona) + 2
> (auto-TTS) remain LIVE.**

Migration-style contract: three separable slices (persona / TTS / KB-bridge),
each independently shippable. Slice order is chosen for fastest visible result.

## Decisions (DEC)

- **DEC-1 — direct :8890 coupling.** Proxy straight to the Zonos gateway, not the
  swappable `ext-tts` LiteLLM alias. Rationale: the emotion dials (the whole
  point — affect-driven voice) don't pass through `ext-tts`. Accept the
  Zonos coupling; the `tts.py` client is the single swap seam if we ever move.
- **DEC-2 — STREAMING, play-as-it-arrives (amended 2026-08-02, operator-directed).**
  The gateway ALREADY streams: `POST /v1/audio/speech` relays a chunked int16 WAV
  (transfer-encoding: chunked, placeholder 0xFFFFFFFF RIFF/data sizes) as it synthesizes
  — TTFB ~0.44s vs ~7s total (infra-ops verified). So ratatoskr PROXIES THE CHUNKS
  STRAIGHT THROUGH (`tts_stream`, `GET /api/tts`) and the browser plays a progressive
  `<audio src>`; NEVER buffer, NEVER rewrite the placeholder header (a rewrite needs the
  whole clip and defeats streaming — the bug the original buffered `tts_synthesize` +
  `_finalize_wav_header` hit). The placeholder-size WAV is DESIGNED for `<audio src>`
  progressive playback (validated in Chromium: plays, currentTime advances, no MSE
  needed). Supersedes the original "full-synth latency accepted / no streaming."
  (Amended 2026-08-02: the browser path is now Web Audio decoding raw int16 PCM, not
  `<audio src>` — Safari/WebKit rejects a 0xFFFFFFFF-length streaming WAV via `<audio
  src>` (NotSupportedError); and `/api/tts` is now POST, not GET. See DEC-10.)
- **DEC-3 — wav only.** `response_format:"wav"` (streaming int16 RIFF/WAVE). `mp3`/`opus`
  are accepted but silently return mislabeled PCM — never request them.
- **DEC-4 — server-side proxy.** Browser → `/api/tts` (nh3-dev) → gateway. The
  irv-ml1 host/URL never reaches the client (INV-TTS-1). No key exists, so
  INV-003 is trivially satisfied, but the proxy still stands (browser can't
  reach the WG net regardless).
- **DEC-5 — serialize one synth per turn.** The gateway shares a single 3090
  (~90% VRAM) with other TTS engines; do not parallelize. A new turn cancels any
  in-flight synth + playback.
- **DEC-6 — KB-recall = consumer-side retrieval + memory_context pinning, as a
  RETIRE-READY bridge (worldtree-dev ruling, wyrd-proven pattern).** Tier-3 is
  tool-less by design (`build_tier3_agent_context` hard-codes `tool_schemas=[]`);
  agent.call:mimir is rejected (recursion). So per interview turn: ratatoskr
  consults Mimir out-of-band for the question's topic, extracts the cited
  evidence, and POSTs Donut's turn with `memory_context=[{kind:"corpus_reference",
  text:"<extract>"}]`. She answers in-voice, framing it as her own memory. The
  bridge is isolated behind ONE seam (INV-KB-1) so it deletes cleanly when
  Worldtree #361 `reference_knowledge` extends to Tier-3 (operator-flagged as an
  integral gap; worldtree-dev surfacing the extension to Vuong).
  - **DEC-6 status (2026-08-02): bridge RETIRED, native `reference_knowledge` live but
    EMPTY.** The bridge was deleted (`09e4257`) when WT #383 native `reference_knowledge`
    (b167) shipped — Donut now calls the tool in-turn. But the tool returns zero hits for
    every query. Root-caused 2026-08-02: Mimir's `search_library` DOES find the DCC corpus
    (main wing, score ~0.03), so the store is NOT empty and this is NOT a ratatoskr gap —
    WT's native `reference_knowledge` isn't surfacing content Mimir retrieves fine (likely
    a wing-scope mismatch — tool scoped to the failed `fiction` wing — or a confidence
    threshold rejecting the weak ~0.03 hits). Escalated to worldtree-dev. Until fixed,
    Donut recalls from her own character knowledge (degrades in-voice per the persona).
- **DEC-7 — affect-driven emotion.** Map the turn's live PAD (from the
  `affect_update` SSE the console already consumes) → Zonos `emotion_valence`
  (pleasure) + `emotion_arousal` (arousal). This reframes the feature as voice
  OBSERVABILITY (hear the affect the persona pane shows), not chat-app TTS.
- **DEC-8 — voice: custom "donut" is REGISTERED (superseded the preset-first plan).**
  The original plan was a theatrical Zonos preset first (Miranda/Penny/Emmie), custom
  "donut" later. But infra-ops registered + verified the custom `voice:"donut"` before
  slice-2 build (GET /v1/voices returns Donut; case-folded), so `_TTS_VOICE_MAP` maps
  `ratatoskr:donut → "donut"` directly — no interim preset. Non-interview agents still
  fall to the gateway default (Cora). (Amended 2026-08-02 per heid-code-review: 3 arms
  flagged the code as DEC-8 drift; a live gateway read INVERTED the remedy — the code is
  correct, DEC-8's "preset now" was stale.)
- **DEC-9 — pin English (added 2026-08-02, operator-directed).** Zonos is multilingual;
  with no `language` field it drifts into other-language phonemes / gibberish on names,
  numerics, and long spans (operator report). `gateway_body` pins `language:"en-us"` on
  every call (gateway accepts it, verified live). The persona's dialogue-only rewrite
  (`docs/characters/donut.md`) removes the other gibberish vector — asterisk RP action
  beats were being voiced verbatim.
- **DEC-10 — long-form chunk-and-concatenate (added 2026-08-02, operator-directed;
  infra-ops recipe 01KZ1FKX…).** The Zonos model hard-caps ONE synthesis at
  `max_tokens=6144` = 71.2s of audio (6144 / 86.3 Hz codec frame rate; the gateway 400s
  above 6144 — an architectural sequence limit, unraisable). So a turn longer than ~71s
  truncated mid-stream. Fix: split the text client-side and concatenate the PCM
  (`chunk_text` + `tts_stream_long`). Chunking is paragraph-first + greedy (operator call:
  fewer, fuller chunks for prosody), targeting ~75% of the cap per chunk (~747 chars @
  ~14 c/s); the 25% headroom covers char→audio-seconds variance (the cap is on TOKENS —
  6144 = 71.2s ALWAYS, a codec-frame constant regardless of delivery, infra-ops — while the
  budget is in CHARS, a proxy that stretches under slow/expressive delivery) so no chunk clips;
  oversized paragraphs fall back to sentence packing, oversized sentences to clause/word
  sub-splitting. Concatenation emits chunk 1's WAV verbatim then chunks 2..N header-
  stripped → ONE continuous int16-PCM stream (never bury a RIFF header mid-stream —
  infra-ops). Identical voice+dials+language per chunk for uniform delivery. `/api/tts`
  becomes POST (DEC-10a) so the full text rides the body, not a length-capped URL; the
  outer text cap rises 2000→8000 (a shared-3090 hold bound, not a URL bound).

## Invariants

- **INV-TTS-1 [hard]** — the Zonos gateway host/URL never reaches the browser;
  all synthesis goes through `/api/tts`.
- **INV-TTS-2 [hard]** — TTS is opt-in: a 🔊 toggle (default OFF), persisted to
  localStorage (mirrors the theme/cot-toggle pattern). No speech without it.
- **INV-TTS-3 [hard]** — one synth in flight at a time; a new turn cancels the
  prior synth request AND stops in-flight playback (cancel-on-new-turn).
- **INV-TTS-4 [hard]** — TTS failure is non-blocking: a gateway error, non-wav
  body, or playback failure skips audio; the turn/transcript is unaffected. Logging is
  scoped to GENUINE failure: a committed-200 mid-stream/later-chunk degrade writes a
  `tts_degrade` stderr line (server) or a `no WAV header` ticker (browser); a browser-side
  ABORT/cancel (INV-TTS-3 new-turn) is deliberately SILENT — cancellation is not a failure.
- **INV-KB-1 [hard]** — the KB bridge is import-isolated behind a single seam:
  `server.py`'s turn path calls exactly one function `pin_kb_context(question,
  agent_id) -> list[memory_context] | []`. Retiring the bridge = delete
  `kb_bridge.py` + that one call-site; no other file changes. `kb_bridge` imports
  nothing from the TTS or SDK-adapter core.
- **INV-KB-2 [hard]** — the retrieval channel is never persisted: `memory_context`
  is per-turn, `kind="corpus_reference"` (a free label), and rides the turn POST
  body only.
- **INV-KB-3 [hard]** — KB-retrieval failure is non-blocking: a failed/empty
  Mimir consult pins nothing (`[]`) and Donut answers without the extract; never
  block or fail her turn on retrieval.
- **INV-KB-4** — the bridge is scoped to characters that opt in (a per-agent
  allowlist / config), not applied to every turn — only interview characters
  that need corpus recall pay the retrieval round-trip.

## FN blocks

### FN tts_stream  (the per-CHUNK primitive — DEC-2 streaming; wrapped by tts_stream_long)
```
tts_stream(text, *, voice, dials, client: httpx.AsyncClient, url=ZONOS_TTS_URL) -> AsyncIterator[bytes]
  # Open the gateway's CHUNKED stream (client.stream("POST", url, json=gateway_body(...))) and YIELD wav
  # chunks as they synthesize. Pass through verbatim — never buffer, never rewrite the placeholder header.
  # gateway_body(text, voice, dials) = {input, voice, response_format:"wav", language:"en-us" (DEC-9),
  #   **dials.to_body()}.
  precondition: text non-empty. Voice membership in /v1/voices is GATEWAY-enforced, not client-asserted.
  postcondition: yields the gateway's chunked int16 streaming WAV bytes unmodified (0xFFFFFFFF placeholder
                 sizes intact). This is ONE synthesis (<= 71.2s cap, DEC-10); tts_stream_long stitches many.
  error: a non-200 OPEN or connect/transport failure -> TtsUnavailable BEFORE the first chunk (so the
         endpoint can still return 503); a mid-stream drop just ends the generator.
  invariant: response_format is ALWAYS "wav" (DEC-3); never mp3/opus. language ALWAYS "en-us" (DEC-9).
```

### FN chunk_text  (DEC-10 long-form splitting; pure)
```
chunk_text(text, budget=_TTS_CHUNK_CHAR_BUDGET) -> list[str]
  # Split into synthesis chunks each <= budget chars. Paragraph-first (seams on blank lines), greedy pack.
  steps:
    - strip; empty/whitespace -> [].
    - whole paragraphs (each <= budget) greedily pack together, joined "\n\n".
    - a paragraph > budget flushes the pending run, then sentence-packs (split (?<=[.!?])\s+, join " ");
      a sentence > budget sub-splits on clause (, ; :) then space, hard-cut mid-word only as last resort.
  postcondition: every chunk non-empty and <= budget; word order preserved; no split mid-word unless the
                 input has no boundary at all. budget = 71.2s * 0.75 * 14 c/s ≈ 747 (75% of cap for prosody).
```

### FN tts_stream_long  (DEC-10 orchestrator — concatenate per-chunk synthesis)
```
tts_stream_long(text, *, voice, dials, client, url=ZONOS_TTS_URL, budget=_TTS_CHUNK_CHAR_BUDGET) -> AsyncIterator[bytes]
  # chunk_text(text) then synth each chunk with tts_stream (identical voice+dials+language). Emit chunk 1
  # VERBATIM (WAV header + PCM); chunks 2..N via _pcm_after_header (strip up to+incl the `data` id+size) so
  # the browser decodes ONE continuous int16-PCM stream after a single leading header (infra-ops: one header).
  error: the pivot is `yielded_any`, NOT the chunk index. A failure BEFORE the first byte (an OPEN failure)
         propagates as TtsUnavailable -> endpoint peek -> 503 (nothing committed yet). A failure AFTER bytes
         have streamed — a MID-STREAM drop on chunk 0 OR a later chunk, past the committed 200 — degrades:
         drop the tail, keep what played, write a `tts_degrade` stderr line, RETURN (never raise into the
         committed StreamingResponse). httpx wraps aiter_bytes in `except RequestError`, so a mid-stream
         drop arrives as TtsUnavailable, not a clean generator end — the yielded_any gate is what keeps a
         chunk-0 mid-stream drop from raising into the 200.
```

### FN pad_to_dials
```
pad_to_dials(pad: PadState | None) -> EmotionDials
  # Map live PAD -> Zonos emotion dials (DEC-7).
  steps:
    - none/malformed pad -> neutral dials (emotion_enabled=False) [degrade-not-crash].
    - emotion_valence = clamp(pad.pleasure, -1, 1); emotion_arousal = clamp(pad.arousal, -1, 1).
    - emotion_enabled = True; emotion_strength from a fixed default (tunable).
  invariant: total over any PAD the DECLARED surface produces (a PadState with float axes /
             None / out-of-range / NaN / inf / a non-PadState object) -> valid dials, never raises.
             (A PadState carrying NON-float axes is a type violation no call site constructs — the
             endpoint coerces via PadState.from_obj; not defended inside pad_to_dials.)
```

### FN tts_endpoint  (server.py, POST /api/tts — DEC-10a)
```
POST /api/tts  {text, agent_id?, p?, a?}  -> audio/wav (chunked StreamingResponse)
  # POST (not GET) so an arbitrarily long turn rides the body, not a length-capped URL (DEC-10). The server
  # chunk-and-concatenates under the 71.2s/call cap (tts_stream_long). text capped 8000 chars, word-boundary
  # truncated (a shared-3090 hold bound; the transcript still shows the full text). ALL of text/agent_id/p/a
  # are untrusted open-world body fields — each degrades, never 500s (INV-TTS-4).
  steps:
    - bad JSON / non-str text -> 400. Scrub lone surrogates from text (else httpx's utf-8 encode of the gateway
      body 500s); if the scrubbed text is blank after strip -> 400. word-boundary truncate to 8000 (with a
      mid-word HARD-CUT fallback when the last space sits at index <= limit//2).
    - resolve voice: per-character map -> "donut", default Cora; a NON-str agent_id (unhashable) -> default voice.
    - dials = pad_to_dials(PadState.from_obj({pleasure:p, arousal:a})) — from_obj hardens the parse (a huge-int
      OverflowError / non-numeric / missing axis -> neutral read), never a 500.
    - acquire the serialize lock (DEC-5, one stream at a time on the shared 3090); open tts_stream_long and PEEK
      the first byte: a bad gateway OPEN on chunk 1 -> 503, and a 200 whose first bytes are NOT a RIFF header
      -> 503 too (a mislabeled non-WAV body would decode as garbage) — both BEFORE committing a 200 (INV-TTS-4).
      Any OTHER escape during the peek (CancelledError, httpx.InvalidURL) releases the lock+client, then propagates.
    - return StreamingResponse piping the concatenated chunks; the generator's finally releases the lock +
      closes the client (incl. the browser-abort path: a new turn's fetch() drops the POST). httpx.Timeout is
      connect=10 / read=120 / write=10 / pool=10 (read=120 per infra-ops: a near-cap chunk can render slowly).
```

### FN pin_kb_context  (kb_bridge.py — RETIRE-READY, INV-KB-1)
```
pin_kb_context(question: str, agent_id: str | None, *, client) -> list[dict]   # memory_context items, or []
  # The bridge. Consumer-side retrieval + pinning (DEC-6).
  steps:
    - gate on the interview-character allowlist (INV-KB-4); not listed / blank question -> [].
    - out-of-band Mimir consult (a throwaway session + one turn), HARD-bounded by a timeout.
    - extract the answer text (prefer DoneEvent.response; fall back to text deltas), bounded length.
    - return [{"kind":"corpus_reference","text":<extract>}].
  error/empty/timeout: any failure or no hits -> [] (INV-KB-3; never raises to the turn path).
                       CancelledError (browser disconnect) is NOT caught — it propagates.
  no-hit sentinel (heid-code-review F7): the consult prompt asks Mimir to emit exactly NO_CORPUS_MATCH
                       when the search finds nothing relevant; pin_kb_context drops any extract containing
                       it -> [], so a non-empty "no results found" answer is never pinned as the character's
                       own memory. The token is artificial (no genuine passage contains it).
  session hygiene (heid-bug-hunt): the throwaway Mimir consult session is deleted (SDK sessions.delete) on
                       success/error/timeout via a caller-owned holder, so consults don't accumulate upstream.
  CONSULT PROMPT (foot-gun mitigation, tuned live 2026-08-02): force "search_library EXACTLY ONCE,
    no read_note" — converges Mimir in ~3-15s. The softer "do one search" phrasing let Mimir loop
    read_note<->search past a 25s ceiling on conversational (non-keyword) questions (live-observed).
  KNOWN LIMIT (surfaced by the live smoke, NOT a bridge defect): the bridge's GROUNDING VALUE is gated
    by Mimir's retrieval quality on the target corpus. DCC's fiction-wing index is currently weak
    (scores ~0.02, failed backfill — a standing worldtree-dev item), so hits are noisy/partial; the
    model's own DCC training knowledge already grounds Donut well, so the bridge is opportunistic here.
    Its real payoff is a corpus the model does NOT know AND that indexes cleanly.
  RETIREMENT: when Worldtree #361 reference_knowledge reaches Tier-3, delete this module + the single
              server.py call-site (wt.stream_turn's memory_context param stays, inert); Donut then
              searches in-voice natively.
```

### FN client: speakOnDone  (index.html — Web Audio STREAMING, DEC-2)
```
on SSE `done`:
  if !ttsEnabled(): return                                  # INV-TTS-2
  cancelTts()                                               # INV-TTS-3: abort fetch + stop scheduled nodes
  POST /api/tts {text (sliced to the 8000 cap), agent_id?, p?, a?} -> reader   # DEC-10a: POST body, not a GET URL
  loop: read chunk -> skip ONE WAV header up to the data chunk (bounded 64KiB) -> int16 LE PCM -> Float32 ->
        AudioBuffer -> BufferSource.start(playAt) GAPLESSLY -> playAt += buf.duration   # progressive, TTFA ~0.5s
  first scheduled node -> "▶ voiced". HARD failure (non-OK HTTP, or 64KiB with no WAV header) -> ticker + skip;
        ABORT/cancel (INV-TTS-3 new-turn) + bare network error -> SILENT skip (INV-TTS-4, cancel is not a failure)

WHY Web Audio, not <audio src>: Safari/WebKit REFUSES a streaming 0xFFFFFFFF-length WAV via <audio src>
(NotSupportedError — it can't compute duration/seek), which was the operator's live failure. Decoding the raw
int16 PCM ourselves and scheduling it into an AudioContext sidesteps every WAV-container quirk and works in all
engines. Validated in Chromium: 43 nodes scheduled, 5.1s decoded, no error.

AUTOPLAY UNLOCK: an AudioContext starts "suspended"; Safari + Chrome require resume() to originate from a user
gesture (then it stays running). _unlockTtsAudio() resumes it on the FIRST interaction anywhere (document
pointerdown/keydown) + toggle-on + each submit, so it's running before the ~15s-delayed speak-on-done. Validated:
ctx is "running" 6.5s after the gesture (past the transient-activation window). Page served no-store so a stale
cache can't hide these updates.
```

## Slice plan

1. **Persona** — author `docs/characters/donut.md` (corpus-grounded, interview frame,
   markdown-RP register) + `tier3 define` Princess Donut. Visible: she's in the picker,
   chattable (answers from training until slice 3). No code.
2. **Auto-TTS** — `tts.py` (gateway client + pad_to_dials) + `/api/tts` proxy + the
   index.html 🔊 toggle + speak-on-done playback + serialize/cancel. Visible: she's voiced,
   affect-modulated. TDD (respx-mock the gateway; live smoke on the console).
3. **KB-bridge** — `kb_bridge.py` + the one server.py seam. Visible: she recalls the DCC
   corpus in-voice. TDD. RETIRE-READY per INV-KB-1.

Out of scope: true audio streaming (DEC-2), custom voice registration (DEC-8, infra-ops
follow-up), extending the bridge to non-interview agents.
