Voice assignment moves from the hardcoded server map to ~/.config/ratatoskr/
voices.json (per-agent voice + optional narration_voice). An agent with a
narration_voice gets a two-voice split: quoted speech in `voice`, narration in
`narration_voice`, synthesized per-span and stitched under one WAV header.
- new src/ratatoskr/voices.py: load_voice_config (degrade-not-crash),
segment_dialogue (quote-based, straight + curly), resolve_voice_spans
- tts.py: tts_stream_stitched replaces tts_stream — serial per-span synth,
span 0 verbatim, spans 1..N header-stripped -> one gapless 48kHz stream;
a single-span list is a byte-identical passthrough (no single-voice regression)
- server.py: _tts_endpoint resolves spans from app.state.voice_config; the
hardcoded _TTS_VOICE_MAP is retired; create_app gains a voice_config param
- entrypoint.py: loads voices.json at startup
- contract DEC-11 + INV-TTS-5/6/7; initial config donut->donut,
sindra->miranda (dialogue) / emmie (narration)
Live-verified on :8765: Sindra mixed turn -> 2 dots calls (emmie+miranda)
stitched into one 48kHz WAV with a single RIFF header; Donut single-voice
unchanged. 545 tests green (incl. new test_voices.py).
Sindra now voices with the dots "miranda" voice (operator-directed);
donut stays on "donut", other agents fall to the "glados" default.
One entry in _TTS_VOICE_MAP + a test; contract DEC-8 updated.
Live-verified on :8765 (/api/tts with agent_id=ratatoskr:sindra -> 200
audio/wav @ 48kHz).
The long-turn "swaps to German" garble was NOT a language leak (infra-ops's
initial framing) and NOT the sampling entropy my interim curb targeted. The real
cause, signal-measured by infra-ops: the Chatterbox Turbo model over-runs its
generation TAIL — a long single generation degrades into garble/dead-air in its
final ~2-3s (voiced-tail zero-crossing rate 1.58x the middle). The gateway's
unbounded chunk-size ratchet built 300-600 char mega-chunks that landed in that
zone, and streaming concatenated each bad tail.
My interim curb (top_k 1000->80, top_p 0.95->0.85, temp 0.8->0.5) made it WORSE:
tight sampling pulls the degradation onset to a shorter length (~200 chars vs
~300 at defaults), so it fights the server-side fix rather than helping.
Fixed server-side (infra-ops, chatterbox-fast image :v2): a max_chunk_chars=250
cap bounds each generation below the ~300-char onset -> clean prosodic chunks
(verified ZCR 1.58x -> 0.64x; operator ear-confirmed clean audio + clean joins).
Consumer side, this commit:
- Revert the sampling knobs: gateway_body back to {text, voice, format, stream},
send full text with the gateway's default sampling. The server chunks at 250.
- Keep the /api/tts empty-200 -> 503 guard as hygiene (DEC-9a; the shared-3090
OOM that produced empty 200s is also resolved — Zonos moved off the card).
Contract DEC-9 rewritten with the resolved root cause + the two wrong hypotheses;
DEC-9a marked kept-as-hygiene. 520 green.
Repoint the TTS client from the Zonos gateway (:8890 /v1/audio/speech) to
chatterbox-fast (:8197 /tts — bespoke non-OpenAI {text,voice,format,stream}
schema, no auth, 24kHz, infra-ops-verified). tts.py stays the single swap seam.
Dropped, no backward-compat (pre-v1):
- Affect (DEC-7): the Turbo checkpoint has no emotion knob, so PadState,
EmotionDials, pad_to_dials, the /api/tts p/a fields, and the browser pad
argument are deleted. Voice is now flat.
- Client-side chunking (DEC-10): chatterbox has no per-synth cap and chunks
internally, so chunk_text/tts_stream_long/_pcm_after_header are deleted; a
single tts_stream call voices a whole turn, the mid-stream yielded_any degrade
folded into it.
- Language pin (DEC-9): no language field; re-purposed to sampling curbs (below).
Fixed / added:
- Browser Web Audio sample rate 44100 -> 24000 (the chatterbox rate).
- Default voice Cora -> glados_25s; donut registered lowercase at /refs/donut.wav.
- English-drift curb: Turbo is multilingual-leaky and wanders off English on a
long generation (the gateway scheduler ratchets chunk size unbounded). Tighten
sampling in gateway_body: top_k 1000->80, top_p 0.95->0.85, temperature
0.8->0.5. These reduce drift probability; the guaranteed fix is a server-side
max-chunk cap (infra-ops, greenlit).
- OOM guard (DEC-9a): a long generation can OOM the shared 3090, returning 200
with a 0-byte body; /api/tts surfaces an empty 200 as 503 rather than
committing silent audio.
Contract donut_voiced_interview.contract.md amended: migration banner, DEC-1/3/8
amended, DEC-7/9/10 retired with historical notes, DEC-9a added.
Tests rewritten to the new wire; 520 green. Live-smoked against the gateway
(24kHz synth + endpoint proxy + web console). persistent-memory.md committed
alongside (commit-along).
TTS fixes + hardening for the Donut voiced interview.
Feature:
- gibberish -> pin `language: "en-us"` on every gateway call (DEC-9); the
multilingual model drifted into other-language phonemes without it.
- truncation -> the Zonos model hard-caps one synthesis at 6144 tokens /
71.2s (infra-ops). Chunk client-side (paragraph-first, greedy to ~75%
of cap for prosody; sentence/clause fallback) and concatenate the int16
PCM behind ONE WAV header (DEC-10). /api/tts becomes POST so a long turn
rides the body, not a length-capped URL (DEC-10a).
- persona -> dialogue-only rewrite (no asterisk RP beats -- they were being
voiced as gibberish) + always consult the native `reference_knowledge`
tool before answering (retires the stale kb_bridge references). Pushed
live to ratatoskr:donut.
Heid code-review + bug-hunt hardening (4-arm panels, triaged):
- untrusted /api/tts body fields degrade, never 500: huge-int PAD
(OverflowError), non-str agent_id (unhashable .get), lone surrogates
(utf-8 encode), whitespace-only text.
- serialize lock + client released on every peek escape (cancel /
InvalidURL) -- previously a permanent deadlock.
- a mid-stream drop after a committed 200 degrades (keeps what played),
never raises into the response; a non-WAV 200 body is rejected (RIFF
sniff + bounded header scan) instead of decoded as garbage.
546 tests green; long-form live-verified (106.6s, one header). Contract
brought canonical (DEC-9/10, FN chunk_text/tts_stream_long, POST endpoint,
INV-TTS-4 logging scope, FN pad_to_dials domain). reference_knowledge
empty-recall root-caused to a Worldtree wing-misfile (escalated to
worldtree-dev; not ratatoskr code).
Operator confirmed the "TTS blocked" was NotSupportedError on Safari — WebKit refuses a
streaming 0xFFFFFFFF-length WAV via <audio src> (can't compute duration/seek), exactly
as infra-ops warned. Replaced the <audio src> playback with a Web Audio path that works
in all engines:
- speakOnDone: fetch the chunked /api/tts stream, skip the WAV header to the data chunk,
decode int16 LE PCM -> Float32, and schedule the samples GAPLESSLY into an AudioContext
as they arrive (BufferSource per chunk, playAt += buf.duration). Progressive, TTFA
~0.5s. Decoding the raw PCM ourselves sidesteps every WAV-container quirk.
- unlock: an AudioContext starts suspended; Safari + Chrome need resume() from a user
gesture. _unlockTtsAudio() now resumes the ctx on the first interaction anywhere +
toggle-on + submit, so it's running before the ~15s-delayed speak-on-done.
- cancelTts: aborts the fetch + stops all scheduled BufferSource nodes.
Validated in Chromium (Playwright, strict autoplay): 43 nodes scheduled, 5.1s of PCM
decoded, ctx "running" 6.5s post-gesture, zero errors. Headless WebKit can't launch here
(missing system libs — an infra-ops install), so the operator's live Safari is the final
check; the code is standard Web Audio Safari has supported for years.
Contract FN client:speakOnDone updated (Web Audio; the Safari NotSupportedError reason).
Operator: play-as-it-arrives, don't wait for the whole clip. infra-ops confirmed the
Zonos gateway ALREADY streams (chunked int16 WAV, TTFB ~0.44s vs ~7s total; placeholder
0xFFFFFFFF sizes are DESIGNED for progressive <audio src>). The buffering was entirely
in our proxy, and the _finalize_wav_header rewrite (6c3c08b) FORCED it — computing the
real sizes needs the whole clip.
The fix — pipe the chunks straight through:
- tts.py: buffered tts_synthesize + _finalize_wav_header REMOVED; tts_stream (an async
generator over the gateway's chunked response) + gateway_body added. Never buffer,
never rewrite the placeholder header.
- server.py: /api/tts is now GET (so a browser <audio src> plays it progressively) →
a chunked StreamingResponse piping the gateway; peeks the first chunk so a bad gateway
OPEN still returns 503; the serialize lock is held across the stream and released on
completion/abort; PAD rides p/a query floats.
- index.html: speakOnDone sets <audio src="/api/tts?..."> (streaming) instead of
fetch->blob; dropped the blob machinery. AUTOPLAY UNLOCK: _unlockTtsAudio() plays a
silent WAV within the toggle/submit gesture so the delayed play() isn't blocked — the
actual cause of "no audio" (play() fires ~15s after the keypress, past the browser's
transient-activation window).
Live-verified: GET /api/tts is transfer-encoding: chunked, TTFB 0.46s. Playwright with
--autoplay-policy=document-user-activation-required: the streaming <audio src> plays
progressively (currentTime advances, no decode error, no MSE fallback needed) 6.5s after
the gesture — proving the unlock's persistent element flag. 521 green.
DEC-2 amended (streaming supersedes "no streaming"); FN tts_stream / tts_endpoint updated.
Worldtree #383 shipped native Tier-3 reference_knowledge (v1.0.0b167, live on :8081 +
demo): every Tier-3 agent context now carries the tool automatically, with evidence
packets (note_id + path provenance, confidence bucket) and a server-side grounding
rule. That supersedes the interim consumer-side memory_context pinning bridge (slice
3), so it is deleted per its INV-KB-1 retire seam.
Removed:
- src/ratatoskr/kb_bridge.py + tests/test_kb_bridge.py (the whole module).
- server.py: the pin_kb_context import + the single turn-path call-site (reverted to
the pre-bridge wt.stream_turn call), the SSE keepalive that only covered the consult
delay, and the bridge-only agent_id plumbing (TurnHandle.agent_id + the submit read).
- index.html: agent_id dropped from the turn POST body.
- test_web_server.py: TestKbBridgeWiring (tested the removed call-site).
Kept:
- wt.stream_turn's memory_context param (inert SDK-parity passthrough; worldtree-dev
concurred it stays) + its forwarding tests.
- the non-str content 400 guard (general input hygiene, not bridge-specific).
Retirement LIVE-VERIFIED before deletion: a Donut session on :8081/b167 carries
builtin_tools=['reference_knowledge']; she called it and grounded in the DCC Collapse
content fully in-voice, degrading gracefully on absent content. 524 green.
Contract marks slice-3 RETIRED (historical record retained). #383 closed.
Triaged the heid-code-review panel (3 arms; reconciled against 56dce00 — three
findings already closed by the bug-hunt, and the two firewalled lenses converged
independently on the same three defects). Fixed the real one + contract precision.
Code:
- kb_bridge: no-hit sentinel (F7, the sharpest solo). The consult prompt asks Mimir
to emit NO_CORPUS_MATCH when nothing is relevant; pin_kb_context drops any extract
containing it -> [], so a non-empty "no results found" answer is never pinned as the
character's own memory. Live-proven: a grounding query pins (5.6s), a nonsense query
-> [] (0.7s); the sentinel is artificial so `in` can't false-positive on a passage.
Contract (the CODE is correct; the spec had drifted/undercounted — kept canonical):
- DEC-8: the custom "donut" voice was registered EARLY (verified live), so mapping
ratatoskr:donut -> "donut" is right; "preset now" was stale. A live gateway read
INVERTED the 3-arm remedy (reverting to a preset would have been the regression).
- FN tts_synthesize: declared the `url` swap-seam param (F3); voice membership is
gateway-enforced not client-asserted (F2); the postcondition is a container-level
RIFF/WAVE check, not a 16-bit-PCM fmt parse.
- FN tts_endpoint: pad is BROWSER-SENT per DEC-7, not a server PAD lookup (F5);
documented the 413 text cap.
- FN pin_kb_context: documented the sentinel + the session-delete hygiene.
Tests (real coverage gaps):
- the read_note prompt test asserts the distinguishing "do NOT call read_note" phrase,
not the bare token an inverted prompt would also carry (#8 mutation-blind).
- extract-bound asserts the literal 2000, not the impl constant it slices by (#9).
- route roster asserts /api/tts + /api/sessions/{id}/messages (#10 undercount).
- new server test: a degraded KB consult ([]) still streams the turn to done (F9).
Accepted (not fixed): caller-supplied agent_id (LAN/no-auth debug-tool trust model);
no DEC-5 concurrency test (asyncio.Lock is trivially correct — a test would test
asyncio, not our code). 545 green.
Grounds the interview character in the ingested corpus while she stays in-voice.
Tier-3 agents are tool-less by design in v1, so this is the consumer-side
workaround (DEC-6, worldtree-dev ruling): per opted-in interview turn, ratatoskr
consults Mimir out-of-band, extracts the passages, and pins them as memory_context
on the character's turn. She frames the pinned corpus as her own memory.
- src/ratatoskr/kb_bridge.py (new, RETIRE-READY): pin_kb_context — THE single seam
(INV-KB-1). Allowlist-gated (INV-KB-4: ratatoskr:donut only), hard-timeout-bounded,
degrades to [] on any failure/timeout/empty (INV-KB-3, never raises; CancelledError
propagates). Imports nothing from the SDK-adapter / TTS core. aclosing() closes the
SDK stream deterministically on the DoneEvent break.
- wt.stream_turn: memory_context passthrough (defaults None — inert for every other
caller and for the bridge's own retirement). Seam-review catch: the contract's
original touch list undercounted wt.py by one file (recorded in the contract).
- web/server.py: TurnHandle.agent_id + the single pin_kb_context call-site on the
turn path; the browser now sends agent_id so the allowlist can gate.
- web/static/index.html: the turn POST carries agent_id.
Consult prompt tuned live: "search_library EXACTLY ONCE, no read_note" converges
Mimir in ~3-15s (the softer "do one search" phrasing looped past 25s on
conversational questions).
TDD: 12 kb_bridge unit tests + wt memory_context forwarding + 2 server wiring tests
(531 green). Live-smoked on :8081/b128: pin_kb_context grounds in the DCC corpus
(real excerpts, <20s) and Donut answers in-voice; degrades cleanly on a slow consult.
KNOWN LIMIT surfaced (not a bridge defect): DCC's fiction index is weak (failed
backfill, a worldtree-dev item), so grounding is opportunistic — the bridge's real
payoff is a corpus the model does not already know.
Per docs/contracts/donut_voiced_interview.contract.md (slice 3 of 3).
Slice 1 of the auto-TTS/voiced-KB-character build (operator ask "add auto-tts
to the web gui"): the donut_voiced_interview contract (validated), the Princess
Donut persona (corpus-grounded from a Mimir DCC pull), and ratatoskr:donut
defined on :8081 (server-side; in the picker). Slices 2 (Zonos auto-TTS) + 3
(retire-ready KB-bridge) are TO BUILD.
Snapshot captures the full build state + design (Zonos gateway :8890, voice
"donut" registered, affect-driven emotion dials; the worldtree-dev-ruled
consumer-side retrieval + memory_context pinning bridge, retire-ready) for the
post-clear resume, plus the arcs since v0.22.0 (SDK 1.1.2 repin, bifrost 1.1.5,
canonical sync, release-only versioning, the Sindra saga + local-index
schema-burial foot-gun, the Mimir #382 reference-consumer finding). Handoff at
/tmp/ratatoskr-dev-handoff.md. Release-only cadence: no tag.
vh