vh
7028c5bc11
Issue #7: mid-stream robustness fix discovered via 2026-05-22 crash. Long mimir TUI conversation (turn 93, 1077 events consumed) crashed on event 1078 with JSONDecodeError("Expecting value: line 1 column 1 (char 0)") from json.loads('') on an empty-data SSE frame. _iter_events unconditionally called json.loads on every dispatched event; when httpx_sse surfaces a frame with id: present but data: empty/missing (a known library-vs-spec divergence), parsing fails and propagates. Two-rule fix in _iter_events: - Empty sse.data (exact `== ''`): SKIP silently per SSE spec (keepalive semantics). Don't yield, don't advance last_sse_id, don't set terminal_seen. ORDERING: skip fires BEFORE _parse_sse_id, so a keepalive with a malformed id is still a keepalive (intentional). - Non-empty sse.data that fails json.loads: raise new MalformedSseData (sibling to MalformedSseId, mirrors raw[:200] truncation pattern). Wire-level protocol error; presenters route to [malformed_sse_data] + exit 22 in cli, transcript label + state→idle in tui (INV-008). Volva paraphrase round: 4 ambiguities, all amended: 1. INV-001 prose tightened — exact `sse.data == ''` rule made prominent; "keepalive" framing demoted to intent-not-rule; whitespace-only data explicitly listed as malformed (not skipped); specific state names (last_sse_id, terminal_seen) instead of vague "any counter". 2. STEPS pseudocode spells out the ordering — empty-skip happens BEFORE _parse_sse_id; empty-data with bad id is silently swallowed. 3. empty_data_skipped test description fixed (had off-by-one count + wrong wording around last_sse_id intermediate state). 4. (paired with #1 above). Volva code-review post-implementation: 3 findings, all addressed: F1 (test-gap): empty_data_skipped proves yielded events but not internal last_sse_id non-advancement. New empty_data_skip_preserves_last_seen_sse_id test probes via SseConnectionDropped.last_seen_sse_id after a drop following the skipped frame — if the skip had transiently advanced last_sse_id, the exception payload would carry the wrong value. F2 (precision, contract amend): MalformedSseData ERROR_ROUTING said "log truncated raw" but stream_turn doesn't log — sse_client is a library, presenters own observability. Amended to "propagate to caller (no logging at sse_client layer); presenters log exc.raw." F3 (test-gap): cli malformed_sse_data test asserted label but not `raw='X'` shape and not truncation. Tightened existing test + added malformed_sse_data_truncation with 5000-char payload — verifies MalformedSseData.raw truncation carries through the presenter's repr() rendering. **Calibration milestone**: issue #7 is the first issue with ZERO drift findings from Volva code-review. TDD caught all runtime behavior cleanly. The 3 findings were assertion-precision and architectural-correctness-of-wording, not behavioral. Hypothesis: tighter contract spec + smaller code surface shifts Volva's role from "catch behavioral drift" to "tighten observability + wording". Cumulative calibration table: #1 (4 findings, 3 drift + 1 test-gap), #2 (3, 1+1+1 precision), #3 (5, 3+1+1), #4 (8, 5+2+1), #7 (3, 0 drift + 2 test-gap + 1 precision). Contracts touched (all drift-check clean): - docs/contracts/issues/7.contract.md (new): the coordinating record. - docs/contracts/issues/1.contract.md: _iter_events STEP 3.0 empty-skip + ordering note; STEP 3.c JSONDecodeError → MalformedSseData; new MalformedSseData ERROR_ROUTING (propagate-to-caller wording per F2); 4 new TESTS entries including F1's last-seen probe. - docs/contracts/issues/3.contract.md: _run_turn ERROR_ROUTING + malformed_sse_data tests (incl. F3 truncation). - docs/contracts/issues/4.contract.md: INV-008 mentions MalformedSseData; _stream_turn_worker ERROR_ROUTING + new TEST.
Ratatoskr
A Worldtree Conversation API debug TUI. Runs up and down Worldtree's API surface — sessions, turns, persona, tools, admin events, Bifrost state — carrying messages between layers. Like the squirrel.
The product is the observability surface; chat is the input mechanism. Devs run Ratatoskr against a local Worldtree to watch a turn flow through every layer of the system, side-by-side, in one terminal.
Status
v0 scaffold. Design locked; implementation hasn't started. The dev team owns the implementation pass.
Read in this order
docs/design-brief.md— the locked design. Read this first. Every architectural decision is recorded with its rationale, the alternatives considered, and (where relevant) the operator's lock-in moment.docs/SPEC-PIN.md— what Worldtree spec version Ratatoskr is built against, where the vendored snapshot lives, and how to bump the pin.docs/conversation-api-spec.md— the vendored Worldtree spec snapshot. Read this to understand the API surface Ratatoskr consumes. Do not import anything from a Worldtree checkout — the boundary is the spec, not the code. Seedocs/design-brief.md§2.CLAUDE.md— Claude Code conventions for this repo (mostly inherited from the Corviduo template).persistent-memory.md— durable intent across context resets. Update as decisions and state evolve.
Quickstart
# 1. Environment
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"
# 2. Verify the spec pin
cat docs/SPEC-PIN.md # documented Worldtree SHA + bump procedure
# 3. Tests (none yet; scaffold only)
uv run pytest
# 4. Run against a local Worldtree (once implementation lands)
# Worldtree must be running: python -m core.conversation_api
ratatoskr --agent mimir
What this repo is NOT
- NOT a polished consumer for Worldtree's end-users — that's the web app.
- NOT a Worldtree admin tool — admin CLI is separate (
sessions_cli.pylives in Worldtree). - NOT a featuretracking shadow of the web app — when the Conversation API surface grows, Ratatoskr does not necessarily grow with it.
- NOT a remote-Worldtree debug client — file-tail surfaces (persona log, server log) assume local-dev posture.
The full negative-clause list lives in docs/design-brief.md §6.
Boundary rule
Ratatoskr depends on three things only:
httpx+httpx-sse(network layer)textual(TUI framework)- Worldtree's published Conversation API spec at the pinned SHA
Hard rule: no imports from a Worldtree checkout. No core.* imports,
no from worldtree.*, no shared models, no submodule of Worldtree. The
spec is the entire surface. Boundary smoke test at tests/test_no_worldtree_imports.py
enforces this in CI.
Version-skew strategy
The dev team is decoupled from Worldtree's dev team. To detect drift when Worldtree changes the API:
- Spec-version pin in
pyproject.toml(worldtree-spec-rev). Bump explicitly; bumps are a tracked action. - Recorded-SSE snapshot tests at
tests/snapshots/. Captured against a live Worldtree; replayed in CI. Re-record after every pin bump. - Conformance smoke test — boots Worldtree via Docker compose in CI, runs a one-turn happy path. Catches integration-level drift.
See docs/SPEC-PIN.md for the bump procedure.
Consumer-side discoveries
If you find a spec gap, ambiguity, or missing-but-needed endpoint while working on Ratatoskr: route the discovery back to worldtree-dev via althing rather than via PR on Worldtree directly. The separate-team boundary is intentional and helps catch spec gaps that an in-tree consumer would paper over.
Related repos
- Worldtree — the API Ratatoskr consumes
- brokkr-smithy — authored this design brief
- Skaldsong — another Worldtree API consumer (Python; render-and-aggregate pattern)
- mead-hall — another Worldtree API consumer (TypeScript; server-side broker)