ratatoskr/persistent-memory.md

# Persistent memory — ratatoskr

_Last updated: 2026-05-21_

This file captures durable intent and supporting evidence (goals, decisions,
foot-gun warnings, in-flight state) across context resets. Read it at session
start; treat it as one input alongside `CLAUDE.md` and the auto-memory system,
not as the single source of truth.

When durable state shifts enough to warrant capture, run `/snapshot` and
commit alongside the next commit per the persistent-memory commit-along rule
in `CLAUDE.md`.

---

## Repo purpose

Ratatoskr is a **dev-grade debug-observability TUI** for Worldtree's
Conversation API. The product IS the observability surface; chat is the
input mechanism. Devs run Ratatoskr against a local Worldtree to watch a
turn flow through every layer of the system, side-by-side, in one terminal:
agent SSE stream, persona/Vili affect dispatch, tool calls, Bifrost
handshake state, admin lifecycle events, optional raw server log.

Named after the squirrel that runs up and down Yggdrasil carrying messages
between layers. On-the-nose Worldtree resonance (Yggdrasil = the World Tree).

Origin: althing ask from worldtree-dev (thread `01KS3R34XD3N6HMK91VXESHGW7`,
2026-05-20) for the shape of a TUI Conversation API consumer. brokkr-smithy
ran the shape pass; operator's reframe routed it as a new repo with a
separate dev team rather than an in-tree Worldtree tool.

## Current state / in-flight

_As of 2026-05-21:_

**Status: v0 milestone landed.** Four issues complete end-to-end against
their contracts: `ratatoskr.sse_client` (#1), `ratatoskr.sessions` (#2),
`ratatoskr.cli --send` non-interactive presenter (#3), `ratatoskr.tui` shell
(#4). 164/164 tests GREEN (42 sse_client + 19 sessions + 58 cli + 43 tui +
2 boundary/metadata); ruff clean; all four issue contracts drift-check
clean. `ratatoskr` console-script entry point dispatches `--send` → cli
(stdout) and bare invocation → TUI (lazy textual import preserves issue #3
INV-001). All work pushed to `origin/main`.

`--send` validated end-to-end against personal Worldtree
(`http://10.250.50.152:8081`, mimir agent on qwen3.6-35-a3b). Smoke key
lives in gitignored `env.sh` (delivered by infra-ops 2026-05-21; see
[[personal-worldtree-smoke-target]] in auto-memory).

**Nothing in flight.** Branch: `main` (clean). Remote: `origin →
git@gitea.phasefinal.com:vh/ratatoskr.git`.

**Next natural moves:**

1. **TUI smoke (operator-side)** — `source env.sh && uv run ratatoskr --new
   --agent mimir` from an interactive terminal. Validates the Textual app
   lifecycle + post-Done markdown re-render end-to-end. Needs a TTY which
   CC sessions don't have.
2. **Side-pane issues** — design-brief §5 lists 5 side panes (Persona,
   Tools, AdminEvents, BifrostState, ServerLog). Each gets its own issue
   + contract + TDD pass on top of the shell. Persona is the natural first
   (file-tail of `persona.log` — cheap; no new Worldtree wire).
3. **Recorded SSE snapshot fixtures** from a running Worldtree. `--send
   --new` redirected to a fixture file IS the recording probe — capture
   outputs to `tests/snapshots/` for replay-based regression coverage.
   The smoke proved the recording probe works.
4. **Startup session picker** — design-brief §4 `DataTable` of `GET
   /sessions`. Modest scope; pairs naturally with the TUI shell.

## Recent decisions

Chronological log of decisions with `[YYYY-MM-DD]` prefix. One line per
decision. Captures rationale that won't be obvious from code alone.

- `[2026-05-20]` Project name **Ratatoskr** (squirrel on Yggdrasil — runs up and down carrying messages). Earlier candidate Andvari demoted on the cursed-ring association.
- `[2026-05-20]` **Separate repo, separate dev team.** Operator's call; the in-tree-at-Worldtree/tools/ alternative was considered and rejected to dogfood the API boundary.
- `[2026-05-20]` **No Worldtree-source imports.** Spec-only dependency. Triple version-skew mitigation: spec-pin in pyproject.toml + recorded-SSE snapshot tests + conformance smoke. Initial pin: `55101e909abcd2219833266b6f905c5bc956e0f0` (Worldtree v0.19.0). See `docs/SPEC-PIN.md`.
- `[2026-05-20]` **Textual** (not rich+prompt_toolkit). Driver: debug observability is the primary purpose, and a multi-pane dashboard with persistent side panes + independent scrollback is structurally application-shell-shaped. Volva consulted via cross-frontier second-opinion and converged on the same call.
- `[2026-05-20]` **`httpx-sse`** for SSE consumption. The server emits composite `{turn_id}:{seq}` `id:` lines (Worldtree INV-014) load-bearing for SSE-resume; hand-rolled `data:`-only parsing (the skaldsong pattern) silently drops these. Ratatoskr becomes the reference Python SSE-resume implementation.
- `[2026-05-20]` **Persona-pane PII posture: label-don't-refuse.** `persona.log` is process-wide; pane title flips between `[Persona — PROCESS-WIDE]` and `[Persona — session <id>…]` based on whether log lines carry session_id. Refuse-against-non-local was considered and rejected as paternalistic.
- `[2026-05-20]` **Server-stdout pane: opt-in via `--server-log <path>`.** No auto-detection of well-known paths.
- `[2026-05-20]` **Two-stage Ctrl-C.** First cancels in-flight turn server-side; second exits app. Ctrl-D bound to immediate exit.
- `[2026-05-20]` **Single-session-per-launch + startup picker.** No in-app `/switch`. CLI flags `--session <id>` and `--new` for scripted use. Session identity always visible in Textual footer.
- `[2026-05-20]` **Markdown rendering default-on; `--raw` opt-out.** Don't pre-design `--no-stream-formatting` (Volva: add only if streaming-markdown rendering is empirically ugly).
- `[2026-05-20]` **Non-interactive `--send` mode.** Single SSE consumer module, two presenters (TUI + stdout). Keeps Ratatoskr honest as an API consumer; useful for CI / scripted probes.
- `[2026-05-20]` **First contract: `ratatoskr.sse_client`.** Bundles `stream_turn` + `reconnect_turn` + `cancel_turn` + private `_parse_sse_id` into one module — the SSE-resume flow is coupled (cancel needs `turn_id` from the SSE wire `id:`, reconnect re-uses the same parsed `SseId`), so they share a contract. Hard invariant INV-002 makes the composite `{turn_id}:{seq}` `id:` parsing load-bearing — closes the foot-gun the design-brief §3 names (hand-rolled `data:`-only parsing silently drops the `id:`).
- `[2026-05-21]` **Contract converted to issue-scoped (issue #1).** Frontmatter shape switched from module-scoped (`module:`/`purpose:`) to issue-scoped (`target_module:`/`scope:`/`prd:`) per CONTRACT-FORMAT §2.1.I. `prd:` block pins to issue body hash. **Known parser stale-ness**: `contract_parser.py --validate` ERRORs on issue-scoped frontmatter — CONTRACT-FORMAT §2.1.L H10, a documented Brokkr-side follow-up. Parser is a canonical sync, so we do NOT patch it locally. Treat parser ERROR-on-issue-scoped as expected until canonical bumps.
- `[2026-05-21]` **Default issue-tracker labels seeded** (17 total). Sleipnir gating, triage, type, resolution, Ratatoskr-specific area labels (sse-client, tui, cli, observability).
- `[2026-05-21]` **Volva paraphrase + code-review across all 4 issues — calibration consistent.** Paraphrase rounds flag 3-5 contract ambiguities per issue; code-review rounds flag 3-8 code-vs-contract drifts after TDD-passing implementation. Hit rates: #1 paraphrase 3-of-5 amended / code-review 4 findings; #2 3-of-5 / 3 findings; #3 5-of-5 / 5 findings; #4 5-of-5 / 8 findings. The post-TDD code-review consistently catches three classes of gap the test-author's hypotheses don't cover: PRE-assertion boundary drift, exception-payload truncation / never-rendered-to-user observability misses, and "tested the state but not whether the user can see it" gaps (issue #4's primary finding: TUI footer state stored but never rendered to a visible widget — same-model TDD would systematically miss this).
- `[2026-05-21]` **Manual smoke is load-bearing — found a real defect tests couldn't.** First wire-level smoke against personal Worldtree (post-TDD, post-Volva-code-review on #4) revealed httpx's default 5s read timeout killed the SSE connection mid-stream during mimir's thinking phase (~30s LLM latency >> 5s read timeout). The unit/contract test infrastructure (respx-mocked SSE wire) doesn't model real LLM latency, so the gap was invisible at the test layer. Fix: caller-owned `httpx.AsyncClient` constructed with `timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0)`; defense in depth: `sse_client.stream_turn` ERROR_ROUTING catches `httpx.ReadTimeout` → `SseConnectionDropped`. Three contracts amended in-place to document the timeout policy. **Lesson: keep manual-smoke step in the per-issue cadence; mock-only validation is insufficient for streaming-against-real-server code.** Re-smoke succeeded: `[done] turn_id=88 model=qwen3.6-35-a3b duration_ms=2351`. Wire-compat envelope (personal v0.16.2 vs ratatoskr's v0.19.0 pin) confirmed end-to-end.

_For per-issue TDD implementation notes, Volva findings, and contract amendments, see the git log (commits `9703eb2..61c3941` carry the full per-issue trail with structured commit messages)._

## Tried and abandoned

Log of approaches that were tried and rejected, with rationale. Future-self
defense against re-attempting the same cul-de-sac.

- `[2026-05-20]` **rich + prompt_toolkit framework choice.** Considered first (during initial shape draft). Volva flagged that §1 and §5 pulled in opposite directions: a real side-panel observability surface would silently become a widget framework reimplementation. Operator's debug-observability reframe sealed the flip to Textual. Don't re-attempt rich+pt unless the scope shrinks to transcript-first REPL (which would also flip back §5 to inline-log-presenter).
- `[2026-05-20]` **In-tree at Worldtree/tools/ratatoskr/.** Earlier draft committed to in-tree-with-import-direction-smoke-test. Rejected at operator-routing — separate dev team forces separate repo.
- `[2026-05-20]` **New `/persona/log` SSE endpoint on Worldtree.** Considered as alternative to file-tailing `persona.log`. Rejected — contract amendment + Vor round + AFK dispatch loop is weeks of consumer-side spec work for a debug feature file-tail handles in a day. Documented follow-up trigger in `docs/design-brief.md` §5: if a Worldtree-on-server / TUI-on-laptop debug case appears, the contract cost becomes worth paying.
- `[2026-05-20]` **Cross-process Last-Event-ID resume.** Considered — would require persisting per-session Last-Event-ID to `~/.config/ratatoskr/`. Deferred to v2 if/when it turns out to matter; v1 ships "reconnect, not resume-across-process."
- `[2026-05-21]` **RichLog widget with `markup=True`.** Default impulse, but Rich interprets `[xxx]` spans as style markup and silently strips them. Every labeled stderr-style line — `[cancel_failed]`, `[done]`, `[error]`, `[busy]`, `[worker_phase]` — would render as just the content after the bracketed label, breaking the user-visible observability surface. Fix: `markup=False`. The post-Done Markdown rendering still works because `rich.markdown.Markdown` is a Renderable that ignores widget-level markup setting. Don't flip back to `markup=True` without first renaming every labeled-line format away from `[bracket]` notation.
- `[2026-05-21]` **Querying `self.query_one("#transcript", RichLog)` from inside a Textual `run_worker` coroutine.** Failed initially with `NoMatches` because the worker fires before the test's `pilot.pause()` allows the Input.Submitted handler to fully dispatch (and thus the widget tree to settle). Initial reactive fix: widen worker signature to take `log` as a parameter (passed from the handler). Volva code-review flagged this as contract drift (signature didn't match spec). Reverted to single-param signature. The real fix was test-side: add `await pilot.pause()` between `inp.action_submit()` and the polling loop in `_submit_and_wait` so the handler finishes dispatching before the worker reads the widget tree. Don't widen worker signatures to dodge test timing.
- `[2026-05-21]` **TUI session-identity rendering via `self.sub_title` + `self.hint` plain attributes.** Stored state but never rendered to a visible widget. The contract's "session-identity-always-visible" invariant was satisfied at the state-attribute level but not the user-visible-widget level. Tests asserted the attributes (which passed); Volva code-review flagged the gap. Fix: dedicated `Static(id="identity")` + `Static(id="hint")` widgets in compose; `_set_hint()` helper mirrors state → widget. Calibration evidence for the "TDD catches state, code-review catches whether the user can see it" pattern.