61c3941ec3
First manual smoke against personal Worldtree (10.250.50.152:8081) produced httpx.ReadTimeout mid-stream after the worker_phase BuildingPrompt event. Root cause: httpx's default 5s read timeout killed the connection during mimir's thinking phase (LLM streaming has multi-second idle gaps between SSE events). Fix at the caller layer (where the AsyncClient is owned): - cli._amain and tui.on_mount now construct AsyncClient with timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0). read=None disables the SSE-killing timeout; connect/write/pool keep modest timeouts so true network failures still surface promptly. Defense in depth in sse_client.stream_turn: - ERROR_ROUTING now also catches httpx.ReadTimeout (was just ReadError | RemoteProtocolError) and surfaces it as SseConnectionDropped, so if a caller misconfigures their client the failure is at least a named exception the presenters handle. Contract amendments (in-place): - Issue #1: new [compatibility] constraint documents the read=None recommendation; ERROR_ROUTING for stream_turn lists ReadTimeout alongside ReadError/RemoteProtocolError. - Issues #3 + #4: AsyncClient construction step now spells out the timeout shape explicitly. Smoke after fix: SSE stream consumed cleanly, agent responded, [done] turn_id=88 model=qwen3.6-35-a3b duration_ms=2351. Stdout-only (2>/dev/null) returned clean agent text + exit 0 — INV-002 stdout/stderr split holds end-to-end against real wire. Wire-compat envelope (personal v0.16.2 vs ratatoskr's v0.19.0 pin) confirmed. 164/164 tests GREEN; ruff clean; all three drift checks clean. Note: TUI mode not smoke-tested from this CC session (needs a TTY; operator-side check via `source env.sh && uv run ratatoskr --new --agent mimir`).
115 lines
37 KiB
Markdown
115 lines
37 KiB
Markdown
# Persistent memory — ratatoskr
|
||
|
||
This file captures durable intent and supporting evidence (goals, decisions,
|
||
foot-gun warnings, in-flight state) across context resets. Read it at session
|
||
start; treat it as one input alongside `CLAUDE.md` and the auto-memory system,
|
||
not as the single source of truth.
|
||
|
||
When durable state shifts enough to warrant capture, run `/snapshot` and
|
||
commit alongside the next commit per the persistent-memory commit-along rule
|
||
in `CLAUDE.md`.
|
||
|
||
---
|
||
|
||
## Repo purpose
|
||
|
||
Ratatoskr is a **dev-grade debug-observability TUI** for Worldtree's
|
||
Conversation API. The product IS the observability surface; chat is the
|
||
input mechanism. Devs run Ratatoskr against a local Worldtree to watch a
|
||
turn flow through every layer of the system, side-by-side, in one terminal:
|
||
agent SSE stream, persona/Vili affect dispatch, tool calls, Bifrost
|
||
handshake state, admin lifecycle events, optional raw server log.
|
||
|
||
Named after the squirrel that runs up and down Yggdrasil carrying messages
|
||
between layers. On-the-nose Worldtree resonance (Yggdrasil = the World Tree).
|
||
|
||
Origin: althing ask from worldtree-dev (thread `01KS3R34XD3N6HMK91VXESHGW7`,
|
||
2026-05-20) for the shape of a TUI Conversation API consumer. brokkr-smithy
|
||
ran the shape pass; operator's reframe routed it as a new repo with a
|
||
separate dev team rather than an in-tree Worldtree tool.
|
||
|
||
## Current state / in-flight
|
||
|
||
**Status: `ratatoskr.sse_client` + `ratatoskr.sessions` + `ratatoskr.cli` + `ratatoskr.tui` (shell) all implemented via TDD against their issue-scoped contracts; `--send` mode validated end-to-end against personal Worldtree.** 164/164 tests GREEN (42 sse_client + 19 sessions + 58 cli + 43 tui + 1 boundary + 1 metadata); ruff clean. `--send` mode + TUI mode both wired into `ratatoskr` console-script entry point (TUI dispatched via lazy import to preserve issue #3 INV-001).
|
||
|
||
What's in the repo:
|
||
- `docs/design-brief.md` — the locked design (copy from `brokkr-smithy/docs/ratatoskr-design-brief.md`).
|
||
- `docs/SPEC-PIN.md` — Worldtree spec pin documentation + bump procedure.
|
||
- `docs/conversation-api-spec.md` — vendored Worldtree spec at the pinned SHA.
|
||
- `docs/conversation_api.contract.md` — vendored Worldtree server-side contract at the pinned SHA.
|
||
- `docs/contracts/issues/1.contract.md` — **issue-scoped contract for issue #1** (https://gitea.phasefinal.com/vh/ratatoskr/issues/1). v2.1, complexity=high. `target_module: ratatoskr.sse_client`. `prd:` block pins to issue body SHA `abcbc49467e86f1d` at `2026-05-21T03:57:37+00:00`. Four FN blocks: `stream_turn`, `reconnect_turn`, `cancel_turn`, `_parse_sse_id`. Drift check (`scripts/contract_drift_check.py`) returns clean.
|
||
- `pyproject.toml` — Python 3.12, hatchling, uv-managed. Deps: httpx, httpx-sse, textual. Dev deps: pytest, pytest-asyncio, respx, ruff, mypy, textual-dev, pyyaml (consumed by `docs/contracts/contract_parser.py` + `scripts/contract_drift_check.py`).
|
||
- `src/ratatoskr/__init__.py` + `cli.py` — stubs.
|
||
- `src/ratatoskr/sse_client.py` — **implemented 2026-05-21** per `docs/contracts/issues/1.contract.md`. Four public entry points + nine typed Event variants + ten custom exceptions. Shared SSE-iteration logic (INV-002 + INV-003 + terminal-break) lives in private `_iter_events(event_source, *, expected_turn_id)` helper consumed by both `stream_turn` and `reconnect_turn` — `expected_turn_id=None` triggers "establish from first event" semantics, `expected_turn_id=N` triggers "first event is already a flip-candidate" semantics (the two-entry-point distinction Volva surfaced).
|
||
- `tests/test_sse_client.py` — 37 tests covering all four FN blocks' TESTS: entries verbatim (13 + 10 + 8 + 6). Real HTTP wire via respx mocks; SSE wire format constructed by helper `_sse_chunk`. Connection-drop test uses custom `httpx.AsyncByteStream` subclass that yields chunks then raises `RemoteProtocolError`.
|
||
- `src/ratatoskr/sessions.py` — **implemented 2026-05-21** per `docs/contracts/issues/2.contract.md`. Two functions (`create_session`, `list_sessions`) + two frozen dataclasses (`SessionInfo`, `SessionPage`) + three exception types (`AgentNotFound`, `InvalidCursor`, `SessionApiFailed`). `SessionInfo` uses origin-conditional defaults per INV-001/INV-002 (create-origin: `name=None`, `archived=False`, `tags=[]`, `message_count=<from body>`; list-origin: same defaults for absent/null fields, `message_count=None`). `SessionApiFailed` truncates `.body` to ≤1024 at construction. No code shared with `sse_client.py` (convention-dependency only per issue #2 `dependencies:`).
|
||
- `tests/test_sessions.py` — 19 tests covering both FN blocks' TESTS: entries verbatim (7 + 12). Helper `_list_item()` builds GET /sessions list-item bodies for tests.
|
||
- `docs/contracts/issues/3.contract.md` — **issue-scoped contract for issue #3** (https://gitea.phasefinal.com/vh/ratatoskr/issues/3). v2.1, complexity=medium. `target_module: ratatoskr.cli`. `prd:` block pins to issue body SHA `206ef51709d43b2c` at `2026-05-21T05:13:29+00:00`. Six FN blocks: `main`, `_parse_args`, `_amain`, `_render_event`, `_run_turn`, `_cancel_and_log`. `dependencies:` block lists issues #1 and #2 as code-level deps (first in the repo to do so — #1 and #2 were convention-only siblings). Drift check returns clean.
|
||
- `src/ratatoskr/cli.py` — **implemented 2026-05-21** per `docs/contracts/issues/3.contract.md`. Six entry points: sync `main` + async `_amain` + four private helpers (`_parse_args`, `_render_event`, `_cancel_and_log`, `_run_turn`). Composes `sessions.create_session` (when `--new`) with `sse_client.stream_turn` + `cancel_turn`. Hard invariant: no `textual` / `rich` imports (raw stdout). `_run_turn` is the load-bearing piece — race-loop pattern that gates `asyncio.create_task(sigint_event.wait())` behind `if not cancelling` to avoid the busy-loop bug Volva flagged in contract review. SIGINT handler installed via `loop.add_signal_handler(SIGINT, sigint_event.set)` so unit tests can fire the event directly without real signals. 305 LOC.
|
||
- `tests/test_cli.py` — 58 tests covering all six FN blocks' TESTS: entries (14 + 10 + 5 + 13 + 7 + 5) plus issue #4 amendments (no_send_marks_tui_mode, raw_flag_default_false, raw_flag_set, no_send_dispatches_to_tui). Helpers: `_sse_chunk` (same shape as `test_sse_client.py`'s helper), `_sse_resp` (wraps respx Response with the `text/event-stream` content-type), `_GatedStream` (custom `httpx.AsyncByteStream` that pauses on `asyncio.Event` entries to make SIGINT-mid-stream tests deterministic without sleep-based timing). Test fixtures: `_clear_env` autouse fixture clears `WORLDTREE_API_KEY` / `WORLDTREE_API_URL` per test for deterministic env-resolution assertions.
|
||
- `docs/contracts/issues/4.contract.md` — **issue-scoped contract for issue #4** (https://gitea.phasefinal.com/vh/ratatoskr/issues/4). v2.1, complexity=medium. `target_module: ratatoskr.tui`. `prd:` block pins to issue body SHA `b1e73e7d2e3dd453` at `2026-05-21T06:21:37+00:00`. Six FN blocks: `run_tui`, `RatatoskrApp` class shape, `on_mount`, `on_input_submitted`, `_stream_turn_worker`, `_render_event_to_log`, `action_interrupt`, `action_quit`, `on_unmount`, `_cancel_via_sse`. Plus a "CLI amendments" section spelling out the issue #3 in-place amendments (`--send` becomes optional + `--raw` flag + `main` dispatch + new tests). `dependencies:` lists issues #1, #2, #3 as code-level deps.
|
||
- `src/ratatoskr/tui.py` — **implemented 2026-05-21** per `docs/contracts/issues/4.contract.md`. One `RatatoskrApp(App[int])` Textual subclass + `run_tui(args)` sync entry. Compose layout: Header + RichLog (markup=False so labeled `[xxx]` lines render verbatim; the post-Done Markdown rendering uses `rich.markdown.Markdown` directly) + Input + Footer. Two-stage Ctrl-C state machine via `action_interrupt` (idle→exit(0); streaming→cancel_turn POST + flip to cancelling; cancelling→exit(3) abandon-drain). Ctrl-D → `action_quit` immediate exit(0). Worker pattern via `self.run_worker(self._stream_turn_worker(content, log))` — the log reference is passed as a parameter because the worker can't `self.query_one(...)` reliably from within Textual's worker context. ~210 LOC.
|
||
- `tests/test_tui.py` — 43 tests covering all six FN blocks' TESTS entries (9 + 4 + 8 + 5 + 8 + 7 + 2). Helpers: `_args_new` / `_args_existing` (ParsedArgs factories), `_spy_writes` (monkeypatch RichLog.write to record calls — works around RichLog's write→render asynchrony where `.lines` isn't populated immediately after .write() returns), `_sse_chunk` / `_sse_resp` (SSE wire helpers), `_submit_and_wait` (drive input + poll for state==idle), `_noop_worker` (fake stream worker for on_input_submitted tests). Tests use Textual's `App.run_test()` + `Pilot` for headless app testing. SIGINT-mid-stream tests use `httpx.AsyncByteStream` subclasses with `asyncio.Event` gates to make timing deterministic.
|
||
- `tests/test_no_worldtree_imports.py` — boundary smoke test (passes; verified 2026-05-20).
|
||
- `tests/snapshots/README.md` — recording/replay convention for SSE snapshot tests.
|
||
|
||
What's NOT in the repo yet:
|
||
- Gitea remote — operator provided `git@gitea.phasefinal.com:vh/ratatoskr.git` on 2026-05-20; about to be added + first push at the same commit as this update.
|
||
- CLAUDE.md customization — currently using the canonical template's
|
||
generic CLAUDE.md. The dev team may want to add Ratatoskr-specific
|
||
conventions on first substantive work.
|
||
|
||
**Branch:** `main`. Remote: `origin → git@gitea.phasefinal.com:vh/ratatoskr.git` (added 2026-05-20).
|
||
|
||
**Next natural moves:**
|
||
1. **TUI smoke (operator-side)** — `source env.sh && uv run ratatoskr --new --agent mimir` from an interactive terminal. Validates the Textual app lifecycle + the post-Done markdown re-render end-to-end. Needs a TTY which this CC session doesn't have.
|
||
2. **Side-pane issues** — design-brief §5 lists 5 side panes (Persona, Tools, AdminEvents, BifrostState, ServerLog). Each gets its own issue + contract + TDD pass on top of the shell. Persona is the natural first (file-tail of `persona.log` — cheap; no new Worldtree wire).
|
||
3. Record real SSE snapshot fixtures from a running Worldtree. `--send --new` redirected to a fixture file IS the recording probe — capture outputs to `tests/snapshots/` for replay-based regression coverage. The smoke just proved the recording probe works.
|
||
4. Startup session picker — design-brief §4 `DataTable` of `GET /sessions`. Modest scope; pairs naturally with the TUI shell.
|
||
|
||
## Recent decisions
|
||
|
||
Chronological log of decisions with `[YYYY-MM-DD]` prefix. One line per
|
||
decision. Captures rationale that won't be obvious from code alone.
|
||
|
||
- `[2026-05-20]` Project name **Ratatoskr** (squirrel on Yggdrasil — runs up and down carrying messages). Earlier candidate Andvari demoted on the cursed-ring association.
|
||
- `[2026-05-20]` **Separate repo, separate dev team.** Operator's call; the in-tree-at-Worldtree/tools/ alternative was considered and rejected to dogfood the API boundary.
|
||
- `[2026-05-20]` **No Worldtree-source imports.** Spec-only dependency. Triple version-skew mitigation: spec-pin in pyproject.toml + recorded-SSE snapshot tests + conformance smoke. Initial pin: `55101e909abcd2219833266b6f905c5bc956e0f0` (Worldtree v0.19.0). See `docs/SPEC-PIN.md`.
|
||
- `[2026-05-20]` **Textual** (not rich+prompt_toolkit). Driver: debug observability is the primary purpose, and a multi-pane dashboard with persistent side panes + independent scrollback is structurally application-shell-shaped. Volva consulted via cross-frontier second-opinion and converged on the same call.
|
||
- `[2026-05-20]` **`httpx-sse`** for SSE consumption. The server emits composite `{turn_id}:{seq}` `id:` lines (Worldtree INV-014) load-bearing for SSE-resume; hand-rolled `data:`-only parsing (the skaldsong pattern) silently drops these. Ratatoskr becomes the reference Python SSE-resume implementation.
|
||
- `[2026-05-20]` **Persona-pane PII posture: label-don't-refuse.** `persona.log` is process-wide; pane title flips between `[Persona — PROCESS-WIDE]` and `[Persona — session <id>…]` based on whether log lines carry session_id. Refuse-against-non-local was considered and rejected as paternalistic.
|
||
- `[2026-05-20]` **Server-stdout pane: opt-in via `--server-log <path>`.** No auto-detection of well-known paths.
|
||
- `[2026-05-20]` **Two-stage Ctrl-C.** First cancels in-flight turn server-side; second exits app. Ctrl-D bound to immediate exit.
|
||
- `[2026-05-20]` **Single-session-per-launch + startup picker.** No in-app `/switch`. CLI flags `--session <id>` and `--new` for scripted use. Session identity always visible in Textual footer.
|
||
- `[2026-05-20]` **Markdown rendering default-on; `--raw` opt-out.** Don't pre-design `--no-stream-formatting` (Volva: add only if streaming-markdown rendering is empirically ugly).
|
||
- `[2026-05-20]` **Non-interactive `--send` mode.** Single SSE consumer module, two presenters (TUI + stdout). Keeps Ratatoskr honest as an API consumer; useful for CI / scripted probes.
|
||
- `[2026-05-20]` **First contract: `ratatoskr.sse_client`.** Bundles `stream_turn` + `reconnect_turn` + `cancel_turn` + private `_parse_sse_id` into one module — the SSE-resume flow is coupled (cancel needs `turn_id` from the SSE wire `id:`, reconnect re-uses the same parsed `SseId`), so they share a contract. Hard invariant INV-002 makes the composite `{turn_id}:{seq}` `id:` parsing load-bearing — closes the foot-gun the design-brief §3 names (hand-rolled `data:`-only parsing silently drops the `id:`). v2.1 test categories `adversarial`/`scenario`/`trace` used freely; parser warns but format spec §2.1.E permits them.
|
||
- `[2026-05-21]` **Contract converted to issue-scoped (issue #1).** Moved `docs/contracts/sse_client.contract.md` → `docs/contracts/issues/1.contract.md`. Frontmatter shape switched from module-scoped (`module:`/`purpose:`) to issue-scoped (`target_module:`/`scope:`/`prd:`) per CONTRACT-FORMAT §2.1.I. `prd:` block pins to issue #1's body hash (`abcbc49467e86f1d`). `scripts/contract_drift_check.py` returns clean. **Known parser stale-ness**: `contract_parser.py --validate` ERRORs on issue-scoped frontmatter (missing `module:`/`purpose:`) — this is CONTRACT-FORMAT §2.1.L H10, a documented Brokkr-side follow-up. Parser is a canonical sync, so we do NOT patch it locally (would drift from canonical). Treat parser ERROR-on-issue-scoped as expected until the canonical bumps.
|
||
- `[2026-05-21]` **Default issue-tracker labels seeded** (17 total). Sleipnir gating (`ready-for-agent`, `blocked-needs-contract`, `blocked-needs-dependency`), triage (`needs-triage`, `needs-architect-decision`, `needs-info`), type (`bug`, `enhancement`, `task`, `documentation`), resolution (`duplicate`, `wontfix`, `invalid`), Ratatoskr-specific area (`sse-client`, `tui`, `cli`, `observability`).
|
||
- `[2026-05-21]` **`ratatoskr.sse_client` implemented via TDD against issue #1's contract.** 37 contract-listed tests authored + GREEN per the tracer-bullet vertical-slice ordering (`_parse_sse_id` → `stream_turn` → `reconnect_turn` → `cancel_turn`). Refactor pass extracted `_iter_events` helper to dedupe INV-002 + INV-003 + terminal-break logic across `stream_turn` and `reconnect_turn`; `expected_turn_id=None` vs `expected_turn_id=N` distinguishes the two entry-point semantics Volva surfaced. Notable choices made during implementation: (a) regex `^-?\d+$` pre-check in `_parse_sse_id` to reject whitespace before `int()` (Python's `int(" 3 ")` would silently strip — this kept the strict-no-whitespace test honest); (b) `_DropAfter` AsyncByteStream subclass in tests to simulate mid-stream `RemoteProtocolError`; (c) ToolResult.result and ToolStart.arguments typed as `Any` (server JSON varies); (d) ruff line-length=100 (per pyproject) forced some test docstrings to be tighter than v0 draft.
|
||
- `[2026-05-21]` **Issue #2 + contract: `ratatoskr.sessions`.** Scope is narrow — `create_session` (POST /sessions) + `list_sessions` (GET /sessions, cursor-paginated) + shared `SessionInfo` and `SessionPage` frozen dataclasses. Bundles two endpoints in one contract because they share the response envelope shape; splitting would duplicate the dataclass. Bifrost binding (Worldtree issue #160), ephemeral sessions, `GET /sessions/{id}`, `PATCH`, `DELETE`, and `GET /sessions/{id}/messages` (history) are explicitly out of scope (codified in the contract's `## Out of scope` H2 — first contract in this repo to carry that section, so future Volva consults resolve cleanly via the default path instead of needing `--out-of-scope` overrides). `prd:` pinned to issue #2 body SHA `01fbbd52b6d90eb0` at `2026-05-21T04:45:06+00:00`; drift check clean. `dependencies:` lists issue #1 as a convention-dependency (no code import; same API-consumption posture).
|
||
- `[2026-05-21]` **`ratatoskr.sessions` implemented via TDD against issue #2's contract.** 19 contract-listed tests authored + GREEN per tracer-bullet vertical-slice (`create_session` first, then `list_sessions`). One internal-inconsistency in the contract spotted at TDD start — POST-003 and `happy_create` test description still said "archived is None, tags is None" while the freshly-amended INV-001 set them to `False` and `[]`; fixed the contract in-place before writing tests so the spec stayed coherent. Implementation is small (~115 LOC for src module); no refactor pass deemed worthwhile (the two functions are ~25 LOC each with distinct error-routing branches).
|
||
- `[2026-05-21]` **Volva code-vs-contract review on `ratatoskr.sessions`.** Three findings, all "fix it" (one with collateral contract amendment). (1) Drift: `archived=item.get("archived", False)` returned `None` for explicit-null because `.get(key, default)` only fires on absent keys, not on null. Fixed to `item.get("archived") or False` (handles absent, null, False, True). INV-002 wording was the source — also amended to spell out the `.get(default)` foot-gun explicitly. (2) Test-gap: no test exercised explicit-null `archived`/`tags`. Added `test_explicit_null_list_defaults` using a raw item dict (the `_list_item()` helper masked the issue with its own defaulting). (3) Precision: `message_count=body.get("message_count")` could silently default to None while POST-003 required it non-None. Aligned: code now uses `body["message_count"]` (matches sibling fields like session_id which use bracket access); INV-001 + STEP 5 updated to spell out strict semantics. Volva's meta-note: "modest weight" — TDD caught the main surface, this caught a narrow Python `.get()` semantics edge that no human reading would have noticed without explicit-null priors. 63 tests GREEN post-fix.
|
||
- `[2026-05-21]` **Volva paraphrase round on `docs/contracts/issues/2.contract.md`.** Volva flagged 5 ambiguities; operator approved amendments to 3 of them. (1) `tags`/`archived`/`name` defaulting semantics now explicit: `tags: list[str]` (default `[]`), `archived: bool` (default `False`), `name: str | None` (default `None`); INV-001/INV-002 + STEPS aligned. (2) `include_archived_query` test tightened: default → URL has NO `include_archived` param at all (was "no param OR explicit false" — softened the assertion against STEP 2's prescriptive behavior). (5) `metadata` populated-vs-defaulted slippage resolved: INV-001 + INV-002 now spell out the defensive `body.get("metadata", {})` default for spec drift tolerance. Volva flags #3 (exception `.body` sensitivity) and #4 (`assert` for runtime validation) reviewed and kept as-is — both intentional and consistent with issue #1's precedent. Drift check still clean (amendments don't touch the pinned issue body).
|
||
- `[2026-05-21]` **Volva code-vs-contract review round on `ratatoskr.sse_client`.** Volva flagged 4 findings (3 drifts + 1 test-gap), all code-side "fix it" recommendations: (1) `_iter_events` fell off cleanly on EOF before terminal, violating INV-001 ("MUST NOT raise StopAsyncIteration before a terminal event arrives unless connection drops"); fix tracks `terminal_seen` flag and raises `SseConnectionDropped` on clean-EOF-without-terminal. (2) Both `SseConnectFailed.body` and `CancelFailed.body` stored full response bytes; ERROR_ROUTING specified truncation to `[:1024]`; fix truncates in `__init__` before storing. (3) `_parse_sse_id` PRE-001 specified `assert isinstance(raw, str)`, but code called `.split(":")` directly (incidental `AttributeError` on non-str); fix adds the assert. (4) Test-gap on cancel_turn's "other status → CancelFailed" branch; fix adds a 503 test with >1024-byte body that double-covers finding #2. Meta-note: Volva said TDD caught the main happy/adversarial shape; the misses were "negative space" cases (clean EOF, exception payload truncation, untested generic cancel branch) — calibration evidence that cross-model review pulls weight on the same-model author's blind spots. 43 tests GREEN post-fix (42 sse_client + 1 boundary), ruff clean.
|
||
- `[2026-05-21]` **Volva paraphrase round on `docs/contracts/issues/1.contract.md`.** Volva flagged 5 ambiguities; operator approved amendments to 3 of them. (1) `reconnect_turn` STEP 2 punt resolved: signature now carries `content: str`; STEP 2 body is `json={"content": content}` matching spec §Reconnect flow example verbatim. Spec line 732 makes the agent's tools+LLM run "exactly once regardless of disconnects/reconnects" — the `content` is a wire-schema requirement, not re-processed server-side. (2) `_parse_sse_id` tightened: `turn_id ≥ 1` AND `seq ≥ 1` (was `≥ 0`); spec §SSE id format line 705 explicitly states `seq` starts at 1, and `turn_id` is SQLite autoincrement (≥1). Test `happy_zero_seq` flipped to `zero_seq [adversarial]`; new `zero_turn_id` + `negative_seq` adversarial tests added. (3) INV-003 clarified to spell out the two-entry-point semantics: `stream_turn` establishes `turn_id` from the first event (first event always yields); `reconnect_turn` parses the expected `turn_id` FROM `last_event_id` BEFORE the connection opens, so the first server event is already a flip-candidate and is NOT yielded on mismatch. Volva flags #3 (MalformedSseId-vs-ValueError split) and #5 (exactly-one-terminal as server-assumed) noted but kept as-is — deliberate distinctions. Drift check still clean against issue #1 (amending the contract doesn't touch the pinned issue body).
|
||
- `[2026-05-21]` **Issue #3 + contract: `ratatoskr.cli --send` non-interactive stdout presenter.** Composes `create_session` (when `--new`) with `stream_turn` + `cancel_turn` into a one-shot CLI. Hard invariants: no `textual` / `rich` imports (raw stdout — `--raw` is TUI-only per design-brief §6); stdout for `Text` deltas + the post-`Done` newline ONLY; everything else labeled to stderr. Six FN blocks (`main`, `_parse_args`, `_amain`, `_render_event`, `_run_turn`, `_cancel_and_log`). The `_run_turn` race-loop is the load-bearing piece: races `__anext__` against `sigint_event.wait()` so a mid-stream SIGINT lands within one event boundary; once cancel is in flight, the race-loop stops creating new `wait()` tasks (the no-busy-loop fix Volva flagged). 9-bucket exit-code table (0/2/3 terminal; 10/11/12 usage/auth/agent; 20/21/22 server/network/protocol). `prd:` pinned to issue #3 body SHA `206ef51709d43b2c` at `2026-05-21T05:13:29+00:00`; first contract in the repo with a code-level `dependencies:` block (issues #1 and #2).
|
||
- `[2026-05-21]` **Volva paraphrase round on `docs/contracts/issues/3.contract.md`.** Volva flagged 5 ambiguities; operator approved amendments to ALL 5 (higher hit rate than #1/#2's 3-of-5 — async + signal-handling has more places for ambiguity to hide). (1) INV-001 wording tightened: "no in-repo modules other than `ratatoskr.sessions` and `ratatoskr.sse_client`" (was "imports from X and Y only" which literally forbade httpx/asyncio/stdlib). (2) INV-002 + `_render_event` POST-002/003 restructured: `Done`'s stdout newline is part of the contract (two stdout cases: Text deltas + post-Done newline), not a contradiction with "no other event writes stdout". (3) `[cancelled] (before any event arrived)` early-exit label added to the Data flow stderr list. (4) SIGINT race-loop pseudocode gated `sigint_task = asyncio.create_task(...)` behind `if not cancelling` — without this gate, once sigint is set, every loop iteration would wake on the already-set event (busy loop). New `no_busy_loop_after_cancel [trace]` test added. (5) `CancelTurnNotFound` + `CancelAlreadyCompleted` added to assumptions import list (they were used in `_cancel_and_log`'s ERROR_ROUTING but missing from the public-surface declaration). Meta-note: Volva said "discipline pulls weight here" — same calibration signal as #1/#2.
|
||
- `[2026-05-21]` **`ratatoskr.cli` implemented via TDD against issue #3's contract.** 54 contract-listed tests authored + GREEN per the vertical-slice ordering (`_parse_args` → `_render_event` → `_cancel_and_log` → `_run_turn` → `_amain` → `main`). One in-flight contract amendment during TDD: the `no_busy_loop_after_cancel` test description originally said "exactly ONE wait()-shaped task created" but the natural race-loop shape produces 2 (iter 1 raced with text-event, iter 2 raced with sigint → flipped cancelling=True; iter 3+ skipped). Amended the contract test description to assert "TWO total wait() coroutines" with rationale; the busy-loop check is preserved (iter 3+ MUST skip wait() creation; the bug would grow N unbounded). Implementation choices: (a) `_UsageErrorParser` subclass overrides `argparse.ArgumentParser.error` to raise `_ArgparseError` instead of SystemExit, then `_parse_args` catches and re-raises as `UsageError` per the contract's ERROR_ROUTING; (b) `_GatedStream` test helper (custom `httpx.AsyncByteStream` that pauses on `asyncio.Event` entries) made SIGINT-mid-stream tests deterministic without sleep-based timing — gates release via side-channels (the cancel-mock setting an event when observed); (c) strong-ref `cancel_task` variable in `_run_turn` holds the fire-and-forget cancel task to suppress RUF006 / asyncio GC warning. 117/117 tests GREEN post-implementation; ruff clean.
|
||
- `[2026-05-21]` **Volva code-vs-contract review on `ratatoskr.cli`.** Five findings, all "fix it" (one with collateral contract amendment). (1) Drift: neither `main` nor `_parse_args` asserted PRE-001 (`argv is None or all(isinstance(a, str) for a in argv)`). Fixed: added assertions at both entry points. (2) Drift: argparse's `--help` raises `SystemExit(0)` which escaped through `main` — unfriendly UX. Code: `main` now catches `SystemExit` and returns `exc.code` verbatim (passthrough; argparse already printed help to stdout). Contract amended in-place: `_parse_args` + `main` ERROR_ROUTING now spell out the SystemExit-from-argparse-clean-exits passthrough; new `help_exits_cleanly [happy]` test added. (3) Precision: `_render_event` lacked the union-type assertion from PRE-001 — unmatched event variants would silently no-op. Fixed: added `assert isinstance(event, (WorkerPhase, Thinking, Text, ...))` at function entry. (4) Drift: `cancel_task` was created but never awaited in `_run_turn`'s `finally` — under fast-stream + slow-cancel scenarios, the `[cancel_failed]` log could miss being written before `_run_turn` returns, AND `_amain` could close the AsyncClient while the cancel POST was still in flight. Fixed: `finally` block now awaits `cancel_task` if present (`_cancel_and_log` already swallows all errors per INV-009, so the await never raises). (5) Test-gap: `text_to_stdout_only` and `done_writes_newline_and_label` used plain `io.StringIO` and didn't verify INV-010's per-chunk flush — the tests would pass even with flush removed. Fixed: new `_FlushCountingIO` subclass counts `flush()` calls; both tests assert `flush_count == 1`. Volva's meta-note: "TDD pass mostly caught central behavior; this review caught contract-hardening edges (PRE asserts, --help, non-guaranteed cancel-failure log)." Same calibration shape as #1 (4 negative-space drifts) and #2 (3 drifts) — the post-TDD code-review consistently catches the assert-boundary and observability-shape gaps the test-author's hypotheses don't cover. 118 tests GREEN post-fix.
|
||
- `[2026-05-21]` **Issue #4 + contract: `ratatoskr.tui` shell.** Textual `App[int]` subclass with single chat-pane layout (Header + RichLog + Input + Footer). Composes existing modules: `create_session` on `--new` + `stream_turn` per turn + `cancel_turn` on Ctrl-C. Hard invariants: INV-002 session-identity-always-visible footer (with `<unknown>` carve-out when `--session` is used without --agent); INV-003 two-stage Ctrl-C state machine (idle/streaming/cancelling per design-brief §8c); INV-005 markdown rendering default-on with `--raw` opt-out (deliberately produces a streaming-then-Markdown-render double-display — accepted v1 trade-off; Static-then-commit refactor deferred); INV-008 mid-session errors render to transcript and return to idle (don't exit). Shell-only scope — design-brief §5's five side panes (Persona, Tools, AdminEvents, BifrostState, ServerLog), startup session picker, Tab bindings, history rendering all deferred to follow-up issues. Concurrent in-place amendment of issue #3: `--send` becomes optional (when omitted, `main` lazy-imports `ratatoskr.tui.run_tui`); `--raw` flag added to `ParsedArgs`; `_parse_args`/`main` TESTS sections updated. The lazy-import preserves issue #3 INV-001 (no textual at cli module scope). `prd:` pinned to issue #4 body SHA `b1e73e7d2e3dd453` at `2026-05-21T06:21:37+00:00`; drift check clean.
|
||
- `[2026-05-21]` **Volva paraphrase round on `docs/contracts/issues/4.contract.md`.** 5 findings, all amended (matching #3's hit rate). (1) INV-002 footer carve-out for `<unknown>` agent slot when `--session` and agent_id unknown; tightened wording so the `<unknown>` placeholder is an accepted satisfaction of "session-identity-always-visible". (2) "Ctrl-C twice to exit" idle hint vs single-press-exits state machine: kept the hint verbatim per design-brief §8c's conservative-by-design rationale, amended INV-003 to spell out the intentional discrepancy + cite the brief so an implementer can't "fix" it by accident. (3) Markdown double-render trade-off: INV-005 strengthened from "trade-off" to explicit "assistant response visibly appears TWICE in the transcript by design"; Static-then-commit refactor explicitly out-of-scope for this shell. (4) Silent input-discard during streaming: now writes `[busy] turn in flight; input ignored` to the transcript (visible notice, not silent swallow); new test for cancelling state too. (5) `{!r:.200}` format spec note: kept (valid Python repr-then-truncate-200), added inline comment explaining the syntax. Volva meta-note: "discipline pulls weight here" — same calibration signal as prior rounds.
|
||
- `[2026-05-21]` **`ratatoskr.tui` shell implemented via TDD against issue #4's contract.** 43 contract-listed tests + 4 issue-#3-amendment tests authored + GREEN per the vertical-slice ordering (`_render_event_to_log` → `_cancel_via_sse` → CLI amendments → app class + on_mount + on_unmount → on_input_submitted → _stream_turn_worker → action_interrupt + action_quit → run_tui). Two in-flight contract amendments during TDD: (a) PRE-002 of `run_tui` was `(args.session_id is None) != args.new` — backwards, fails when session_id is set + new=False; corrected to `bool(args.session_id) != bool(args.new)`; (b) RichLog created with `markup=False` (was `markup=True` in contract) because Rich interprets `[xxx]` as style spans and strips them — would break every labeled stderr-style line. The post-Done Markdown rendering still works because `rich.markdown.Markdown` is a Renderable that doesn't need widget-level markup. Implementation choices: (a) worker takes the `log` widget as a parameter (passed from on_input_submitted) — querying `#transcript` from inside a Textual worker context fails with NoMatches; (b) `_spy_writes(monkeypatch)` test helper records every RichLog.write call because RichLog's `.lines` Strip buffer isn't populated synchronously after .write() returns, making post-app-shutdown inspection unreliable; (c) SIGINT-mid-stream tests use custom `httpx.AsyncByteStream` subclasses with `asyncio.Event` gates to make timing deterministic without sleep-based polling. 164/164 tests GREEN; ruff clean (one `# noqa: RUF001,RUF003` for the intentional `❯` INV-006 prompt prefix).
|
||
- `[2026-05-21]` **First manual smoke against personal Worldtree — successful + uncovered a real defect.** `source env.sh && uv run ratatoskr --send "hello mimir..." --new --agent mimir` against `http://10.250.50.152:8081`. First run produced `httpx.ReadTimeout` mid-stream after the `worker_phase BuildingPrompt` event — httpx's default 5s read timeout killed the connection during mimir's thinking phase (LLM streaming has multi-second idle gaps that exceed the default). Fix: caller-owned `httpx.AsyncClient` now constructed with `timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0)` in both cli (`_amain`) and tui (`on_mount`); `read=None` disables the SSE-killing timeout while keeping modest connect/write/pool timeouts. Defense in depth: `sse_client.stream_turn` ERROR_ROUTING also catches `httpx.ReadTimeout` → `SseConnectionDropped` (was just `ReadError | RemoteProtocolError`). Three contracts amended in-place: issue #1 adds [compatibility] constraint about caller timeout config + adds ReadTimeout to ERROR_ROUTING; issues #3/#4 spell out the explicit timeout shape in their AsyncClient construction step. Re-smoke after fix: SSE stream consumed cleanly, agent text appeared, `[done] turn_id=88 model=qwen3.6-35-a3b duration_ms=2351 ... cached_input_tokens=0`. Stdout-only smoke (`2>/dev/null`) returned clean text + exit 0 — INV-002 (stdout/stderr split) holds end-to-end against real wire. Wire-compat envelope from infra-ops/worldtree-dev's analysis (personal v0.16.2 vs ratatoskr's v0.19.0 pin) confirmed by smoke — no malformed_sse_id, no turn_id_flip, no spec drift. The personal Worldtree's mimir runs qwen3.6-35-a3b. **TUI mode not smoke-tested from this session** — requires TTY; operator should smoke via `source env.sh && uv run ratatoskr --new --agent mimir`.
|
||
- `[2026-05-21]` **Volva code-vs-contract review on `ratatoskr.tui` shell.** 8 findings — 5 drifts, 2 precision (contract-amend), 1 test-gap (follows from F1). All addressed. (1) DRIFT, primary: `self.sub_title` (Header) and `self.hint` (plain attr) were set but never rendered to a visible Footer widget — INV-002 + INV-003 require visible Footer-area rendering. Fixed: added `Static(id="identity")` + `Static(id="hint")` widgets in compose; `_set_hint()` helper mirrors state→widget; on_mount calls `query_one("#identity", Static).update(...)`. Same-model TDD missed this because tests asserted `app.sub_title` / `app.hint` internal state, not visible widget content. Volva caught immediately. (2) DRIFT: `_stream_turn_worker` signature widened to `(content, log)` during TDD as a workaround for a NoMatches-during-worker-execution test failure. Reverted to single-param `(content)` per contract; queries `#transcript` internally. The original test failure was actually a test-timing issue resolved by adding `await pilot.pause()` before the polling loop in `_submit_and_wait`. (3) DRIFT: `run_worker(coro)` missing `exclusive=True`. Restored. (4) DRIFT: `run_tui` PRE-001 missing `isinstance(args, ParsedArgs)` check; only had `args.send_content is None`. Added. Required hoisting `from ratatoskr.cli import ParsedArgs` out of `TYPE_CHECKING` (runtime import — fine, no circular dependency: cli imports tui lazily inside main; tui imports cli unconditionally at module load). (5) DRIFT: `_render_event_to_log` missing PRE-001 union assertion. Added. (6) PRECISION (amend contract): RichLog `markup=True, highlight=True` in contract; implementation chose `markup=False, highlight=False` to preserve labeled-line semantics. Contract amended with rationale comment in COMPOSE shape. (7) PRECISION (amend contract): `on_mount` POST-003 said `agent_id is non-empty`; for `--session` attach without `--agent` it's None per INV-002 carve-out. Contract amended. (8) TEST-GAP: tests checked `app.sub_title` / `app.hint`; updated to query `Static(#identity).render()` / `Static(#hint).render()` (the actual visible widgets). Volva meta-note: "TDD pass caught most stream/session/error mechanics, but tested internal state where the contract required visible Footer behavior, so same-model TDD would plausibly miss the primary drift." Calibration evidence: cross-model review pulls weight on UI-observability gaps in particular, where the test-author can convince themselves the state is correct without verifying the user can see it. 164 tests GREEN post-fix.
|
||
|
||
## Tried and abandoned
|
||
|
||
Log of approaches that were tried and rejected, with rationale. Future-self
|
||
defense against re-attempting the same cul-de-sac.
|
||
|
||
- `[2026-05-20]` **rich + prompt_toolkit framework choice.** Considered first (during initial shape draft). Volva flagged that §1 and §5 pulled in opposite directions: a real side-panel observability surface would silently become a widget framework reimplementation. Operator's debug-observability reframe sealed the flip to Textual. Don't re-attempt rich+pt unless the scope shrinks to transcript-first REPL (which would also flip back §5 to inline-log-presenter).
|
||
- `[2026-05-20]` **In-tree at Worldtree/tools/ratatoskr/.** Earlier draft committed to in-tree-with-import-direction-smoke-test. Rejected at operator-routing — separate dev team forces separate repo.
|
||
- `[2026-05-20]` **New `/persona/log` SSE endpoint on Worldtree.** Considered as alternative to file-tailing `persona.log`. Rejected — contract amendment + Vor round + AFK dispatch loop is weeks of consumer-side spec work for a debug feature file-tail handles in a day. Documented follow-up trigger in `docs/design-brief.md` §5: if a Worldtree-on-server / TUI-on-laptop debug case appears, the contract cost becomes worth paying.
|
||
- `[2026-05-20]` **Cross-process Last-Event-ID resume.** Considered — would require persisting per-session Last-Event-ID to `~/.config/ratatoskr/`. Deferred to v2 if/when it turns out to matter; v1 ships "reconnect, not resume-across-process."
|