# Ratatoskr — Design Brief for a Worldtree Conversation API Debug TUI

**Status:** Draft for the Ratatoskr dev team. Shape recommendation — not an implementation, not a contract. Operator (Vuong) holds the calls flagged `[OP]`; once those lock, this brief hands off to the team.

**Origin:** Conversation with worldtree-dev (althing thread `01KS3R34XD3N6HMK91VXESHGW7`, 2026-05-20) identified the need for a dev-grade TUI client that exercises the Worldtree Conversation API end-to-end. Operator's reframe during draft routing: Ratatoskr ships as a **new repo with a separate dev team** — not in-tree to Worldtree, not built by worldtree-dev. This brief is the design-team handoff artifact (paralleling `docs/storyengine-ui-design-brief.md`).

**Name:** Ratatoskr — the squirrel that runs up and down Yggdrasil carrying messages between the eagle at the top and Níðhöggr at the bottom. The TUI literally runs up and down Worldtree's API surface — sessions, turns, persona, tools, admin events, Bifrost state — carrying messages between layers. On-the-nose Worldtree resonance, no curse-the-ring baggage (Andvari was the earlier candidate; demoted on the cursed-ring association).

**Refs consulted:**
- `Worldtree/docs/conversation-api-spec.md` (v1.0, 2026-04-15)
- `Worldtree/docs/contracts/conversation_api.contract.md` (INV-001..INV-052, plus amendments)
- `Worldtree/core/transports/terminal.py` (1497 lines — existing in-tree non-API chat; rich.live + readline)
- `Worldtree/core/transports/sessions_cli.py` (210 lines — direct-store admin CLI)
- `skaldsong/src/skaldsong/llm/worldtree.py` (576 lines — separate-repo Python consumer; the existing Corviduo reference pattern)
- `mead-hall/src/server/` (TypeScript broker — separate-repo, server-side proxy, not a TUI reference)

---

## Frame: what this artifact is for

A **dev-grade debug-observability TUI** for the Conversation API. The point is to see *everything* Worldtree is doing on each turn: agent SSE stream + persona/Vili affect dispatch + tool calls + Bifrost handshake state + admin lifecycle events + raw server logs, side-by-side, in one terminal. Smoke-testing the API is what happens when you turn this thing on and a turn flows through it; the value is the **observability surface**, not the chat interface.

The forcing function "watch Vili dispatch fire on each turn against real vastblueai" (v0.19.0 / #177) is one instance of the general purpose: any new persona / agent / tool / contract work should be debuggable through this client.

It is NOT: a polished consumer for end-users (that's the web app); an admin tool (that's `sessions_cli.py`); a multi-session orchestrator; cross-platform UI testing; a featuretracking shadow of the web app.

**This reframe (debug-observability primary, smoke-test secondary) changes the framework call.** The earlier draft committed to `rich + prompt_toolkit` for a small chat-shaped tool. Multi-pane log surface with independent scrollback per pane is the wrong shape for `rich.live` — it tends to reinvent a widget framework. §1 below now recommends Textual; §5 expands the observability surface; §6 lists the log streams as positive clauses.

---

## 1. TUI framework — recommend Textual

**Options canvassed:** textual, rich (alone), urwid, prompt_toolkit.

**Recommendation:** **Textual.**

**The load-bearing distinction (per Volva's reframe):** the call is not "Textual is too large" vs "rich is small enough"; it is **"are we building a full-screen application shell, or not?"**

- **REPL-shaped tool** with linear transcript + optional inline log block + single-prompt input: `rich + prompt_toolkit` is the right call.
- **Application-shell tool** with durable split panes, independent scrollback per pane, mouse/focus behavior, live status widgets, a continuously updating persona panel, AND several log surfaces side-by-side: **Textual is the simpler honest dependency.**

The operator's reframe ("lots and lots of debug logging surface, since that's the primary purpose") moves Ratatoskr into the second category. So does §5's log-surface inventory (persona / audit / Bifrost / tool-events / `/admin/events` / server stdout). A real side panel + streaming chat + prompt redraw composed ad hoc on rich+pt is exactly where the maintainer accidentally reinvents a widget framework — Volva flagged this directly: §1 (rich+pt) and §5 (multi-pane observability) pull in opposite directions under the earlier draft.

**Reasoning:**

- Textual's first-class primitives — `Log` widget with per-pane scrollback, `TabbedContent` for log-surface switching, `Horizontal`/`Vertical` layout with resizable splits, key-binding dispatch, mouse focus, async-native event loop — are exactly what a multi-pane debug-observability TUI needs. Building this on `rich.live` re-renders the entire canvas per frame and requires manual scrollback management per region; the layering becomes the work.
- The "Textual rewards long-lived product UIs" framing in the prior draft was wrong-axis. Textual ALSO rewards small TUIs that happen to have persistent panes and async streams. The size of the application surface (panes × async sources × persistent state) is the right axis, not "polished product vs dev tool." Ratatoskr is small in feature-count but structurally application-shell-shaped.
- `terminal.py`'s pattern (`rich.live` + readline) handles ONE active output region (the agent transcript) plus collapsible thinking-panel during streaming. It does not handle multiple concurrent log streams with independent scrollback. The precedent does not scale to Ratatoskr's scope.
- `urwid` has no async story and is the wrong size for SSE + multi-stream consumption. Reject.
- `prompt_toolkit` alone is REPL-shaped, not application-shell-shaped. Use it as the input-field widget INSIDE Textual if needed, but it doesn't carry the multi-pane layout.

**Implications for the rest of the brief:** the framework flip means §6 can drop the "stays small" framing in favor of "stays focused on debug observability"; the maintenance argument shifts from "few LOC" to "boring framework that does what we'd otherwise build by hand." See §8 for the prompt-redraw-strategy issue this resolves.

**Locked by operator:** Textual, application-shell. The REPL-shell alternative was considered and rejected — Ratatoskr is structurally application-shell-shaped because §5's observability surface is multi-pane.

---

## 2. Repo placement and version-skew strategy

**Resolved by operator:** Ratatoskr ships as a **separate repo with a separate dev team**, paralleling the Corviduo consumer cadence (skaldsong, web app, mead-hall, matrix bridge). No in-tree-vs-separate debate to run — the team boundary forces the repo boundary.

**Recommended home:** `gitea.phasefinal.com/vh/ratatoskr` (matching the Corviduo `vh/<name>` convention). Python 3.12+, `uv`-managed per the operator's tooling preference, `pyproject.toml` with `hatchling` backend.

**Boundary discipline:** Ratatoskr depends ONLY on:
- `httpx` + `httpx-sse` (network layer)
- `textual` (TUI framework — see §1)
- Worldtree's **published Conversation API spec** (`docs/conversation-api-spec.md`) — consumed by READING the spec, not by importing Worldtree code

Hard rule: **Ratatoskr does NOT import anything from Worldtree's source tree.** No `core.*` imports, no `from worldtree.*`, no shared models, no submodule. The spec is the entire surface.

**Version-skew handling (the methodological cost of separate-repo, made concrete):**

When Worldtree changes the SSE event vocabulary, session lifecycle, or auth surface, Ratatoskr's local tests do not run. Three mitigations the team should adopt:

1. **Spec-version pin.** Pin to a Worldtree git SHA in Ratatoskr's `pyproject.toml` (`worldtree-spec-rev = "abc1234"`). Bump it explicitly. Drift between pin and live Worldtree is the version-skew, and it's now a tracked field rather than implicit.
2. **Recorded-SSE snapshot tests.** Capture real SSE transcripts from Worldtree (`pytest --record-against http://localhost:8080`); replay them in Ratatoskr's tests. When Worldtree changes the event shape, the snapshot replay fails loud. Tradeoff: re-recording is a manual step.
3. **Conformance smoke test in Ratatoskr CI.** A single test that boots Worldtree (via Docker compose in CI) and runs a one-turn happy path. Catches gross breaks. Expensive (full Worldtree dep) but high-confidence.

**Recommend all three.** Spec-pin is cheap and load-bearing for "what version are we built against." Snapshot tests catch unit-level drift. Conformance smoke catches integration-level drift. Triple coverage is appropriate for a consumer where the dev team is decoupled from the spec author.

**Consumer-side discoveries flow back to worldtree-dev via althing**, not via PR — Ratatoskr's team finds spec gaps; brokkr-smithy or the operator routes them. The separate-repo posture means consumer-side validation actively happens (matches the `[[feedback_consumer_side_validation_catches_what_surveys_cannot]]` discipline).

---

## 3. SSE consumption pattern — recommend `httpx-sse`

**Options canvassed:** `httpx-sse`, hand-rolled `httpx.AsyncClient.stream` + `aiter_lines` (the skaldsong pattern), `aiohttp-sse-client`.

**Recommendation:** `httpx-sse`.

**Reasoning:**

- The server emits `id:` per event in the composite `{turn_id}:{seq}` format (INV-014). This is load-bearing for the SSE-resume flow (issue #113) and for `Last-Event-ID` reconnect against `/admin/events`. Hand-rolled `data:`-only parsing — which is what skaldsong's `_aggregate_sse` does — silently drops the `id:` line and re-implementation will not get the multi-line `data:` corner cases right the first time.
- `httpx-sse` is small (~300 LOC), depends only on `httpx` (already in tree), and parses the full SSE wire format including `event:`, `id:`, `retry:`, multi-line `data:`, and comment lines. It's the boring-correct option.
- `aiohttp-sse-client` requires `aiohttp`. Adds an unrelated dependency. Reject.
- The skaldsong precedent is real but explicitly does NOT handle resume — skaldsong's call shape is single-shot render-and-aggregate, not reconnect-on-disconnect. The TUI client wants reconnect (long-lived dev session, laptop suspends, etc.), so the precedent is the wrong reference.

**Reusability note:** The TUI client's SSE consumer can serve as the **reference Python implementation** for future Python consumers that need SSE-resume semantics. Worth writing as a single small module in `tools/<name>/sse_client.py` rather than burying it inline — other consumers will want to crib.

---

## 4. Session model — recommend (a) single-session, auto-resume, plus a startup picker

**Options canvassed:** (a) single session resume-on-relaunch (matches `terminal.py`); (b) session picker + in-app `/switch` command (matches web app chat commands).

**Recommendation:** **(a) single-session-per-launch**, with a startup picker invoked when more than one session exists for the current user, plus CLI flags `--session <id>` and `--new` to skip the picker.

**Reasoning:**

- The dev mental model coming from `terminal.py` is "open the tool, talk to the agent, close the tool." Multi-session orchestration is a different UI surface and adds in-app state (focus management between session lists and the active session, `/switch` semantics, dirty-state-on-switch, etc.) that the dev tool doesn't need.
- "Pick a session at startup" covers 100% of the real cases — devs either want to resume the last session they were poking at, or start fresh for a new probe. Both are one-shot decisions made at launch, not in-app navigation.
- `--session <id>` covers the scripted/CI case (smoke a specific session deterministically). `--new` covers the loop case (`for i in {1..5}; do ratatoskr --new --agent mimir --send "hello"; done`).
- The session picker UI is a Textual `DataTable` of `GET /sessions` results with key-bind selection.

**Session-identity-always-visible invariant (per Volva):** even with single-session-per-launch, the active session's `agent_id` + last 8 chars of `session_id` MUST appear in a persistent status widget (Textual `Footer` or `Header`). Devs running multiple `ratatoskr` instances in different terminals need to know which is which without sending a turn to find out.

**Negative clauses:** no in-app `/switch`; no in-app session creation (`--new` flag only); no in-app session deletion (use `sessions_cli.py`); no in-app rename (use `PATCH /sessions/{id}` outside the tool).

---

## 5. Debug-observability surface — recommend multi-pane log dashboard

**Reframe:** The earlier draft scoped this to "persona/Vili observability via persona.log file-tail." The operator's reframe ("lots and lots of debug logging surface, since that's the primary purpose") expands this to: **every observable surface Worldtree exposes during a turn should be visible side-by-side, switchable, with independent scrollback.**

**The observability surfaces, ranked by load-bearing-ness for live debug:**

| Surface | Source | Transport | Scope | v1? |
|---|---|---|---|---|
| Agent SSE stream (chat) | `POST /sessions/{id}/messages` | SSE (httpx-sse) | This session | ✅ primary pane |
| Persona / Vili affect log | `${WORLDTREE_DATA_PATH}/persona.log` | file-tail | Process-wide | ✅ side pane |
| Tool-call events | `GET /sessions/{id}/tool-events` (#123 if opted in) OR SSE `tool_start`/`tool_result` | HTTP poll OR inline-from-SSE | This session | ✅ side pane (inline-from-SSE for v1) |
| Admin lifecycle events | `GET /admin/events` SSE | SSE | Process-wide (filterable by session_id) | ✅ side pane (admin-key only) |
| Bifrost dispatch state | `GET /admin/sessions/{id}/bifrost` | HTTP poll on turn boundary | This session | ✅ status widget (refresh per turn) |
| Tool surface inventory | `GET /admin/sessions/{id}/tools` | HTTP once at session-attach + on agent change | This session | ✅ status widget |
| Server stdout (raw logs) | Worldtree stdout / structured log file | file-tail | Process-wide | ✅ opt-in v1 — pane appears only when `--server-log <path>` is passed; no auto-detection of well-known paths (per operator) |
| Pending tasks | `GET /pending-tasks` (#119) | HTTP poll | This user | ⚠️ optional v1 (skip unless someone asks) |
| Search hits (debug aid) | `GET /search` | HTTP one-shot | This user | ❌ defer to v2 |

**Layout shape:** Textual `Horizontal` with two columns. Left column is the chat transcript (primary pane, `RichLog` widget) with input field below. Right column is `TabbedContent` cycling through the side panes — `Persona`, `Tools`, `AdminEvents`, `BifrostState`, `ServerLog`. Tab key (`Ctrl+1..5`) jumps between tabs without losing focus on the input field. Status footer carries session id, agent id, admin-key-present indicator, and current pane name.

**Persona pane scope (locked by operator: label-don't-refuse):**

The `persona.log` is **process-wide** — one file gets writes from every session active on the server. Ratatoskr always tails the log without filtering or refusing, but **labels the pane to make the leak visible**:

- Pane title: `[Persona — PROCESS-WIDE]` when log lines lack consistent `session_id` scoping.
- Pane title: `[Persona — session 3a8f…]` when log lines reliably carry the active session_id AND Ratatoskr can filter cleanly.

Rationale: the dev is responsible for knowing they're against a shared instance; the tool's job is to make that visible, not to second-guess the configuration. Refusing-to-tail against a non-local Worldtree was considered and rejected — too paternalistic for a dev tool. Filter-if-metadata-allows is the auto-applied refinement when the log schema supports it.

**Persona-log schema check (action item for the dev team):** confirm what fields `persona.log` lines actually carry. `core/persona/registry.py:_init_persona_log` writes via a logger with format `"%(asctime)s %(message)s"` — whatever the call sites at `_persona_log.debug(...)` pass is what shows up. The dev team should grep `_persona_log.debug` call sites in `core/persona/` to determine whether session_id is reliably present; the answer determines which pane title is used.

**Why file-tail over a new endpoint:**

- `/admin/events` (already exists, INV-046..INV-052) explicitly does NOT carry persona affect state (INV-049: "Events NEVER carry [...] persona affect state [...]"). So it's the wrong surface for live affect observation.
- A new `/persona/log` endpoint would need: contract amendment to `conversation_api.contract.md`, a `/vor` round, ready-for-agent + AFK-dispatch loop. Weeks of consumer-side spec work for a debug feature that file-tail handles in a day.
- File-tail is fine **for the local-dev posture** that the rest of the brief commits to. Open question 3 surfaces the shared-instance case as the follow-up trigger.

**Follow-up trigger for path (b) (new `/persona/log` SSE endpoint):** if (i) a Worldtree-on-server / TUI-on-laptop debug case appears, OR (ii) the persona-log-pane needs proper per-session scoping that the file-line schema can't carry, the contract cost of the new endpoint becomes worth paying.

---

## 6. Scope creep guards — frame is correct, one narrowing

**Originating frame from worldtree-dev's ask:** "dev tool, parity with web app SSE consumption, no multi-user, no auth UX beyond `--api-key` flag, no history-search, no markdown rendering of agent output (raw text only)."

**Recommendation:** Adopt the frame with **one change**: enable markdown rendering on agent output by default, with `--raw` flag to opt out.

**Reasoning for the one change:** The dev tool exists to debug what the user actually sees, including formatting. The agent's prose IS formatted (the web app renders markdown). Stripping rendering means the dev's mental model of "what does the user see?" diverges from what the harness shows. `terminal.py` already renders markdown via `rich.markdown.Markdown` — the precedent is in-house. The `--raw` flag covers the case where the dev wants to see the underlying text shape (e.g., debugging a model that emits malformed markdown). **Do not pre-design a `--no-stream-formatting` mid-tier flag** (per Volva); add it only if streaming-Markdown rendering is empirically ugly on partial deltas.

**Explicit negative clauses (the "is NOT" list):**

- NOT a polished consumer for Worldtree's end-users.
- NOT a Worldtree-admin tool (admin CLI is separate).
- NOT a multi-session orchestrator — one session per launch.
- NOT cross-platform — terminal-native, no Windows-cmd compat target.
- NOT a featuretracking shadow of the web app — when the API grows, the TUI does not necessarily grow with it.
- NOT supporting uploads in v1.
- NOT a cross-session observability dashboard (single-session focus on chat; `/admin/events` is consumed but filtered to the active session).
- NOT a benchmarking harness (use `tests/` for benchmarks).
- NOT a remote-Worldtree debug client (file-tail surfaces assume local-dev posture).

**Explicit positive clauses (the "is" list):**

- **Maximum-observability dev tool for the Conversation API.** Every log/event surface Worldtree exposes during a turn is visible side-by-side. Debug observability IS the product; chat is the input mechanism.
- Multi-pane log dashboard: chat transcript + persona pane + tool-event pane + admin-events pane + Bifrost-state widget + tool-surface widget + (optional) raw server log.
- Reference Python implementation of SSE-resume + multi-stream concurrent SSE consumption against the v1 spec.
- Doubles as a reusable Python harness for cross-consumer dev work (skaldsong devs, mead-hall devs, future persona / agent / contract work).
- Honest API consumer: no `core.*` imports, boundary-enforced by smoke test (see §2).

---

## 7. Naming — locked: Ratatoskr

**Locked by operator.** Ratatoskr, the squirrel that runs up and down Yggdrasil carrying messages between the eagle at the top and Níðhöggr at the bottom.

The fit:
- Yggdrasil = Worldtree (literal name resonance). Ratatoskr runs up and down the World Tree carrying messages between levels — the TUI runs up and down Worldtree's API surface (sessions, turns, persona, tools, admin events, Bifrost state) carrying messages between layers.
- Chatty / gossip-carrier framing matches the debug-logging-primary purpose. The squirrel constantly talks; the TUI constantly shows everything.
- No curse, no death, no weighty mythological arc. Small, fast, useful.

Earlier candidate Andvari was demoted on the cursed-ring association.

---

## 8. Terminal-mechanics and shape decisions (per Volva's fresh-look)

Four issues the earlier draft missed. Decisions made here become CLI-shape decisions the Ratatoskr dev team should fold into their initial contract / implementation.

### 8a. Prompt redraw strategy under concurrent renders

With Textual: this is handled — the framework owns the screen, widgets serialize their renders, the input field is a widget that doesn't compete with log streams.

If §1 flips back to `rich + prompt_toolkit`: use `prompt_toolkit`'s `patch_stdout` context manager around all `console.print` / `rich.live` calls, OR serialize all output through a single asyncio task that owns the renderer. The failure mode without this discipline is: SSE chunks and log-tail lines corrupt the input prompt mid-typing. Name the strategy explicitly in the impl spec; don't leave it as "we'll figure it out."

### 8b. Non-interactive `--send` mode

The interactive TUI is the primary product. ALSO ship a non-interactive mode:

```bash
ratatoskr --send "hello mimir" --agent mimir --new --no-tui
```

Streams the response to stdout and exits. Useful for: CI smoke tests, scripted dev probes, `for i in ...` loops, paste-into-a-shell debugging. Keeps Ratatoskr honest as an API consumer — non-interactive mode exercises the SSE consumer code path without any TUI dependency.

Implementation: same `httpx-sse` client module under the hood; the TUI is just one of two presenters (TUI presenter vs stdout presenter). Don't fork the API-consumption code; share the consumer, branch the presenter.

### 8c. Ctrl-C semantics (locked: two-stage cancel-then-exit)

**Locked by operator.** Two-stage cooperative Ctrl-C:

1. **First Ctrl-C while a turn is streaming:** send `POST /sessions/{id}/turns/{turn_id}/cancel` server-side. Stream terminates with a `cancelled` event. Prompt returns to user. Footer hint flips to: `"Press Ctrl-C again to exit"`.
2. **Second Ctrl-C (or first Ctrl-C while idle):** exit the app gracefully.

The footer-hint state transition is load-bearing — the dev needs to see that the next Ctrl-C will exit, otherwise they hit it again expecting another cancel and lose their session. Reset the footer hint to `"Ctrl-C twice to exit"` after the first cancel completes and the user issues their next input.

Ctrl-D bound to immediate exit (no two-stage; EOF on the input field is unambiguous).

### 8d. Last-Event-ID persistence shape

The brief commits Ratatoskr to being the reference Python SSE-resume implementation. Concretely:

- **Per-stream `Last-Event-ID` lives in-process** (Python dict keyed by `turn_id` for chat stream; separate key for `/admin/events`). Reconnect-within-process works.
- **Cross-process resume is NOT supported in v1.** When Ratatoskr exits and restarts, in-flight turns are abandoned (server-side `cancelled` event will fire eventually via stall watchdog). Resume-across-process would require persisting `Last-Event-ID` to disk under `~/.config/ratatoskr/` keyed by session_id — defer to v2 if it turns out to matter.

Call this out in the README explicitly: **"reconnect, not resume-across-process."** Volva flagged the distinction.

---

---

## Resolved decisions (locked)

All shape decisions are now locked. Recording for the dev team:

- **Naming:** Ratatoskr.
- **Repo placement:** separate repo at `gitea.phasefinal.com/vh/ratatoskr`, separate dev team.
- **No Worldtree-source imports.** Triple-mitigation for version-skew (spec-pin + recorded-SSE snapshot tests + conformance smoke). See §2.
- **Framework:** Textual. Application-shell shape; multi-pane debug-observability dashboard. See §1.
- **Persona-pane PII posture:** label-don't-refuse. Always tail `persona.log`; title prefix carries the leak-visibility (`[Persona — PROCESS-WIDE]` or `[Persona — session <id>…]` when log schema supports filtering). See §5.
- **Server-stdout pane:** opt-in via `--server-log <path>` flag. No auto-detection. See §5 table + §5 prose.
- **Ctrl-C semantics:** two-stage cooperative (first cancels in-flight turn server-side; second exits app). Footer hint flips on first cancel. See §8c.

---

## Suggested next steps

1. **Volva second-opinion: completed.** Reply landed in thread `01KS3VF6W33N3V5FNMGQ91YNVD` (msg `01KS3VGSJ6QJ`, 2026-05-20). Her reframe converged with the operator's debug-observability reframe on the framework call; her four fresh-look misses (prompt redraw, `--send` mode, Ctrl-C, Last-Event-ID persistence) are folded into §8.
2. **Brief commits to `brokkr-smithy/docs/ratatoskr-design-brief.md`** (this file). Once committed, the design is locked at this version; the Ratatoskr dev team consumes from this artifact.
3. **Hand-off to the Ratatoskr dev team is operator-mediated** — not via althing reply to worldtree-dev. The separate-team boundary is now in effect.
4. **Ratatoskr dev team owns implementation:** new repo scaffold, Textual app skeleton, `httpx-sse` consumer module, observability-pane wire-up, contract authoring on their tree. brokkr-smithy's role on Ratatoskr concludes with this brief.
5. **Optional follow-up:** brokkr-smithy posts a courtesy ack to worldtree-dev on thread `01KS3R34XD3N6HMK91VXESHGW7` — "your ask spawned a new repo with a separate team; we'll ping you when there's something to consume against Worldtree."
