diff --git a/docs/contracts/issues/1.contract.md b/docs/contracts/issues/1.contract.md index 25ef26d..13bcf12 100644 --- a/docs/contracts/issues/1.contract.md +++ b/docs/contracts/issues/1.contract.md @@ -89,6 +89,7 @@ The "reconnect, not resume-across-process" framing in the design-brief §8d MUST - **[compatibility]** Module must work against the spec pin (`55101e909abcd2219833266b6f905c5bc956e0f0`, Worldtree v0.19.0). Spec bumps trigger an explicit re-record of the recorded-SSE snapshot fixtures (see `tests/snapshots/README.md`). - **[performance]** Streaming MUST NOT buffer the full turn in memory — events are yielded as they arrive. The complete-response field on `Done` is what the server sends; the consumer does not re-aggregate from `text` events. +- **[compatibility]** Callers MUST configure their `httpx.AsyncClient` with a long-or-disabled `read` timeout for SSE flows. LLM streaming has multi-second idle gaps between events (especially during prompt-building, thinking phases, and long completions); httpx's default 5s read timeout would kill the connection mid-stream. The recommended shape is `httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0)` — disable read timeout, keep modest connect/write/pool timeouts so true network failures still surface promptly. As defense in depth, `stream_turn`'s ERROR_ROUTING also catches `httpx.ReadTimeout` and surfaces it as `SseConnectionDropped` — but the right place to configure is the caller-owned client. - **[security]** `Authorization` header lives on the caller's `httpx.AsyncClient`. Module does not log the header, does not log full event bodies (they contain user message content). Logs are limited to `(turn_id, seq, type)` triples. - **[style]** Async-native. No sync entry points. The consumer is `async def` + `async for`; presenters are async too. @@ -108,7 +109,7 @@ ERROR_ROUTING: local_handling: re-raise as SseConnectFailed(status=resp.status_code, body=resp.read()[:1024]) — server returned non-2xx before stream started (e.g., 404 session_not_found) flow_control: abort state_recovery: none (no events yielded yet) - httpx.ReadError | httpx.RemoteProtocolError: + httpx.ReadError | httpx.RemoteProtocolError | httpx.ReadTimeout: local_handling: re-raise as SseConnectionDropped(last_seen_sse_id=) flow_control: abort state_recovery: none (caller may reconnect_turn) diff --git a/docs/contracts/issues/3.contract.md b/docs/contracts/issues/3.contract.md index 5f97695..562e4f4 100644 --- a/docs/contracts/issues/3.contract.md +++ b/docs/contracts/issues/3.contract.md @@ -323,7 +323,7 @@ ERROR_ROUTING: flow_control: abort state_recovery: none STEPS: - 1. [setup, flexibility=prescriptive] OPEN httpx.AsyncClient(base_url=args.server_url, headers={"Authorization": f"Bearer {args.api_key}"}) via async-with + 1. [setup, flexibility=prescriptive] OPEN httpx.AsyncClient(base_url=args.server_url, headers={"Authorization": f"Bearer {args.api_key}"}, timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0)) via async-with — read=None disables the SSE-killing 5s default (issue #1 [compatibility] constraint) 2. [branch, flexibility=prescriptive] IF args.new: TRY: info = await sessions.create_session(client, args.agent_id) ON AgentNotFound as exc: diff --git a/docs/contracts/issues/4.contract.md b/docs/contracts/issues/4.contract.md index 3cd767e..cbecd5d 100644 --- a/docs/contracts/issues/4.contract.md +++ b/docs/contracts/issues/4.contract.md @@ -212,7 +212,7 @@ ERROR_ROUTING: state_recovery: none STEPS: 1. [setup, flexibility=prescriptive] Validate PRE-001 - 2. [sequential, flexibility=prescriptive] Open AsyncClient: self.client = httpx.AsyncClient(base_url=args.server_url, headers={"Authorization": f"Bearer {args.api_key}"}) + 2. [sequential, flexibility=prescriptive] Open AsyncClient: self.client = httpx.AsyncClient(base_url=args.server_url, headers={"Authorization": f"Bearer {args.api_key}"}, timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0)) — read=None disables the SSE-killing 5s default (issue #1 [compatibility] constraint) 3. [branch, flexibility=prescriptive] IF args.new: TRY: info = await create_session(self.client, args.agent_id) ON AgentNotFound | SessionApiFailed | httpx.ConnectError | httpx.ReadTimeout | httpx.TransportError: handle per ERROR_ROUTING (append + exit) diff --git a/persistent-memory.md b/persistent-memory.md index 8a609b6..2645998 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -30,7 +30,7 @@ separate dev team rather than an in-tree Worldtree tool. ## Current state / in-flight -**Status: `ratatoskr.sse_client` + `ratatoskr.sessions` + `ratatoskr.cli` + `ratatoskr.tui` (shell) all implemented via TDD against their issue-scoped contracts.** 164/164 tests GREEN (42 sse_client + 19 sessions + 58 cli + 43 tui + 1 boundary + 1 metadata); ruff clean. `--send` mode + TUI mode both wired into `ratatoskr` console-script entry point (TUI dispatched via lazy import to preserve issue #3 INV-001). +**Status: `ratatoskr.sse_client` + `ratatoskr.sessions` + `ratatoskr.cli` + `ratatoskr.tui` (shell) all implemented via TDD against their issue-scoped contracts; `--send` mode validated end-to-end against personal Worldtree.** 164/164 tests GREEN (42 sse_client + 19 sessions + 58 cli + 43 tui + 1 boundary + 1 metadata); ruff clean. `--send` mode + TUI mode both wired into `ratatoskr` console-script entry point (TUI dispatched via lazy import to preserve issue #3 INV-001). What's in the repo: - `docs/design-brief.md` — the locked design (copy from `brokkr-smithy/docs/ratatoskr-design-brief.md`). @@ -62,11 +62,10 @@ What's NOT in the repo yet: **Branch:** `main`. Remote: `origin → git@gitea.phasefinal.com:vh/ratatoskr.git` (added 2026-05-20). **Next natural moves:** -1. **`/volva-code-review docs/contracts/issues/4.contract.md`** against the freshly-landed `ratatoskr.tui` shell (precedent: #1 caught 4 drifts, #2 caught 3, #3 caught 5). The Textual `App` lifecycle + worker race patterns are likely sources of new drifts — cross-model fresh-eyes review on async UI code is high-value. -2. **Manual smoke against the personal Worldtree**: key landed in `env.sh` (see [[personal-worldtree-smoke-target]] memory); `source env.sh && ratatoskr --send "hello" --new --agent mimir` for stdout smoke OR `ratatoskr --new --agent mimir` for TUI smoke. First end-to-end wire validation against the live server. -3. **Side-pane issues** — design-brief §5 lists 5 side panes (Persona, Tools, AdminEvents, BifrostState, ServerLog). Each gets its own issue + contract + TDD pass on top of the shell. Persona is the natural first (file-tail of `persona.log` — cheap; no new Worldtree wire). -4. Record real SSE snapshot fixtures from a running Worldtree. `--send --new` redirected to a fixture file IS the recording probe — capture outputs to `tests/snapshots/` for replay-based regression coverage. -5. Startup session picker — design-brief §4 `DataTable` of `GET /sessions`. Modest scope; pairs naturally with the TUI shell. +1. **TUI smoke (operator-side)** — `source env.sh && uv run ratatoskr --new --agent mimir` from an interactive terminal. Validates the Textual app lifecycle + the post-Done markdown re-render end-to-end. Needs a TTY which this CC session doesn't have. +2. **Side-pane issues** — design-brief §5 lists 5 side panes (Persona, Tools, AdminEvents, BifrostState, ServerLog). Each gets its own issue + contract + TDD pass on top of the shell. Persona is the natural first (file-tail of `persona.log` — cheap; no new Worldtree wire). +3. Record real SSE snapshot fixtures from a running Worldtree. `--send --new` redirected to a fixture file IS the recording probe — capture outputs to `tests/snapshots/` for replay-based regression coverage. The smoke just proved the recording probe works. +4. Startup session picker — design-brief §4 `DataTable` of `GET /sessions`. Modest scope; pairs naturally with the TUI shell. ## Recent decisions @@ -101,6 +100,7 @@ decision. Captures rationale that won't be obvious from code alone. - `[2026-05-21]` **Issue #4 + contract: `ratatoskr.tui` shell.** Textual `App[int]` subclass with single chat-pane layout (Header + RichLog + Input + Footer). Composes existing modules: `create_session` on `--new` + `stream_turn` per turn + `cancel_turn` on Ctrl-C. Hard invariants: INV-002 session-identity-always-visible footer (with `` carve-out when `--session` is used without --agent); INV-003 two-stage Ctrl-C state machine (idle/streaming/cancelling per design-brief §8c); INV-005 markdown rendering default-on with `--raw` opt-out (deliberately produces a streaming-then-Markdown-render double-display — accepted v1 trade-off; Static-then-commit refactor deferred); INV-008 mid-session errors render to transcript and return to idle (don't exit). Shell-only scope — design-brief §5's five side panes (Persona, Tools, AdminEvents, BifrostState, ServerLog), startup session picker, Tab bindings, history rendering all deferred to follow-up issues. Concurrent in-place amendment of issue #3: `--send` becomes optional (when omitted, `main` lazy-imports `ratatoskr.tui.run_tui`); `--raw` flag added to `ParsedArgs`; `_parse_args`/`main` TESTS sections updated. The lazy-import preserves issue #3 INV-001 (no textual at cli module scope). `prd:` pinned to issue #4 body SHA `b1e73e7d2e3dd453` at `2026-05-21T06:21:37+00:00`; drift check clean. - `[2026-05-21]` **Volva paraphrase round on `docs/contracts/issues/4.contract.md`.** 5 findings, all amended (matching #3's hit rate). (1) INV-002 footer carve-out for `` agent slot when `--session` and agent_id unknown; tightened wording so the `` placeholder is an accepted satisfaction of "session-identity-always-visible". (2) "Ctrl-C twice to exit" idle hint vs single-press-exits state machine: kept the hint verbatim per design-brief §8c's conservative-by-design rationale, amended INV-003 to spell out the intentional discrepancy + cite the brief so an implementer can't "fix" it by accident. (3) Markdown double-render trade-off: INV-005 strengthened from "trade-off" to explicit "assistant response visibly appears TWICE in the transcript by design"; Static-then-commit refactor explicitly out-of-scope for this shell. (4) Silent input-discard during streaming: now writes `[busy] turn in flight; input ignored` to the transcript (visible notice, not silent swallow); new test for cancelling state too. (5) `{!r:.200}` format spec note: kept (valid Python repr-then-truncate-200), added inline comment explaining the syntax. Volva meta-note: "discipline pulls weight here" — same calibration signal as prior rounds. - `[2026-05-21]` **`ratatoskr.tui` shell implemented via TDD against issue #4's contract.** 43 contract-listed tests + 4 issue-#3-amendment tests authored + GREEN per the vertical-slice ordering (`_render_event_to_log` → `_cancel_via_sse` → CLI amendments → app class + on_mount + on_unmount → on_input_submitted → _stream_turn_worker → action_interrupt + action_quit → run_tui). Two in-flight contract amendments during TDD: (a) PRE-002 of `run_tui` was `(args.session_id is None) != args.new` — backwards, fails when session_id is set + new=False; corrected to `bool(args.session_id) != bool(args.new)`; (b) RichLog created with `markup=False` (was `markup=True` in contract) because Rich interprets `[xxx]` as style spans and strips them — would break every labeled stderr-style line. The post-Done Markdown rendering still works because `rich.markdown.Markdown` is a Renderable that doesn't need widget-level markup. Implementation choices: (a) worker takes the `log` widget as a parameter (passed from on_input_submitted) — querying `#transcript` from inside a Textual worker context fails with NoMatches; (b) `_spy_writes(monkeypatch)` test helper records every RichLog.write call because RichLog's `.lines` Strip buffer isn't populated synchronously after .write() returns, making post-app-shutdown inspection unreliable; (c) SIGINT-mid-stream tests use custom `httpx.AsyncByteStream` subclasses with `asyncio.Event` gates to make timing deterministic without sleep-based polling. 164/164 tests GREEN; ruff clean (one `# noqa: RUF001,RUF003` for the intentional `❯` INV-006 prompt prefix). +- `[2026-05-21]` **First manual smoke against personal Worldtree — successful + uncovered a real defect.** `source env.sh && uv run ratatoskr --send "hello mimir..." --new --agent mimir` against `http://10.250.50.152:8081`. First run produced `httpx.ReadTimeout` mid-stream after the `worker_phase BuildingPrompt` event — httpx's default 5s read timeout killed the connection during mimir's thinking phase (LLM streaming has multi-second idle gaps that exceed the default). Fix: caller-owned `httpx.AsyncClient` now constructed with `timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0)` in both cli (`_amain`) and tui (`on_mount`); `read=None` disables the SSE-killing timeout while keeping modest connect/write/pool timeouts. Defense in depth: `sse_client.stream_turn` ERROR_ROUTING also catches `httpx.ReadTimeout` → `SseConnectionDropped` (was just `ReadError | RemoteProtocolError`). Three contracts amended in-place: issue #1 adds [compatibility] constraint about caller timeout config + adds ReadTimeout to ERROR_ROUTING; issues #3/#4 spell out the explicit timeout shape in their AsyncClient construction step. Re-smoke after fix: SSE stream consumed cleanly, agent text appeared, `[done] turn_id=88 model=qwen3.6-35-a3b duration_ms=2351 ... cached_input_tokens=0`. Stdout-only smoke (`2>/dev/null`) returned clean text + exit 0 — INV-002 (stdout/stderr split) holds end-to-end against real wire. Wire-compat envelope from infra-ops/worldtree-dev's analysis (personal v0.16.2 vs ratatoskr's v0.19.0 pin) confirmed by smoke — no malformed_sse_id, no turn_id_flip, no spec drift. The personal Worldtree's mimir runs qwen3.6-35-a3b. **TUI mode not smoke-tested from this session** — requires TTY; operator should smoke via `source env.sh && uv run ratatoskr --new --agent mimir`. - `[2026-05-21]` **Volva code-vs-contract review on `ratatoskr.tui` shell.** 8 findings — 5 drifts, 2 precision (contract-amend), 1 test-gap (follows from F1). All addressed. (1) DRIFT, primary: `self.sub_title` (Header) and `self.hint` (plain attr) were set but never rendered to a visible Footer widget — INV-002 + INV-003 require visible Footer-area rendering. Fixed: added `Static(id="identity")` + `Static(id="hint")` widgets in compose; `_set_hint()` helper mirrors state→widget; on_mount calls `query_one("#identity", Static).update(...)`. Same-model TDD missed this because tests asserted `app.sub_title` / `app.hint` internal state, not visible widget content. Volva caught immediately. (2) DRIFT: `_stream_turn_worker` signature widened to `(content, log)` during TDD as a workaround for a NoMatches-during-worker-execution test failure. Reverted to single-param `(content)` per contract; queries `#transcript` internally. The original test failure was actually a test-timing issue resolved by adding `await pilot.pause()` before the polling loop in `_submit_and_wait`. (3) DRIFT: `run_worker(coro)` missing `exclusive=True`. Restored. (4) DRIFT: `run_tui` PRE-001 missing `isinstance(args, ParsedArgs)` check; only had `args.send_content is None`. Added. Required hoisting `from ratatoskr.cli import ParsedArgs` out of `TYPE_CHECKING` (runtime import — fine, no circular dependency: cli imports tui lazily inside main; tui imports cli unconditionally at module load). (5) DRIFT: `_render_event_to_log` missing PRE-001 union assertion. Added. (6) PRECISION (amend contract): RichLog `markup=True, highlight=True` in contract; implementation chose `markup=False, highlight=False` to preserve labeled-line semantics. Contract amended with rationale comment in COMPOSE shape. (7) PRECISION (amend contract): `on_mount` POST-003 said `agent_id is non-empty`; for `--session` attach without `--agent` it's None per INV-002 carve-out. Contract amended. (8) TEST-GAP: tests checked `app.sub_title` / `app.hint`; updated to query `Static(#identity).render()` / `Static(#hint).render()` (the actual visible widgets). Volva meta-note: "TDD pass caught most stream/session/error mechanics, but tested internal state where the contract required visible Footer behavior, so same-model TDD would plausibly miss the primary drift." Calibration evidence: cross-model review pulls weight on UI-observability gaps in particular, where the test-author can convince themselves the state is correct without verifying the user can see it. 164 tests GREEN post-fix. ## Tried and abandoned diff --git a/src/ratatoskr/cli.py b/src/ratatoskr/cli.py index ad9a79b..1c4366c 100644 --- a/src/ratatoskr/cli.py +++ b/src/ratatoskr/cli.py @@ -263,6 +263,11 @@ async def _amain(args: ParsedArgs) -> int: async with httpx.AsyncClient( base_url=args.server_url, headers={"Authorization": f"Bearer {args.api_key}"}, + # SSE streaming sits idle between events while the LLM thinks. + # Default 5s read timeout would kill mid-stream; disable it. + # connect/write/pool keep modest timeouts so true network failures + # still surface promptly. + timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0), ) as client: if args.new: assert args.agent_id is not None diff --git a/src/ratatoskr/sse_client.py b/src/ratatoskr/sse_client.py index 6a5d9d0..0d654c0 100644 --- a/src/ratatoskr/sse_client.py +++ b/src/ratatoskr/sse_client.py @@ -304,7 +304,12 @@ async def _iter_events( if isinstance(event, (Done, Error, Cancelled)): terminal_seen = True return - except (httpx.ReadError, httpx.RemoteProtocolError) as exc: + except (httpx.ReadError, httpx.RemoteProtocolError, httpx.ReadTimeout) as exc: + # ReadTimeout covers idle gaps that exceed httpx's read timeout — the SSE + # stream went quiet long enough for httpx to give up. Treat the same as a + # raw read error: surface as SseConnectionDropped so the caller can decide + # whether to reconnect_turn. (Callers SHOULD configure a long-or-disabled + # read timeout on their AsyncClient for SSE; this is defense in depth.) raise SseConnectionDropped(last_seen_sse_id=last_sse_id) from exc if not terminal_seen: # Clean EOF before terminal event — INV-001 says stream MUST NOT end diff --git a/src/ratatoskr/tui.py b/src/ratatoskr/tui.py index fbd9edb..9733c77 100644 --- a/src/ratatoskr/tui.py +++ b/src/ratatoskr/tui.py @@ -120,6 +120,9 @@ class RatatoskrApp(App[int]): self.client = httpx.AsyncClient( base_url=self.args.server_url, headers={"Authorization": f"Bearer {self.args.api_key}"}, + # SSE streaming sits idle between events while the LLM thinks. + # Default 5s read timeout would kill mid-stream; disable it. + timeout=httpx.Timeout(connect=10.0, read=None, write=10.0, pool=10.0), ) log = self.query_one("#transcript", RichLog) if self.args.new: