From 93e41763468a8bc2fb440ed90438a6acb0d1a435 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 30 Jun 2026 14:41:06 -0700 Subject: [PATCH] docs: author v1 coverage-map ledger; lock scope mandate A MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First coverage map — every Worldtree v1-FROZEN I/O point x ratatoskr status. Anchored on WT's frozen machine-readable artifacts (OpenAPI 2.2.0 = 40 REST path-groups + SSE schema = 11 events + bifrost wire v0.6), not the stale vendored prose markdown. - SSE 11/11 and Bifrost provider planes 8/8 covered + live-proven; client REST 7/40 live, 11 in-scope frontier, 22 excluded-by-design. - Scope mandate A (operator): v1 done = every frozen point classified (covered-or-excluded-with-rationale), zero unaccounted; not a feature-complete client. - Finding P-1: vendored prose markdown is byte-identical to live WT but frozen at v0.35.16-era content; b2's surface lives in the OpenAPI 2.2.0 + SSE-schema JSON we don't vendor. Pin-remediation pending operator nod. No version bump (docs-only). --- docs/coverage-map.md | 247 +++++++++++++++++++++++++++++++++++++++++++ persistent-memory.md | 3 + 2 files changed, 250 insertions(+) create mode 100644 docs/coverage-map.md diff --git a/docs/coverage-map.md b/docs/coverage-map.md new file mode 100644 index 0000000..65e19a9 --- /dev/null +++ b/docs/coverage-map.md @@ -0,0 +1,247 @@ +# Ratatoskr v1 coverage map + +_The v1 convergence-target ledger. Ratatoskr has **no self-defined feature +roadmap**: v1 = consume all of Worldtree's I/O points, reached when Worldtree +hits 1.0 (operator, 2026-06-19; auto-memory +`project-ratatoskr-v1-derived-from-worldtree-io-coverage`). This file is that +coverage map — every Worldtree v1-frozen I/O point × ratatoskr's coverage +status, so "are we at v1?" is a ledger lookup, not a judgement call._ + +**First authored:** 2026-06-30 (the v1 coverage-audit kickoff). + +--- + +## Frozen target + +Worldtree is at **v1.0.0b2** — approaching 1.0, with its wire surfaces now +**FROZEN** (Worldtree `docs/v1-schema-freeze-manifest.md`, #326). The audit +anchors against the frozen machine-readable artifacts, NOT the prose markdown: + +| Worldtree v1 surface | Frozen anchor | Ratatoskr role | +|---|---|---| +| Conversation REST API | OpenAPI `info.version` **2.2.0** (`Worldtree/docs/conversation-api-openapi.json`, sha `dbdf4e24…`) — **40 path×method groups** | **client** (debug TUI / web) | +| Conversation SSE events | `conversation-api-sse-events.schema.json` (sha `9deeebf4…`) — **11 discriminated event types** | **client** | +| Bifrost wire (consumer protocol) | wire **v0.6** STABLE/FROZEN (`bifrost==1.0.0`) — memory + affect planes | **provider** (Worldtree dispatches into us) | + +> **Pin drift (finding P-1).** Ratatoskr vendors the **prose markdown** +> (`docs/conversation-api-spec.md`), which is **byte-identical to live +> Worldtree's markdown** but frozen at v0.35.16-era content (last WT edit +> 2026-05-31). The markdown does **not** document the b2 surface: 7 endpoints +> (below), the 409/503 on messages-POST (#331), the unified error envelope +> (#328), or the SSE schema. **Worldtree's authoritative v1 truth is now the +> OpenAPI 2.2.0 + SSE schema JSON**, which ratatoskr does not vendor. See +> § Pin remediation. + +**7 endpoints new in b2 OpenAPI, absent from our vendored markdown:** +`/admin/keys/bulk`, `/admin/persona/archive`, `/admin/persona/erase`, +`/admin/usage`, `/embed`, `/judgments`, `/me/usage`. + +--- + +## Scorecard + +**Scope mandate: A (ledger-mandate), locked by operator 2026-06-30.** v1 "done" += every frozen I/O point is **classified** (covered or excluded-with-rationale), +zero unaccounted. NOT "feature-complete client." All scope-pending rows are now +resolved (§ Surface 1, scope-resolution table). + +| Surface | Points | ✅ covered-live | ⬜ gap (in-scope) | 🚫 excluded-by-design | +|---|---|---|---|---| +| REST (OpenAPI 2.2.0, path groups) | 40 | 7 | 11 | 22 | +| SSE events | 11 | 11 | 0 | 0 | +| Bifrost provider planes | 8 verbs | 8 | 0 | (10 gated verbs deferred) | + +**Legend.** ✅ consumed in code AND live-proven against real Worldtree · ⬜ a +debug-observability I/O point we should cover but don't yet (the convergence +frontier) · 🚫 deliberate non-goal per the design-brief negative clauses + the A +mandate. Counts are at the **path-group** level; mixed-method groups are +footnoted (e.g. `/sessions` POST is ✅ but its `GET` picker is an unwired +sub-gap). + +--- + +## Surface 1 — Conversation REST API (OpenAPI 2.2.0) + +### Covered — client path (ratatoskr's core identity) + +| Endpoint | Status | Where consumed | Note | +|---|---|---|---| +| `POST /sessions` | ✅ | `sessions.py:307` → `cli.py:482`,`tui.py:1508`,`web/server.py:155` | + `end_user_id`, `bifrost` binding; 404→AgentNotFound, 502→BifrostHandshakeFailed | +| `POST /sessions/{id}/messages` (turn stream, SSE) | ✅ | `sse_client.py:484` `stream_turn` → cli/tui/web | the primary surface; 409→AgentNotAvailable, 503→TurnLaunchUnavailable (b2 #331) | +| `POST /sessions/{id}/turns/{turn_id}/cancel` | ✅ | `sse_client.py:581` → cli/tui/web | two-stage Ctrl-C; 404/409 mapped | +| `GET /agents` | ✅ | `sessions.py:341` → `tui.py:1472`,`web/server.py:100` | Tier-1 roster; merged with local index | +| `GET /agents/{id}/persona_state` | ✅ | `sessions.py:384` → `tui.py:1132`,`web/server.py:386` | persona hydrate; 404/403 mapped | +| `POST /agents/define` | ✅ | `tier3.py:175` → `_run_define` | Tier-3 create | +| `PATCH /agents/{id}` | ✅ | `tier3.py:219` → `_run_patch` | Tier-3 mutate (system_prompt/model) | +| `DELETE /agents/{id}` | ✅ | `tier3.py:242` → `_run_delete` | Tier-3 hard-delete | + +**Sub-gaps inside ✅ path groups** (the method we use is live; a sibling method +on the same path is an unwired frontier item — see frontier Tier 1): +- `GET /sessions` — `sessions.py:198` `list_sessions` exists, **no caller**: the + startup session-picker (design-brief §4 v1) was never wired. +- `POST /sessions/{id}/messages` + `Last-Event-ID` — `sse_client.py:524` + `reconnect_turn` exists, **no caller**: the reference SSE-resume impl + (design-brief §8d) was never wired. +- `GET /agents/{id}` — consumer-agent lookup (`GET /agents/:` with + the owner key) is **manual-curl-only**, not in code. + +### In-scope gaps — the convergence frontier (debug-observability path) + +**Tier 1 — the debug-observability core (design-brief'd for v1, unbuilt):** + +| Endpoint | Status | Why in-scope | +|---|---|---| +| `GET /admin/events` | ⬜ | design-brief §5 v1 **AdminEvents pane**; = issue **#11**, **blocked** on `admin.events.read` scope (infra-ops) | +| `GET /admin/sessions/{id}/bifrost` | ⬜ | design-brief §5 v1 **BifrostState widget** — never built; admin-key-gated | +| `GET /admin/sessions/{id}/tools` | ⬜ | design-brief §5 v1 **Tools widget** — never built; admin-key-gated | +| `GET /capabilities` | ⬜ | server capability discovery — a turn flows through what's advertised | +| `GET /me` | ⬜ | whoami / key-identity — "which key am I against" is a debug primitive | +| (`GET /sessions` picker · resume) | ⬜ | sub-gaps above — presenter-wiring only, wrappers exist | + +**Tier 2 — rounds out I/O coverage under A (postdates the design-brief):** + +| Endpoint | Status | Why in-scope | +|---|---|---| +| `GET /sessions/{id}/tools` | ⬜ | Tier-3 owner-scoped tool introspection (#183) — reachable with the **consumer key** (no admin scope), the unblocked tool-introspection path | +| `POST /sessions/{id}/persona_state` (write) | ⬜ | affect-injection is debug-relevant; pairs with our provider affect plane | +| `POST /characters` · `DELETE /characters/{id}` · `GET /characters/{id}/state` · `GET /models/available-for-characters` | ⬜ | transient-characters (Echo) is a session-creation **routing path** a debug client should be able to drive a turn through | + +### Excluded by design — the design-brief negative clauses + +| Endpoint(s) | Status | Rationale (design-brief / memory) | +|---|---|---| +| `PATCH /sessions/{id}` · `DELETE /sessions/{id}` | 🚫 | §4: rename/delete happen outside the tool (`sessions_cli.py`) | +| `GET /sessions/{id}/messages` (history) | 🚫 | §6: single-session live transcript, no history fetch | +| `GET /sessions/{id}` | 🚫 | session detail — identity is footer-visible, no detail view | +| `GET /sessions/{id}/tool-events` | 🚫 | §5: tool calls observed **inline from SSE** `tool_start`/`tool_result`; persisted-events endpoint is opt-in only | +| `GET/POST /admin/keys` · `DELETE/POST /admin/keys/{id}` · `POST /admin/keys/{id}/rotate` · `DELETE/POST /admin/keys/bulk` · `POST /admin/keys/bulk/rotate` | 🚫 | §6: **NOT a Worldtree-admin tool** (key mgmt) | +| `POST /admin/sessions/{id}/retire` | 🚫 | admin session mutation | +| `POST /admin/persona/{archive,erase}` | 🚫 | admin persona GDPR ops (new in b2) | +| `POST /admin/users/{id}/tier` | 🚫 | admin user mgmt | +| `GET /me/usage` · `GET /admin/usage` | 🚫 | usage metering — not turn-flow observability (new in b2) | +| `GET /healthz` · `GET /readyz` | 🚫 | liveness probes — low debug value (could become a connect preflight; park) | +| `GET /search` | 🚫 | §5: consumer-product feature, not turn-flow (was "defer to v2") | +| `GET/POST /uploads` · `DELETE/GET /uploads/{id}` | 🚫 | §6: no uploads — consumer-product feature | +| `GET /pending` · `GET /sessions/{id}/pending` | 🚫 | §5: poll-only, no turn-flow signal (was "optional, skip") | +| `POST /embed` | 🚫 | embedding utility — no turn flows through it | +| `POST /judgments` | 🚫 | LLM-as-judge A/B eval (`response_a`/`response_b`/`rubric`) — standalone eval utility, not turn-flow | + +### Scope-resolution record (the 11 ❓ rows, resolved under A) + +The 2026-05-20 design-brief deferred several surfaces; the 2026-06-19 reframe +("v1 = full I/O coverage") put them back in tension. **Resolved 2026-06-30 under +mandate A** (debug-observability identity intact; classify, don't build-all): + +| Endpoint(s) | Resolution | +|---|---| +| `GET /search` · `uploads` (×2) · `pending` (×2) · `POST /embed` · `POST /judgments` | 🚫 **excluded** — consumer-product / eval / poll utilities, not turn-flow observability | +| `characters` (×4) · `POST /sessions/{id}/persona_state` | ⬜ **in-scope** (frontier Tier 2) — session-routing + affect-injection debug paths | + +Nothing remains ❓. The ⬜-vs-🚫 line follows the debug-observability test: *does +a turn flow through it / is it a layer worth watching live?* + +--- + +## Surface 2 — SSE events (11/11 ✅) + +Every frozen SSE event type is parsed in `sse_client.py:_envelope_for_type` +(342-411) and rendered by all three presenters (cli/tui/web). **Full coverage.** + +`text` · `worker_phase` · `thinking` · `text_boundary` · `tool_start` · +`tool_result` · `done` · `error` · `cancelled` · `awaiting_llm_first_token` · +`affect_update` + +> Caveat (not a gap): `affect_update` is wire-verified to emit **zero** events +> for consumer (Tier-3) agents — the persona-strip SSE path never populates for +> them (memory 2026-06-18). The handler is correct; the upstream emitter is +> silent. PAD for consumer agents is surfaced via our own provider read route +> (`GET /affect/state/{id}`, #18 D2), not this event. + +--- + +## Surface 3 — Bifrost provider planes (8/8 ✅, live-proven) + +Ratatoskr **implements** the provider side; Worldtree dispatches into it. +Live-proven end-to-end through real WT turns (#17/#18 smokes; combined `:8392` +WT-driven smoke 2026-06-20). + +**Memory plane** — covers the entire `bifrost.memory.MemoryDataStore` protocol +(required: `describe_store`, `get`, `get_many`, `search`, `upsert_many`) **plus** +`delete_many`: + +| Verb | Where | | +|---|---|---| +| `describe_store` | `memory_store.py:140` | advertises caps (sync) | +| `search` | `memory_store.py:224` | vector recall; scope_all AND / scope_any OR | +| `get` / `get_many` | `memory_store.py:293` / `:305` | point reads | +| `upsert_many` | `memory_store.py:150` | idempotent batch write; optimistic lock | +| `delete_many` | `memory_store.py:314` | transactional delete | + +**Affect plane** — covers `bifrost…InMemoryAffectStore` (`emit`, `fetch`): + +| Verb | Where | | +|---|---|---| +| `emit` | `affect_store.py:47` | conduit-opaque snapshot upsert (LWW) | +| `fetch` | `affect_store.py:116` | `{found, snapshot}`; mandatory since bifrost 0.10.0 strong-or-absent gate | + +Plus the non-wire PAD read route `GET /affect/state/{agent_id}` +(`affect_store.py:189`) and the combined `:8392` endpoint advertising both caps +by store-presence (`combined.py:46`). + +**Deferred-gated (advertised-unsupported, correctly out-of-scope for the basic +plane):** `scan`, `get_edges_for`, `upsert_edges`, `mark_invalid`, +`mark_superseded`, `patch_many`, `commit_checkpoint`, `lease_job`, +`read_checkpoint`, `health`. These live only in the bifrost reference +*extended* store, not the `MemoryDataStore` protocol; deferred per the #294 +re-scope (memory 2026-06-15). Re-evaluate only if Worldtree's Tier-3 path +starts exercising them. + +--- + +## Convergence frontier (the v1 to-do) + +**Tier 1 — debug-observability core**, in dependency order: + +1. **Session picker + SSE-resume** — wrappers exist (`list_sessions`, + `reconnect_turn`), need presenter wiring only. **Cheapest; unblocked.** +2. **`GET /capabilities` + `GET /me`** — cheap debug primitives. Unblocked. +3. **BifrostState + Tools widgets** (`GET /admin/sessions/{id}/{bifrost,tools}`) + — design-brief'd v1, unbuilt. Admin-key-gated reads. +4. **#11 — AdminEvents pane** — **blocked** on an `admin.events.read` scope grant + (infra-ops). The single externally-blocked item; everything else can ship + without it. + +**Tier 2 — rounds out coverage** (lower priority): + +5. **`GET /sessions/{id}/tools`** — owner-scoped tool introspection; consumer-key + reachable (no admin scope), so unblocked. +6. **Transient-characters routing** (4 endpoints) + **`POST /sessions/{id}/persona_state`**. + +--- + +## Pin remediation (finding P-1) + +The vendored prose markdown is stale against the frozen OpenAPI 2.2.0. Options +(operator-owned; see § Open decision item 2): + +- **Re-pin to the frozen machine-readable artifacts.** Vendor + `conversation-api-openapi.json` (2.2.0) + `conversation-api-sse-events.schema.json`, + add `.corviduo-canonicals.toml` pins (drift-checkable via `canonical_drift.py`), + keep the markdown as prose reference. Makes this coverage map reproducible + against a frozen, diffable target — the OpenAPI is the v1 truth. +- **Re-vendor the markdown only.** Near-no-op (content identical); leaves the + b2 surface undocumented in our pin. Not recommended. + +--- + +## Decisions + +1. **Scope mandate — RESOLVED 2026-06-30: A (ledger-mandate).** v1 "done" = every + frozen I/O point classified, zero unaccounted; NOT feature-complete client. + The ⬜-vs-🚫 line for the 11 formerly-pending rows is locked above (§ + Scope-resolution record). +2. **Pin remediation (P-1) — PENDING.** Re-pin to the frozen OpenAPI 2.2.0 + + SSE-schema JSON (recommended; drift-checkable via `canonical_drift.py`) vs + re-vendor markdown only (near-no-op). Adds vendored artifacts + a + canonical-sync pin → deferred to an explicit operator nod (substrate change + with CI-gating reach). diff --git a/persistent-memory.md b/persistent-memory.md index f3db45d..3cd4a0e 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -144,6 +144,9 @@ decision. Captures rationale that won't be obvious from code alone. - `[2026-06-30]` **Worldtree v1.0.0b1→b2 consumer adaptation: eager turn-launch statuses (`v0.18.3` `b2e4901`, `v0.18.4` `e4317f6`).** Worldtree #331 decoupled turn execution from the SSE connection → turn-launch failures now arrive EAGERLY as a status before any stream: 409 `agent_not_available` (pre-b1 a 200 + in-stream error event), 503 retryable. Mapped both in `stream_turn` to typed `SseConnectFailed` subclasses keyed on STATUS, parsing the `{detail:{error_code,message}}` envelope — POST-003 preserved (no synthetic event yielded), existing handlers still catch (the design fork vs yield-an-Error-event was decided by POST-003). **DEFERRED follow-ups** (tracked here; bundle with the v1 coverage-audit): (1) live-prove the 409/503 end-to-end on personal-b2 (now unblocked — personal on b2, my key works there); (2) full `conversation-api-spec.md` markdown re-vendor to the b2 era (ratatoskr vendors the markdown, not the OpenAPI JSON). - `[2026-06-30]` **Verify-against-the-real-spec-before-committing caught a real upstream gap.** Holding the v0.18.3 commit to verify against demo's OpenAPI surfaced that the FROZEN OpenAPI 2.1.0 didn't document the 409/503 the heads-up described (`agent_not_available` was in the ErrorCode enum, but NO 503/turn-launch code). worldtree-dev confirmed it was THEIR gap (#331 added the statuses without extending the #328 `openapi()` override), shipped the fix in **v1.0.0b2 / OpenAPI 2.2.0** (409/503 now enumerated, 503 code finalized as `not_ready`). "The consumer-oracle earning its keep." Lesson: a provider's prose heads-up can diverge from its frozen machine-readable spec — verify the actual spec before committing a consumer adaptation. - `[2026-06-30]` **regard is a DEAD AXIS in Worldtree's emitted affect (caught provider-side; worldtree-dev confirmed + escalated to Vuong).** Across all our affect snapshots, `valence[].regard` is EXACTLY 0.15 regardless of agent/end_user/interaction_count, while familiarity accumulates (vuong 0.18→0.69 over 14 turns). Root cause (worldtree-dev, code-grounded): 0.15 = `base_regard = agreeableness*0.3` (sindra A=0.5); regard's only human-writer `update_regard` early-returns unless an emotion is `about="other"`, but the Vili appraiser's `ViliResponse` schema has NO directedness axis (everything hardcoded `about="situation"`) — producer side lost in the #265 Vili rework; consumer machinery intact. NOT WAD; the fix (reintroduce other-directed classification) is an affect-model change touching every agent + a directedness-classification design call → worldtree-dev filing an issue to Vuong. [the consumer/provider thesis paying off again] +- `[2026-06-30]` **v1 coverage-audit kicked off; coverage ledger written (`docs/coverage-map.md`) — the first one.** Every Worldtree v1-FROZEN I/O point × ratatoskr status. Anchored on WT's frozen machine-readable artifacts (OpenAPI **2.2.0** `conversation-api-openapi.json` = 40 REST path-groups + SSE schema = 11 events + bifrost wire v0.6), NOT the stale vendored prose markdown. Result: **SSE 11/11 ✅; Bifrost provider planes 8/8 ✅ live-proven** (covers the full `bifrost.memory.MemoryDataStore` protocol = describe_store/get/get_many/search/upsert_many + delete_many, and affect emit/fetch; **`health` is extended-reference-store-only, NOT in the base protocol → correctly deferred, NOT a gap** — settles the prior "health" ambiguity); **client REST 7/40 ✅ live, 11 ⬜ in-scope, 22 🚫 excluded.** +- `[2026-06-30]` **Scope mandate A locked (operator): v1 "done" = every frozen I/O point CLASSIFIED (covered-or-excluded-with-rationale), zero unaccounted — NOT a feature-complete client.** The coverage map is a LEDGER, not a build-everything mandate. Reconciles the 2026-06-19 "consume all I/O" reframe with the 2026-05-20 design-brief's "NOT an admin tool" + deferral negative clauses (which predate both the provider identity and the reframe). Resolved the 11 design-brief-vs-reframe ❓ rows via the debug-observability test (*does a turn flow through it?*): 🚫 search / uploads / pending / embed / judgments (consumer-product + eval utilities); ⬜ transient-characters routing (4) + persona_state-write (Tier-2 frontier). **Frontier Tier 1 (all unblocked except #11):** session-picker + SSE-resume (wrappers `list_sessions`/`reconnect_turn` exist with NO caller — presenter-wiring only) → `GET /capabilities` + `GET /me` → BifrostState/Tools widgets (`GET /admin/sessions/{id}/{bifrost,tools}`, admin-key) → **#11 AdminEvents BLOCKED on `admin.events.read` scope**. The 3 admin-observability widgets + picker + resume were design-brief §5/§4/§8d v1 items that **were never built**. +- `[2026-06-30]` **Finding P-1 (pin drift) + pin-remediation PENDING.** We vendor the PROSE markdown (`docs/conversation-api-spec.md`), which is **byte-identical to live WT's** but frozen at v0.35.16-era content (last WT edit 2026-05-31) — it does NOT capture b2: 7 new endpoints (admin/keys/bulk, admin/persona/{archive,erase}, admin/usage, embed, judgments, me/usage), the 409/503 on messages-POST (#331), the unified error envelope (#328), or the SSE schema. **WT's authoritative v1 truth is now the FROZEN OpenAPI 2.2.0 + SSE-schema JSON** (`Worldtree/docs/v1-schema-freeze-manifest.md`). So the previously-deferred "re-vendor markdown to b2" is a **near-no-op** (markdown content identical). **Pending operator nod:** re-pin to the machine-readable artifacts (recommended — drift-checkable via `canonical_drift.py`, makes the coverage map reproducible vs a frozen diffable target) vs markdown-only. Deferred (not auto-applied) because it adds vendored artifacts + a canonical-sync pin = substrate change with CI-gating reach. _41 older entries (2026-05-* — the original debug-TUI/web build era) archived to archival-memory.md._