From 5cdb69a6f341c2dc055e3d116ed7c08226c464b9 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 15 Jun 2026 01:45:11 -0700 Subject: [PATCH] =?UTF-8?q?memory:=20snapshot=20=E2=80=94=20Bifrost=20cons?= =?UTF-8?q?umer=20arc=20(affect=20live,=20memory=20contract=20v1.1)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- persistent-memory.md | 103 ++++++++++++++++++++----------------------- 1 file changed, 49 insertions(+), 54 deletions(-) diff --git a/persistent-memory.md b/persistent-memory.md index 533988c..9e491ad 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -1,6 +1,6 @@ # Persistent memory — ratatoskr -_Last updated: 2026-05-29_ +_Last updated: 2026-06-15_ This file captures durable intent and supporting evidence (goals, decisions, foot-gun warnings, in-flight state) across context resets. Read it at session @@ -41,67 +41,53 @@ model output is untrusted); upstream API key stays server-side (INV-003). ## Current state / in-flight -_As of 2026-05-29 (post-v0.17.0 frontend redesign):_ +_As of 2026-06-15:_ -**Status: v0.17.0 shipped and pushed** (main + tag `v0.17.0` on origin, -2026-05-29). 378 tests passing. +**Ratatoskr now has a SECOND identity: the v1 Bifrost Tier-3 consumer** — the +durable persistence provider Worldtree writes Tier-3 agent affect/persona + +memory into — alongside the original debug-observability TUI/web. The +Bifrost-consumer work lives in `src/ratatoskr/provider/` and depends on +`bifrost>=0.6.1` (a `provider` optional-extra from the gitea PyPI index), +SEPARATE from the Worldtree conversation-API spec pin. -**Last commits on `main`:** -- `922ef34` feat(web): frontend redesign — aurora telemetry instrument + live Markdown (v0.17.0) [LOCAL ONLY] -- `bbeaa23` docs: AGENTS.md — Codex-implementer session conventions -- `f7ff5a4` fix(web): close Heid pass-2 findings — stream vocab + disconnect catch (v0.16.1) -- `369857d` feat(web): address Heid code-review findings — issue #16 (v0.16.0) -- `0fbbeb1` fix(sessions): unwrap FastAPI detail envelope in get_persona_state (v0.15.1) -- `1228c37` feat(web): in-browser debug companion — issue #16 (v0.15.0) -- `85143b8` fix(tui): disable RichLog min_width floor so wrap actually applies (v0.14.2) -- `00854ce` fix(cli): wire AffectUpdate + AwaitingLlmFirstToken into --send presenter (v0.14.1) -- `78bfcad` feat(sse,tui): bump spec pin to v0.29.0 + AwaitingLlmFirstToken (v0.14.0) -- `4413859` feat(tui): persona surface — sticky header + TabPane (v0.13.0) -- `d516537` feat(sessions): get_persona_state client + persona error taxonomy (v0.12.0) -- `92aa05c` feat(sse,tui): bump spec pin to v0.28.0 + AffectUpdate event (v0.11.0) -- `209427a` feat(tui): debug-pane audit logging surface (v0.10.0) -- `139771c` feat(tui): live Markdown rendering during text streaming (v0.9.0) +**AFFECT plane: SHIPPED + LIVE-PROVEN** (v0.17.2). Running now as a dev +background shell (`ratatoskr-provider`, `0.0.0.0:8390`, env-sourced from +`~/.config/ratatoskr/provider.env`). Smoked end-to-end against personal +Worldtree **v0.35.2** (`10.250.50.152`): handshake 200 + `affect.emit` 200 → +durable row persisted, opacity held. -**Worldtree spec pin:** v0.29.0 (commit `562001a`, pinned 2026-05-26). +**MEMORY plane: contract done, TDD next.** +`docs/contracts/bifrost_memory_provider.contract.md` v1.1 (Heid-panel-reviewed, +committed `1f94e5f`). NEXT (fresh session): add `sqlite-vec` to the `provider` +extra → TDD (tracer `basic_upsert` → search/scope-isolation/optimistic-lock/ +conflict → #195 parity vs `InMemoryMemoryStore`) → `/heid-code-review` → ship a +`ratatoskr-memory-provider` dev shell. Worldtree v0.35.3 already negotiates the +basic memory plane — the memory server is the only missing piece. -**Personal Worldtree smoke target:** `http://10.250.50.152:8081` (corviduo-dev -LAN, reachable via PFI VPN). Currently running ≥ v0.29.13 (carries Worldtree -#204 persona-state + #201 awaiting-llm-first-token heartbeat + the -GemmaProvider reasoning_content fix). Sindra Tier 3 agent -(`ratatoskr:sindra`, model `artemis-31b-v1i`) live and chain-of-thought -flowing end-to-end. +**Sindra:** a REGISTERED Tier-3 agent (`ratatoskr:sindra`, was model +`artemis-31b-v1i`) — registration is REQUIRED to use a Tier-3 character (a +session against an unregistered `agent_id` 404s), and she's been used. OPEN: +the v0.35.2 personal rebuild may have wiped the agent DB — re-verify via +`GET /agents` (needs a WORLDTREE_API_KEY, broker via infra-ops) and re-register +if gone. Her persona only persists durably once personal is bound to our +RUNNING affect provider for HER sessions (the smoke used synthetic +`ratatoskr:smoke`); that binding is the persona-carry gap, independent of +registration. -**Outstanding eyeball items:** -- **v0.17.0 frontend redesign — operator-confirmed 2026-05-29** ("redesign - looks good"); pushed to origin (main + tag `v0.17.0`). Eyeball gate cleared. -- **Optional**: courtesy ack to brokkr-smithy-dev on the Codex-first pilot - thread (thread `01KSTH3Y8JKKSY9S9P41X3CM77`). +**Heimdall key (Bifrost consumer):** persists env-only at +`~/.config/ratatoskr/provider.env` (mode 600, nh3-dev) — `consumer="ratatoskr"`, +HS256 = the API-key STRING utf-8-encoded; rotate via infra-ops. -**Issue tracker (cleaned up 2026-05-29):** closed the stale-but-shipped -backlog — #1–7, #9, #12, #14, #15 (all implemented across the v0→v0.17.0 arc; -#9 superseded — spec pin advanced to v0.29.0, seven minors past its v0.22.1 -target). Only two issues remain open, both deferred by design: -- **#10 subject:{type,id} migration tracking** — Worldtree-side breaking - change at future v0.22.x/v0.23.0; don't pre-implement per worldtree-dev. -- **#11 AdminEvents pane** — needs `admin.events.read` Heimdall scope (admin - tier). Future side-pane work. +**Unpushed:** `main` is 3 commits ahead of origin (origin at `d90a58d`/v0.17.1; +local at `1f94e5f`) — `bcdcd71` (v0.17.2), `eebab46`, `1f94e5f` + the local +`v0.17.2` tag are unpushed. Push is the operator's call. -**Codex-first discipline pilot — Ratatoskr selected** (althing thread -`01KSTH3Y8JKKSY9S9P41X3CM77`, brokkr-smithy commit `5dd061c`, tag `v0.5.3`): -- `AGENTS.md` committed (bbeaa23) — Codex-implementer conventions, sibling - to `CLAUDE.md`. Doesn't change how Claude-Code sessions operate; - governs the future `ratatoskr-codex` session. -- `ratatoskr-codex` handle declared on althing. -- `/codex-dispatch` skill pending galdrabok implementation. -- **No action needed until operator spins up a codex session** in this same - working tree; bootstrap handshake at that point per `codex-first- - discipline.md` §5 (codex sends `codex-online` → ratatoskr-dev replies with - active branches + WIP state). -- Per-dispatch opt-in: default Sleipnir Claude-implementer path remains - available; Codex used only when operator routes via `/codex-dispatch `. +**Still standing from before:** Worldtree spec pin v0.29.0 (`562001a`) for the +conversation-API/TUI surface (untouched by the Bifrost work). Codex-first pilot +still dormant (no codex session spun up — see the 2026-05-29 decision). Open +issues: #10 (subject migration, deferred), #11 (AdminEvents pane, deferred). -Branch: `main` (clean post-v0.17.0). Remote: -`origin → git@gitea.phasefinal.com:vh/ratatoskr.git`. +Branch: `main`. Remote: `origin → git@gitea.phasefinal.com:vh/ratatoskr.git`. ## Recent decisions @@ -149,6 +135,12 @@ decision. Captures rationale that won't be obvious from code alone. - `[2026-05-28]` **Sindra Tier 3 agent: FORM ASSUMPTION gate + new physical-form description.** Persistent agent state changes via `python -m ratatoskr.tier3 patch`: (1) model migrated from `qwen3.6-35-a3b-heretic` to `artemis-31b-v1i`; (2) added FORM ASSUMPTION section — when instructed to become another character she IS that character (identity/environment/psychology/parameters), believes the environment as fact, no Sindra/holo-deck/parameter references, sticky until explicit revert; (3) replaced the abstract "classically beautiful" default-form sketch with a specific anti-artifice physical description (5'8", golden-copper skin, asymmetric features, oversize dark-green knit, bare feet). System prompt file is at `/tmp/personal-worldtree-sindra_system_prompt.md` (transient; not committed to repo). - `[2026-05-29]` **v0.17.0 frontend redesign — aurora telemetry instrument.** `/frontend-design` pass on the web companion: all-monospace technical-instrument aesthetic with the Australis dark palette + aurora-borealis accent band. Top command bar with live connection dot (idle/streaming/error states), inline persona summary with P/A/D micro-bars, animated awaiting-token, terminal-event status chips. **Live Markdown rendering in transcript + thinking panes** via a hand-rolled `markdownSafe()` (escape-first, whitelist subset of headings/bold/italic/inline-code/fenced/lists/quote/links; link-scheme whitelist; XSS-verified under a node harness). Thinking pane now has per-turn labeled dividers + a fresh MD-rendered block per turn. **Tools / Debug / Persona panes stay literal monospace** by deliberate choice — they carry structured audit lines + JSON, where MD would corrupt readability (underscores in tool names, JSON braces). Single-file vanilla HTML/CSS/JS, no build, no CDN, no node_modules. - `[2026-05-29]` **Codex-first discipline pilot — Ratatoskr selected.** brokkr-smithy-dev pushed `AGENTS.md` (commit `bbeaa23`) and declared the `ratatoskr-codex` handle per `brokkr-smithy/docs/codex-first-discipline.md` v0.1 (brokkr-smithy commit `5dd061c`, tag `v0.5.3`). Per-dispatch opt-in model: default Sleipnir Claude-implementer path remains available; Codex used only when operator routes via `/codex-dispatch `. Bootstrap handshake when operator spins up a codex session: codex sends `codex-online` → ratatoskr-dev replies with active branches + WIP state. Galdrabok was rejected as pilot (Codex authoring Claude skills is a category error); Skaldsong was the other candidate. +- `[2026-06-14]` **Ratatoskr becomes the v1 Bifrost Tier-3 consumer.** A second identity beyond the debug TUI: the durable persistence Worldtree writes Tier-3 agent affect (persona) + memory into. Pin `bifrost>=0.6.1` in a `provider` optional-extra (gitea PyPI index, auth via `~/.netrc`; 0.6.0 was yanked for a circular import). Implement bifrost's OWN `MemoryDataStore`/affect Protocols (NOT worldtree-memory's); `describe_store` is SYNC; affect is conduit-opaque. New module `src/ratatoskr/provider/`. Authoritative how-to: `~/development/bifrost/docs/implementing-a-consumer.md`. (commits `1a73d77` pin, `d90a58d` affect store v0.17.1, `bcdcd71` serve entrypoint v0.17.2) +- `[2026-06-14]` **Backend = SQLite + sqlite-vec; affect-first then memory; separate DB per plane** (operator-chosen). Affect = blind conduit (reads only `agent_id`+`end_user_id`); memory = structural index (reads vector/scope/id/origin to serve search). Conformance for both = #195 parity vs bifrost's `InMemory*Store` through the real `dispatch_*_call`. +- `[2026-06-14]` **The affect contract's idempotency model was WRONG; real-lib TDD caught it.** First draft modeled same-idempotency-key-different-payload as an LWW overwrite; bifrost actually raises a CONFLICT (`AffectIdempotencyConflict`), actor-scoped. The artifact-only `/heid-contract-review` STRUCTURALLY cannot catch this class (it never sees bifrost's source) — TDD against the shipped library is the gate; the executable reference store + #195 parity are the backstop. Filed the guide §6 gap to bifrost-dev, who fixed it (bifrost `c0d0a11`). +- `[2026-06-15]` **Memory v1 = the bifrost BASIC plane only** (search/get/upsert/delete + describe_store/health) per worldtree-dev re-scope (#294) — the only surface Tier-3's live path touches; gated verbs (edges/scan/atomic_supersede/mark/patch/maintenance) deferred + advertised-unsupported. Worldtree v0.35.3 already requests+maps it — no Worldtree-side blocker. Memory contract committed v1.0 (`eebab46`) → v1.1 Heid-reviewed (`1f94e5f`). +- `[2026-06-15]` **Providers run as dev-box BACKGROUND SHELLS, not infra-ops/systemd** (operator call — it's a dev box). `ratatoskr-provider` (affect) + a future `ratatoskr-memory-provider` as background processes; no productionization track. +- `[2026-06-15]` **Affect plane shipped (v0.17.2) + LIVE-PROVEN end-to-end against real Worldtree v0.35.2.** Personal handshake 200 + `affect.emit` 200 from `10.250.50.152` → durable row persisted (opacity held). HS256 key = the consumer's Heimdall API-key STRING utf-8-encoded (NOT base64/raw — the tripwire); cross-subnet route + `BIFROST_CLIENT_ALLOWED_HOSTS` allowlist all held (infra-ops-owned). worldtree-dev confirmed ADR-0009 holding as designed. _For per-issue TDD implementation notes, Volva findings, and contract amendments, see the git log — every per-issue commit carries a structured message capturing the trail._ @@ -175,3 +167,6 @@ defense against re-attempting the same cul-de-sac. - `[2026-05-27]` **`get_persona_state` mocked flat error envelope; real Worldtree wraps in `detail`.** v0.12.0 tests used `{"error_code": "auth_scope_denied"}` but real wire (FastAPI default) returns `{"detail": {"error_code": "auth_scope_denied", "message": "…"}}`. The parser only checked top-level so the typed exception was never raised; calls fell through to `SessionApiFailed(403)`, which the web persona endpoint surfaced as HTTP 500. v0.15.1 patches both shapes. **Lesson: test-side mock envelopes must match the REAL wire shape; live smoke is load-bearing for envelope-shape verification, not just happy paths.** - `[2026-05-27]` **Mid-session `system_prompt` mutation: universal omission across surveyed mature systems.** brokkr-smithy R13 panel (3-arm, strong convergence) confirmed: no surveyed system ships live PATCH-on-active-session (OpenAI Assistants/Responses, Anthropic Messages, Vertex AI, MCP, LangChain, LlamaIndex, Ollama, vLLM). The omission IS the answer. 12 additional threat vectors beyond ratatoskr's initial 7. **Don't re-propose this for ratatoskr;** if a future production conversational shell wants iterative-prompt-tuning ergonomics, the consensus shape is fork-via-client (PATCH agent → new session → replay context). - `[2026-05-28]` **Browser-local turn_id used for upstream cancel URL — old cancel tests ENCODED the bug.** Web companion v0.15.x cancel paths posted to `/sessions/{sid}/turns/{LOCAL_ID}/cancel`. Tests mocked the local-id URL so they encoded the bug rather than detecting it. Hulda caught it in Heid pass 1. Fix in v0.16.0: capture upstream_turn_id from the first SSE event's `sse_id.turn_id`; all cancel paths use it; cancel before first event is `{"cancelled": false, "reason": "not_started"}`. **Rule: when designing cancel/match paths against an external service, test fixtures must mock what would actually be hit upstream — mocking your own derived id encodes the bug instead of catching it.** +- `[2026-06-15]` **"Sindra hasn't been registered" was an under-verified inference — WRONG.** Concluded it from grepping ratatoskr's CODE (`sindra` absent from `src/`), but Tier-3 registration is SERVER-SIDE (`POST /agents/define` on the Worldtree instance) — a code grep structurally can't see it. Registration IS required to use a Tier-3 character (a session against an unregistered `agent_id` 404s), so since Sindra has been used, she WAS registered (`ratatoskr:sindra`). **Rule: to check whether a Tier-3 agent exists, query the Worldtree instance's `GET /agents`, never the consumer repo's code.** (Residual: the v0.35.2 personal rebuild may have wiped her — re-verify.) +- `[2026-06-14]` **Artifact-only contract review can't validate against a dependency's ACTUAL behavior.** `/heid-contract-review` sees only the contract, never the external library (bifrost) — so "the consumer under-built against bifrost's real semantics" is invisible to it by construction (the affect idempotency model shipped wrong because of this). Real-lib TDD against the shipped library + the executable reference store + the #195 parity test are the gate for any consumer plane with non-trivial state semantics. Don't treat a clean contract review as evidence the code matches the dependency. +- `[2026-06-15]` **"byte-equal" round-trip slip propagated affect→memory via copy-paste.** The affect contract's byte-identical→semantic fix reappeared in the memory contract's INV-001 (sibling copy). Only an INDEPENDENT `/heid-contract-review` of the memory contract re-caught it. **Paraphrase every sibling contract fresh — don't amortize one review across a family; copies carry the parent's slips.** (also a feedback auto-memory)