memory: snapshot — Bifrost consumer arc (affect live, memory contract v1.1)

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — ratatoskr
 
-_Last updated: 2026-05-29_
+_Last updated: 2026-06-15_
 
 This file captures durable intent and supporting evidence (goals, decisions,
 foot-gun warnings, in-flight state) across context resets. Read it at session
@@ -41,67 +41,53 @@ model output is untrusted); upstream API key stays server-side (INV-003).
 
 ## Current state / in-flight
 
-_As of 2026-05-29 (post-v0.17.0 frontend redesign):_
+_As of 2026-06-15:_
 
-**Status: v0.17.0 shipped and pushed** (main + tag `v0.17.0` on origin,
-2026-05-29). 378 tests passing.
+**Ratatoskr now has a SECOND identity: the v1 Bifrost Tier-3 consumer** — the
+durable persistence provider Worldtree writes Tier-3 agent affect/persona +
+memory into — alongside the original debug-observability TUI/web. The
+Bifrost-consumer work lives in `src/ratatoskr/provider/` and depends on
+`bifrost>=0.6.1` (a `provider` optional-extra from the gitea PyPI index),
+SEPARATE from the Worldtree conversation-API spec pin.
 
-**Last commits on `main`:**
-- `922ef34` feat(web): frontend redesign — aurora telemetry instrument + live Markdown (v0.17.0) [LOCAL ONLY]
-- `bbeaa23` docs: AGENTS.md — Codex-implementer session conventions
-- `f7ff5a4` fix(web): close Heid pass-2 findings — stream vocab + disconnect catch (v0.16.1)
-- `369857d` feat(web): address Heid code-review findings — issue #16 (v0.16.0)
-- `0fbbeb1` fix(sessions): unwrap FastAPI detail envelope in get_persona_state (v0.15.1)
-- `1228c37` feat(web): in-browser debug companion — issue #16 (v0.15.0)
-- `85143b8` fix(tui): disable RichLog min_width floor so wrap actually applies (v0.14.2)
-- `00854ce` fix(cli): wire AffectUpdate + AwaitingLlmFirstToken into --send presenter (v0.14.1)
-- `78bfcad` feat(sse,tui): bump spec pin to v0.29.0 + AwaitingLlmFirstToken (v0.14.0)
-- `4413859` feat(tui): persona surface — sticky header + TabPane (v0.13.0)
-- `d516537` feat(sessions): get_persona_state client + persona error taxonomy (v0.12.0)
-- `92aa05c` feat(sse,tui): bump spec pin to v0.28.0 + AffectUpdate event (v0.11.0)
-- `209427a` feat(tui): debug-pane audit logging surface (v0.10.0)
-- `139771c` feat(tui): live Markdown rendering during text streaming (v0.9.0)
+**AFFECT plane: SHIPPED + LIVE-PROVEN** (v0.17.2). Running now as a dev
+background shell (`ratatoskr-provider`, `0.0.0.0:8390`, env-sourced from
+`~/.config/ratatoskr/provider.env`). Smoked end-to-end against personal
+Worldtree **v0.35.2** (`10.250.50.152`): handshake 200 + `affect.emit` 200 →
+durable row persisted, opacity held.
 
-**Worldtree spec pin:** v0.29.0 (commit `562001a`, pinned 2026-05-26).
+**MEMORY plane: contract done, TDD next.**
+`docs/contracts/bifrost_memory_provider.contract.md` v1.1 (Heid-panel-reviewed,
+committed `1f94e5f`). NEXT (fresh session): add `sqlite-vec` to the `provider`
+extra → TDD (tracer `basic_upsert` → search/scope-isolation/optimistic-lock/
+conflict → #195 parity vs `InMemoryMemoryStore`) → `/heid-code-review` → ship a
+`ratatoskr-memory-provider` dev shell. Worldtree v0.35.3 already negotiates the
+basic memory plane — the memory server is the only missing piece.
 
-**Personal Worldtree smoke target:** `http://10.250.50.152:8081` (corviduo-dev
-LAN, reachable via PFI VPN). Currently running ≥ v0.29.13 (carries Worldtree
-#204 persona-state + #201 awaiting-llm-first-token heartbeat + the
-GemmaProvider reasoning_content fix). Sindra Tier 3 agent
-(`ratatoskr:sindra`, model `artemis-31b-v1i`) live and chain-of-thought
-flowing end-to-end.
+**Sindra:** a REGISTERED Tier-3 agent (`ratatoskr:sindra`, was model
+`artemis-31b-v1i`) — registration is REQUIRED to use a Tier-3 character (a
+session against an unregistered `agent_id` 404s), and she's been used. OPEN:
+the v0.35.2 personal rebuild may have wiped the agent DB — re-verify via
+`GET /agents` (needs a WORLDTREE_API_KEY, broker via infra-ops) and re-register
+if gone. Her persona only persists durably once personal is bound to our
+RUNNING affect provider for HER sessions (the smoke used synthetic
+`ratatoskr:smoke`); that binding is the persona-carry gap, independent of
+registration.
 
-**Outstanding eyeball items:**
-- **v0.17.0 frontend redesign — operator-confirmed 2026-05-29** ("redesign
-  looks good"); pushed to origin (main + tag `v0.17.0`). Eyeball gate cleared.
-- **Optional**: courtesy ack to brokkr-smithy-dev on the Codex-first pilot
-  thread (thread `01KSTH3Y8JKKSY9S9P41X3CM77`).
+**Heimdall key (Bifrost consumer):** persists env-only at
+`~/.config/ratatoskr/provider.env` (mode 600, nh3-dev) — `consumer="ratatoskr"`,
+HS256 = the API-key STRING utf-8-encoded; rotate via infra-ops.
 
-**Issue tracker (cleaned up 2026-05-29):** closed the stale-but-shipped
-backlog — #1–7, #9, #12, #14, #15 (all implemented across the v0→v0.17.0 arc;
-#9 superseded — spec pin advanced to v0.29.0, seven minors past its v0.22.1
-target). Only two issues remain open, both deferred by design:
-- **#10 subject:{type,id} migration tracking** — Worldtree-side breaking
-  change at future v0.22.x/v0.23.0; don't pre-implement per worldtree-dev.
-- **#11 AdminEvents pane** — needs `admin.events.read` Heimdall scope (admin
-  tier). Future side-pane work.
+**Unpushed:** `main` is 3 commits ahead of origin (origin at `d90a58d`/v0.17.1;
+local at `1f94e5f`) — `bcdcd71` (v0.17.2), `eebab46`, `1f94e5f` + the local
+`v0.17.2` tag are unpushed. Push is the operator's call.
 
-**Codex-first discipline pilot — Ratatoskr selected** (althing thread
-`01KSTH3Y8JKKSY9S9P41X3CM77`, brokkr-smithy commit `5dd061c`, tag `v0.5.3`):
-- `AGENTS.md` committed (bbeaa23) — Codex-implementer conventions, sibling
-  to `CLAUDE.md`. Doesn't change how Claude-Code sessions operate;
-  governs the future `ratatoskr-codex` session.
-- `ratatoskr-codex` handle declared on althing.
-- `/codex-dispatch` skill pending galdrabok implementation.
-- **No action needed until operator spins up a codex session** in this same
-  working tree; bootstrap handshake at that point per `codex-first-
-  discipline.md` §5 (codex sends `codex-online` → ratatoskr-dev replies with
-  active branches + WIP state).
-- Per-dispatch opt-in: default Sleipnir Claude-implementer path remains
-  available; Codex used only when operator routes via `/codex-dispatch <N>`.
+**Still standing from before:** Worldtree spec pin v0.29.0 (`562001a`) for the
+conversation-API/TUI surface (untouched by the Bifrost work). Codex-first pilot
+still dormant (no codex session spun up — see the 2026-05-29 decision). Open
+issues: #10 (subject migration, deferred), #11 (AdminEvents pane, deferred).
 
-Branch: `main` (clean post-v0.17.0). Remote:
-`origin → git@gitea.phasefinal.com:vh/ratatoskr.git`.
+Branch: `main`. Remote: `origin → git@gitea.phasefinal.com:vh/ratatoskr.git`.
 
 ## Recent decisions
 
@@ -149,6 +135,12 @@ decision. Captures rationale that won't be obvious from code alone.
 - `[2026-05-28]` **Sindra Tier 3 agent: FORM ASSUMPTION gate + new physical-form description.** Persistent agent state changes via `python -m ratatoskr.tier3 patch`: (1) model migrated from `qwen3.6-35-a3b-heretic` to `artemis-31b-v1i`; (2) added FORM ASSUMPTION section — when instructed to become another character she IS that character (identity/environment/psychology/parameters), believes the environment as fact, no Sindra/holo-deck/parameter references, sticky until explicit revert; (3) replaced the abstract "classically beautiful" default-form sketch with a specific anti-artifice physical description (5'8", golden-copper skin, asymmetric features, oversize dark-green knit, bare feet). System prompt file is at `/tmp/personal-worldtree-sindra_system_prompt.md` (transient; not committed to repo).
 - `[2026-05-29]` **v0.17.0 frontend redesign — aurora telemetry instrument.** `/frontend-design` pass on the web companion: all-monospace technical-instrument aesthetic with the Australis dark palette + aurora-borealis accent band. Top command bar with live connection dot (idle/streaming/error states), inline persona summary with P/A/D micro-bars, animated awaiting-token, terminal-event status chips. **Live Markdown rendering in transcript + thinking panes** via a hand-rolled `markdownSafe()` (escape-first, whitelist subset of headings/bold/italic/inline-code/fenced/lists/quote/links; link-scheme whitelist; XSS-verified under a node harness). Thinking pane now has per-turn labeled dividers + a fresh MD-rendered block per turn. **Tools / Debug / Persona panes stay literal monospace** by deliberate choice — they carry structured audit lines + JSON, where MD would corrupt readability (underscores in tool names, JSON braces). Single-file vanilla HTML/CSS/JS, no build, no CDN, no node_modules.
 - `[2026-05-29]` **Codex-first discipline pilot — Ratatoskr selected.** brokkr-smithy-dev pushed `AGENTS.md` (commit `bbeaa23`) and declared the `ratatoskr-codex` handle per `brokkr-smithy/docs/codex-first-discipline.md` v0.1 (brokkr-smithy commit `5dd061c`, tag `v0.5.3`). Per-dispatch opt-in model: default Sleipnir Claude-implementer path remains available; Codex used only when operator routes via `/codex-dispatch <N>`. Bootstrap handshake when operator spins up a codex session: codex sends `codex-online` → ratatoskr-dev replies with active branches + WIP state. Galdrabok was rejected as pilot (Codex authoring Claude skills is a category error); Skaldsong was the other candidate.
+- `[2026-06-14]` **Ratatoskr becomes the v1 Bifrost Tier-3 consumer.** A second identity beyond the debug TUI: the durable persistence Worldtree writes Tier-3 agent affect (persona) + memory into. Pin `bifrost>=0.6.1` in a `provider` optional-extra (gitea PyPI index, auth via `~/.netrc`; 0.6.0 was yanked for a circular import). Implement bifrost's OWN `MemoryDataStore`/affect Protocols (NOT worldtree-memory's); `describe_store` is SYNC; affect is conduit-opaque. New module `src/ratatoskr/provider/`. Authoritative how-to: `~/development/bifrost/docs/implementing-a-consumer.md`. (commits `1a73d77` pin, `d90a58d` affect store v0.17.1, `bcdcd71` serve entrypoint v0.17.2)
+- `[2026-06-14]` **Backend = SQLite + sqlite-vec; affect-first then memory; separate DB per plane** (operator-chosen). Affect = blind conduit (reads only `agent_id`+`end_user_id`); memory = structural index (reads vector/scope/id/origin to serve search). Conformance for both = #195 parity vs bifrost's `InMemory*Store` through the real `dispatch_*_call`.
+- `[2026-06-14]` **The affect contract's idempotency model was WRONG; real-lib TDD caught it.** First draft modeled same-idempotency-key-different-payload as an LWW overwrite; bifrost actually raises a CONFLICT (`AffectIdempotencyConflict`), actor-scoped. The artifact-only `/heid-contract-review` STRUCTURALLY cannot catch this class (it never sees bifrost's source) — TDD against the shipped library is the gate; the executable reference store + #195 parity are the backstop. Filed the guide §6 gap to bifrost-dev, who fixed it (bifrost `c0d0a11`).
+- `[2026-06-15]` **Memory v1 = the bifrost BASIC plane only** (search/get/upsert/delete + describe_store/health) per worldtree-dev re-scope (#294) — the only surface Tier-3's live path touches; gated verbs (edges/scan/atomic_supersede/mark/patch/maintenance) deferred + advertised-unsupported. Worldtree v0.35.3 already requests+maps it — no Worldtree-side blocker. Memory contract committed v1.0 (`eebab46`) → v1.1 Heid-reviewed (`1f94e5f`).
+- `[2026-06-15]` **Providers run as dev-box BACKGROUND SHELLS, not infra-ops/systemd** (operator call — it's a dev box). `ratatoskr-provider` (affect) + a future `ratatoskr-memory-provider` as background processes; no productionization track.
+- `[2026-06-15]` **Affect plane shipped (v0.17.2) + LIVE-PROVEN end-to-end against real Worldtree v0.35.2.** Personal handshake 200 + `affect.emit` 200 from `10.250.50.152` → durable row persisted (opacity held). HS256 key = the consumer's Heimdall API-key STRING utf-8-encoded (NOT base64/raw — the tripwire); cross-subnet route + `BIFROST_CLIENT_ALLOWED_HOSTS` allowlist all held (infra-ops-owned). worldtree-dev confirmed ADR-0009 holding as designed.
 
 _For per-issue TDD implementation notes, Volva findings, and contract amendments, see the git log — every per-issue commit carries a structured message capturing the trail._
 
@@ -175,3 +167,6 @@ defense against re-attempting the same cul-de-sac.
 - `[2026-05-27]` **`get_persona_state` mocked flat error envelope; real Worldtree wraps in `detail`.** v0.12.0 tests used `{"error_code": "auth_scope_denied"}` but real wire (FastAPI default) returns `{"detail": {"error_code": "auth_scope_denied", "message": "…"}}`. The parser only checked top-level so the typed exception was never raised; calls fell through to `SessionApiFailed(403)`, which the web persona endpoint surfaced as HTTP 500. v0.15.1 patches both shapes. **Lesson: test-side mock envelopes must match the REAL wire shape; live smoke is load-bearing for envelope-shape verification, not just happy paths.**
 - `[2026-05-27]` **Mid-session `system_prompt` mutation: universal omission across surveyed mature systems.** brokkr-smithy R13 panel (3-arm, strong convergence) confirmed: no surveyed system ships live PATCH-on-active-session (OpenAI Assistants/Responses, Anthropic Messages, Vertex AI, MCP, LangChain, LlamaIndex, Ollama, vLLM). The omission IS the answer. 12 additional threat vectors beyond ratatoskr's initial 7. **Don't re-propose this for ratatoskr;** if a future production conversational shell wants iterative-prompt-tuning ergonomics, the consensus shape is fork-via-client (PATCH agent → new session → replay context).
 - `[2026-05-28]` **Browser-local turn_id used for upstream cancel URL — old cancel tests ENCODED the bug.** Web companion v0.15.x cancel paths posted to `/sessions/{sid}/turns/{LOCAL_ID}/cancel`. Tests mocked the local-id URL so they encoded the bug rather than detecting it. Hulda caught it in Heid pass 1. Fix in v0.16.0: capture upstream_turn_id from the first SSE event's `sse_id.turn_id`; all cancel paths use it; cancel before first event is `{"cancelled": false, "reason": "not_started"}`. **Rule: when designing cancel/match paths against an external service, test fixtures must mock what would actually be hit upstream — mocking your own derived id encodes the bug instead of catching it.**
+- `[2026-06-15]` **"Sindra hasn't been registered" was an under-verified inference — WRONG.** Concluded it from grepping ratatoskr's CODE (`sindra` absent from `src/`), but Tier-3 registration is SERVER-SIDE (`POST /agents/define` on the Worldtree instance) — a code grep structurally can't see it. Registration IS required to use a Tier-3 character (a session against an unregistered `agent_id` 404s), so since Sindra has been used, she WAS registered (`ratatoskr:sindra`). **Rule: to check whether a Tier-3 agent exists, query the Worldtree instance's `GET /agents`, never the consumer repo's code.** (Residual: the v0.35.2 personal rebuild may have wiped her — re-verify.)
+- `[2026-06-14]` **Artifact-only contract review can't validate against a dependency's ACTUAL behavior.** `/heid-contract-review` sees only the contract, never the external library (bifrost) — so "the consumer under-built against bifrost's real semantics" is invisible to it by construction (the affect idempotency model shipped wrong because of this). Real-lib TDD against the shipped library + the executable reference store + the #195 parity test are the gate for any consumer plane with non-trivial state semantics. Don't treat a clean contract review as evidence the code matches the dependency.
+- `[2026-06-15]` **"byte-equal" round-trip slip propagated affect→memory via copy-paste.** The affect contract's byte-identical→semantic fix reappeared in the memory contract's INV-001 (sibling copy). Only an INDEPENDENT `/heid-contract-review` of the memory contract re-caught it. **Paraphrase every sibling contract fresh — don't amortize one review across a family; copies carry the parent's slips.** (also a feedback auto-memory)