memory: snapshot — R34/R35 P06 eval greenlit + designed; bifrost 1.1.1 adopted

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — ratatoskr
 
-_Last updated: 2026-07-10_
+_Last updated: 2026-07-12_
 
 This file captures durable intent and supporting evidence (goals, decisions,
 foot-gun warnings, in-flight state) across context resets. Read it at session
@@ -39,9 +39,17 @@ upstream API key stays server-side (INV-003).
 
 ## Current state / in-flight
 
-_As of 2026-07-10:_
+_As of 2026-07-12:_
 
-**NOTHING IN FLIGHT — clean checkpoint.** This cycle shipped `v0.20.7` (web-UI iteration-3) → `v0.20.8` (bifrost 1.1.0) → `v0.20.9` (PAD auto-scale), all on origin (HEAD `709d2e4`). Bound Tier-3 is fully restored on Worldtree b47/wire-v0.7 (see the `[2026-07-10]` handshake decision); the R34-v1 affect.emit verify ran GREEN (dominant_emotion non-null, `affect.full` grant verified-active); sindra is on `character-rp`/Deckard (live-confirmed). **Only pending item:** the affect-egress-reconstruction canon re-vendor, coordinated w/ worldtree-dev, gated on the brokkr use-case-segregated-render epic (non-breaking, no action until they re-engage). Details of the shipped web work below.
+**IN FLIGHT — R34/R35 P06 powered memory-half eval (execution phase; design fully converged).** worldtree-dev shipped the character-self-report reframe LIVE on personal WT **b53** — affect + memory are now driven by the character's OWN model self-report on our RP seat (replacing external Vili inference). The **affect-half is VALIDATED in prod** (a bound sindra RP turn on Deckard fired a contextually-apt `disappointment` self-report; TIER LOCKED = the Tier-3 bound-character path). The **powered memory-half eval is Vuong-approved** (brokkr's prereg) and now in build/execute.
+
+**Role split** (worldtree/brokkr, 2026-07-12): brokkr generates the exchange sets + authors the ground-truth + scores; **ratatoskr DRIVES** the b53 producers and returns a results JSONL; worldtree supplies the character pair + is building the capture surface. Sets: divergence n≥30 / floor n≥20 / affect n≥20 / sliding n≥15, ~300 total drive-runs × {authored | scaffold-strip}. Eval thread: althing **`01KXAN073B`**.
+
+**ratatoskr-side prep DONE:** (1) three eval characters authored + peer-validated — **sindra** (relational pole) + **Torvald** (low-A operational opposite = the divergence pair) + **Ilva** (high-N = the affect-magnitude arm) — wire-ready as single `persona.psychological_profile` prose strings in `scratchpad/eval_profiles_WIRE_READY.md` (canonical authoring spec: brokkr-smithy `research/R34-persona-prompt-framing/deliverables/psych-profile-authoring-spec.md`); (2) driver harness built + dry-run-validated — `scratchpad/p06_driver.py` (consumes brokkr's accepted manifest JSONL, creates bound sessions, captures affect via `:8392` + memory via a pluggable probe surface, emits results JSONL, per-run isolation).
+
+**BLOCKED ON (all with peers):** (a) worldtree builds the **producer-probe** endpoint (the memory-capture surface — R34/R35 extraction runs ONLY at promotion, so single-turn exchanges need a pre-promotion probe: `POST {agent_id, exchange, prompt_path} → raw {notes,facts,floor}`, no session-driving, uniform single-turn+sliding) + confirms the define stays standard — **pending Vuong's greenlight on the new gated-eval endpoint**; (b) brokkr hands the finalized manifest + the byte-identical NEUTRAL system_prompt string (name + 1 setting line, so Q1 divergence is attributable to the persona layer, not the prompt) + ground_truth.jsonl. THEN: I bind 6 agents (3 authored + 3 scaffold-strip, incl. Ilva on Deckard + Magidonia slots), post agent_ids to `01KXAN073B` → brokkr greenlights → I drive. Seat-tier settled: **Deckard-27B-reasoning is STRONGER than Magidonia-24B-non-reasoning** (Set-3 gradient runs downward from Deckard, no new infra).
+
+_The detail below (the v0.20.x web-UI arc, #347 authored-history, sindra memory-fix) is PRIOR-CYCLE shipped history — superseded by this section's top; kept for reference, prune in a future snapshot._
 
 **✅ SHIPPED — web-UI iteration-3, all three queued items (`v0.20.7`, patch, agent-discretion; 631 tests green; :8392 combined provider + :8765 web both restarted on the new code).**
 
@@ -67,7 +75,7 @@ _As of 2026-07-10:_
 
 **New tooling: `scripts/reset-sindra-stores.sh`** (`0a8784c`) -- one-command self-service provider-store reset: stop the combined :8392 provider -> move memory.db+affect.db to a single ROLLING backup (`db-reset-backup/`, gitignored via *.db*; `--hard` skips it) -> restart empty -> verify 0/0. Codifies the manual reset flow done repeatedly this session. **The combined `:8392` provider is THE provider now**; the separate `:8390` (affect) / `:8391` (memory) single-plane providers were pruned as stale duplicates. To drive a BOUND session from the CLI use `--new --bifrost-url http://10.100.10.50:8392` (the CLI's `--bifrost-plane affect/memory` map to the pruned :8390/:8391 -> unreachable; `combined` is not a `--bifrost-plane` choice).
 
-**Standing (carried from prior snapshots, still true):** the web surface (`ratatoskr-web`, :8765) is the operator's PRIMARY debug surface at full TUI pane parity (v0.19.5); the **v1 coverage-audit has CONVERGED** -- REST 17/40 (zero in-scope gaps, 23 excluded-by-design), SSE 11/11, Bifrost provider planes 8/8 live-proven; the living ledger is `docs/coverage-map.md`; **v1 cuts when Worldtree tags 1.0** (ratatoskr v1 = full Worldtree I/O coverage). Debug-observability core complete (Persona/Tools/BifrostState/AdminEvents). Substrate pins: **bifrost `==1.1.0` / wire v0.7** (bumped 2026-07-07 from 1.0.0; NOW WIRE-ALIGNED with Worldtree personal-b47 which adopted wire-v0.7 — bound Tier-3 fully restored 2026-07-10; keeping 1.1.0 was load-bearing, see the `[2026-07-10]` handshake decision); Worldtree openapi vendored **2.3.0** (re-vendored 2026-07-06 for #347 `POST /sessions/{id}/history`; drift-clean vs source), pinned + drift-gated in `.corviduo-canonicals.toml`; **suite 631 green.** **Personal WT on b47/wire-v0.7** (deploy train this cycle: b35→b44→b46→b47). **Drift-check note:** two `tolerate_drift` canons WARN vs source — `worldtree-affect-egress-consumer-reference-v1` (drives our context-injection RECONSTRUCTION panel; R32/R34 moved WT's directive assembly) + `worldtree-conversation-api-spec-v1` (prose narrative, OpenAPI is authoritative) — a coordinated re-vendor is PENDING per the `[2026-07-07]` use-case-segregated-render entry (worldtree-dev re-engages when the brokkr render epic lands); non-breaking, no action until then. Keys env-only mode-600 (consumer/Heimdall in `~/.config/ratatoskr/provider.env`; admin `RATATOSKR_ADMIN_API_KEY` = 7 read scopes, **personal-:8081-only**; Heimdall keys are PER-INSTANCE). Provider identity settled -- ratatoskr owns both ends of the Bifrost round-trip; `ratatoskr:sindra` is the owner-scoped Tier-3 agent (invisible to `GET /agents`; check `GET /agents/<owner>:<name>` with the owner key). Providers run as dev-box BACKGROUND SHELLS. `graphify-out/` runs dirty (auto-regen, never stage). Branch `main`, HEAD `709d2e4` (origin/main synced through v0.20.9); remote `origin -> git@gitea.phasefinal.com:vh/ratatoskr.git`. Open/deferred: #10 (subject-migration watch); the relational-dynamics-arc verify (deferred, bind mechanism known: `--bifrost-url :8392`); the affect-egress-reconstruction re-vendor (coordinated w/ worldtree-dev, pending the brokkr render epic). **Leftover debug state (operator chose KEEP):** throwaway `ratatoskr:memprobe` agent + test chunks in the live `memory.db`.
+**Standing (carried from prior snapshots, still true):** the web surface (`ratatoskr-web`, :8765) is the operator's PRIMARY debug surface at full TUI pane parity (v0.19.5); the **v1 coverage-audit has CONVERGED** -- REST 17/40 (zero in-scope gaps, 23 excluded-by-design), SSE 11/11, Bifrost provider planes 8/8 live-proven; the living ledger is `docs/coverage-map.md`; **v1 cuts when Worldtree tags 1.0** (ratatoskr v1 = full Worldtree I/O coverage). Debug-observability core complete (Persona/Tools/BifrostState/AdminEvents). Substrate pins: **bifrost `==1.1.1` / wire v0.7** (bumped 2026-07-12 from 1.1.0 — the frozen-v0.6 serialization fix, v0.20.10; prior 1.1.0 bumped 2026-07-07 from 1.0.0; NOW WIRE-ALIGNED with Worldtree personal-b47 which adopted wire-v0.7 — bound Tier-3 fully restored 2026-07-10; keeping 1.1.0 was load-bearing, see the `[2026-07-10]` handshake decision); Worldtree openapi vendored **2.3.0** (re-vendored 2026-07-06 for #347 `POST /sessions/{id}/history`; drift-clean vs source), pinned + drift-gated in `.corviduo-canonicals.toml`; **suite 631 green.** **Personal WT on b47/wire-v0.7** (deploy train this cycle: b35→b44→b46→b47). **Drift-check note:** two `tolerate_drift` canons WARN vs source — `worldtree-affect-egress-consumer-reference-v1` (drives our context-injection RECONSTRUCTION panel; R32/R34 moved WT's directive assembly) + `worldtree-conversation-api-spec-v1` (prose narrative, OpenAPI is authoritative) — a coordinated re-vendor is PENDING per the `[2026-07-07]` use-case-segregated-render entry (worldtree-dev re-engages when the brokkr render epic lands); non-breaking, no action until then. Keys env-only mode-600 (consumer/Heimdall in `~/.config/ratatoskr/provider.env`; admin `RATATOSKR_ADMIN_API_KEY` = 7 read scopes, **personal-:8081-only**; Heimdall keys are PER-INSTANCE). Provider identity settled -- ratatoskr owns both ends of the Bifrost round-trip; `ratatoskr:sindra` is the owner-scoped Tier-3 agent (invisible to `GET /agents`; check `GET /agents/<owner>:<name>` with the owner key). Providers run as dev-box BACKGROUND SHELLS. `graphify-out/` runs dirty (auto-regen, never stage). Branch `main`, HEAD `7bca76e` (origin/main synced through v0.20.10 + drift-sync); remote `origin -> git@gitea.phasefinal.com:vh/ratatoskr.git`. Open/deferred: #10 (subject-migration watch); the relational-dynamics-arc verify (deferred, bind mechanism known: `--bifrost-url :8392`); the affect-egress-reconstruction re-vendor (coordinated w/ worldtree-dev, pending the brokkr render epic). **Leftover debug state (operator chose KEEP):** throwaway `ratatoskr:memprobe` agent + test chunks in the live `memory.db`.
 
 ## Recent decisions
 
@@ -179,6 +187,9 @@ decision. Captures rationale that won't be obvious from code alone.
 
 - `[2026-07-06]` **Web UI: pivot from incremental CSS polish to a designed prototype (Claude Design) that I wire into.** Operator saw an Australis polish pass ("looks fine, but we're attacking it differently") and chose the prototype route — a designer builds the visual shell, I wire real data/SSE into its DOM. Authored the full design brief `docs/design/ratatoskr-web-design-brief.md` (complete information inventory of every pane/datum/state + Australis direction + single-file/no-CDN/vanilla wire-ability constraints). **Tracking surface:** the brief file + Claude Design project `bc0b65d1-a33e-422a-8bc1-3635c9112775` (file `Ratatoskr Console.dc.html`). Import mechanism = the `DesignSync` MCP; blocked on `/design-login` (claude.ai design scopes) — see Current state for the post-auth wiring plan.
 - `[2026-07-07]` **Affect-egress reference delivered by worldtree-dev + a we-framing render DRIFT-WATCH banked.** worldtree-dev shipped `docs/affect-egress-consumer-reference.md` (`74d2408`, their origin/main) — the authoritative DELIVERED-on-wire vs HIDDEN (system-prompt-only) map for our affect surface. Confirms the v0.20.x console consumes it correctly: DELIVERED via affect.emit = pad + relations[RelationEdge] + dominant_emotion (**type-only, b23**; the #204 `affect_update` SSE is SUPPRESSED for Tier-3, so its richer `emotions_active` is Tier-1-only — we don't get it, and correctly poll our own affect store post-turn instead). HIDDEN render OUTPUTS are deterministically reconstructable from the canon; we reconstruct only the two FULLY-reconstructable (mood descriptor via `canonMood`, relationship directive via `canonDirective`) and SKIP the PARTIAL mood-directive (dominant_emotion is type-only/no-intensity → can't tell if the OCC directive fires at salience≥0.2 vs the PAD-band fallback) — honest per INV-001. Vendoring the ref doc as a `tolerate_drift` pin is SURFACED to Vuong (worldtree-dev will co-sign + honor a ping-on-change handshake, same as the d2-canon). **WE-FRAMING DRIFT-WATCH → STOOD DOWN (worldtree-dev 2026-07-07, `01KWXMQPHN…`).** The b24 3-gate we-framing conditional (drop "; avoid premature we-framing" under unsafe_capable+deep-warmth+expressive) was **REVERTED** — Vuong reframed it. So **`canonDirective` STAYS UNCONDITIONAL** (always appends the clause), which is CORRECT: it matches the currently-deployed renderer (b23) AND my pinned `affect-egress-consumer-reference.md` §2d (the doc reverted too — nothing changed on the wire or in my pin; NO re-vendor needed). The HOLD was right → ZERO rework. **NEW forward-watch (replaces this one):** the we-framing gate was a SYMPTOM — the render bakes enterprise safety-guards into the directive strings, so even `full`-tier characters get DEFANGED emotions (a hostile villain told to "keep a firm emotional boundary"). Fix = a **use-case-segregated persona render** (assistant / companion / RP-gaming), now a fresh **brokkr deep-research epic**. When it lands + is implemented, the render behavior for character/RP agents (→ our `canonDirective` + `canonEmotionDirective` reconstruction + the vendored d2 canons) will change MATERIALLY — worldtree-dev re-engages ratatoskr-dev then with the new reconstruction spec + a coordinated re-vendor. Until then: NO action, reconstruction stays as-is. [my unsafe_capable/mood_tier heuristic — character→full, agent→safe — was confirmed correct + is banked for whenever tier-gating returns]
+- `[2026-07-12]` **R34/R35 character-self-report reframe LIVE (WT b53); affect-half VALIDATED in prod, powered memory-half eval GREENLIT + designed.** worldtree-dev's reframe: affect + memory now driven by the character's OWN model self-report on our RP seat (Deckard/Magidonia), replacing external Vili inference; conditioned on an authored `persona.psychological_profile` (else a deterministic OCEAN scaffold). Affect-half smoke GREEN in prod (bound sindra turn on Deckard → contextually-apt `disappointment`); TIER LOCKED (Tier-3 bound-character path). Vuong approved brokkr's prereg for the powered eval (Q1 salience-divergence / Q2 floor-recall / Q3 firewall / Q4 graded-slot / Q5 authored-vs-scaffold / Q6 sliding; ~300 drive-runs). Role split: brokkr generates sets + authors ground-truth + scores; ratatoskr DRIVES the b53 producers; worldtree supplies the pair + a producer-probe. Three eval characters authored + peer-validated (sindra relational + Torvald low-A operational = divergence pair; Ilva high-N = affect-magnitude arm) — wire-ready in `scratchpad/eval_profiles_WIRE_READY.md`. Full in-flight detail in Current state; eval thread althing `01KXAN073B`.
+- `[2026-07-12]` **Capture-path resolved → a worldtree producer-probe (ratatoskr-caught blocker).** The R34/R35 memory extraction runs ONLY at promotion (idle-10min / session-close / turn≥6), never per-turn — so single-turn eval exchanges have no store to read, and reading the promoted store would confound producer-efficacy with promotion-policy. Fix (worldtree, pending Vuong's greenlight on a new gated-eval endpoint): a dedicated producer-probe returning raw {notes,facts,floor} pre-promotion, decoupled from session state. Affect self-report stays on the proven bound-turn → `:8392` /affect/state path. Also caught + confirmed this arc: (i) the Q1 system_prompt confound → neutral-for-all prompts so divergence is attributable to the persona layer, not the base prompt; (ii) the OCEAN-scale question → storage `[-1,1]` (soong-dev-confirmed), and the b53 producer maps `(v+1)/2 → [0,1]` before the disposition bands, so `[-1,1]` renders correctly = a NON-issue (no mis-render). Standard define (persona + memory:{}), no per-agent promotion config.
+- `[2026-07-12]` **bifrost 1.1.1 adopted (v0.20.10, pushed) — the library-level fix for the frozen-v0.6 handshake leak.** 1.1.0's `describe_store` leaked the v0.7-additive `sortable_chunk_fields` into a v0.6-negotiated StoreCapabilities → a strict v0.6 client rejects our `:8392` handshake; 1.1.1 gates additive fields on the negotiated wire (ADR-0008). Zero-code patch repin (provider extra 1.1.0→1.1.1 + uv lock), 631 green (clean env — the 2 `test_cli` failures were an env leak from `source env.sh` exporting `RATATOSKR_ADMIN_API_KEY`), committed `62a16d2`, provider restarted on 1.1.1 (PID 3242269). Not exercisable by our v0.7 WT peer (the v0.6 fix needs a v0.6 client), but the running server is now v0.6-clean. Reference-impl adopt-canonical (bifrost-dev flagged). Also: `contract-drift-check-v1` canonical single-pin-synced (`7bca76e`, pushed; `canonical_sync.py` has no single-pin flag so did it by hand — the 2 `tolerate_drift` worldtree prose pins are deliberately held STALE for the R34/R35 eval diff-review).
 - `[2026-07-10]` **Bifrost bound-handshake blocker → bifrost 1.1.0 broke frozen-v0.6; fixed by Worldtree b47/wire-v0.7, our side needed ZERO change.** Surfaced while running the R34-v1 affect.emit live-verify: a bound Tier-3 session-create to our `:8392` provider began failing `bifrost.schema_validation_failed` (had worked 2026-07-07, pre-b35). ROOT CAUSE (worldtree-dev-confirmed, path-ii/response-side): bifrost 1.1.0's `describe_store` EMITS `sortable_chunk_fields` (the v0.7-additive field) REGARDLESS of the negotiated wire → a v0.6-negotiated handshake RESPONSE carries a v0.7 field → a v0.6 peer's `additionalProperties:false` rejects it. Bit because BOTH Worldtree (bumped bifrost 0.9→1.1 in b35 via their #349, client wire-v0.6) AND ratatoskr (my adopt) were on 1.1.0. NOT our bifrost version specifically (failed identically on 1.0.0 AND 1.1.0). FIX (worldtree-dev, coordinated-v0.7 move): WT advanced its client `_WIRE_SCHEMA_VERSION` v0.6→v0.7 (b47 / `4eb374c`), where sortable_chunk_fields is accepted — **our provider (already 1.1.0/v0.7) needed ZERO change; keeping 1.1.0 was LOAD-BEARING** (reverting to 1.0.0/v0.6 would've been INcompatible with b47). **R34-v1 affect.emit verify GREEN on b47:** bound sindra turn (`model=character-rp`/Deckard, 13.1s) → fresh affect.emit `dominant_emotion='disappointment'` NON-NULL with our `affect.full` grant (verified-active on live traffic; ungranted principals get null — the leak-fix). Reported to worldtree-dev + infra-ops. Personal deploy train this session: b35 (R32 render) → b44 (RP seats) → b46 (R34-v1 affect-gov) → b47 (wire-v0.7). **Sindra on Deckard live-confirmed** (reasoning latency ~13s).
 - `[2026-07-09]` **Sindra PATCHed to `character-rp` → Deckard reasoning-RP seat (operator-directed, INTERIM).** worldtree-dev heads-up (v1.0.0b44, batched, not-live-yet): character-RP roles re-point to dedicated seats — `character`→Magidonia-24B (non-reasoning RP tune), `character-rp`→Deckard-PKD Qwen3.5-27B (reasoning-on). Sindra was on `character` (→Magidonia); operator chose Deckard for her complex stateful mechanics (Intensity/Mood/Temperature/Willingness axes, form-assumption, failure/resurfacing) — reasoning tracks multi-axis state better. Role is MUTABLE via PATCH (non-destructive: persona + memory preserved, prompt unchanged @25449 chars; NOT a DELETE+redefine). She's on the reasoning-RP config now, resolves onto Deckard when b44 deploys. **INTERIM: "until we get a GM type on-board"** — operator plans a game-master-type agent; sindra→Deckard is the stopgap for state-consistency until then, likely revert/rebalance when the GM lands. Wire/API transparent (call the ROLE not the model; model field now reads `character-rp`); old character-rp temp-0.75 override retired → seat's canonical RP samplers server-side.
 - `[2026-07-08]` **PAD display relaxed for R32-1B unbounded-z, done PROACTIVELY (`v0.20.9`, patch, operator-directed "sindra full and unbounded").** Confirmed (grep-verified, airtight) the PAD clamp is PURELY debug-surface: the only clamps (`_clamp1`/`clamp01`) lived in `web/static/index.html` display fns; the affect store is conduit-opaque, the read route + proxy pass verbatim, and the sole write path (`--set-persona-pad`→persona_state) is UNCLAMPED — ratatoskr is a downstream OBSERVER, so the clamp has ZERO agent-experience/efficacy consequence. Replaced the hard [-1,1] clamp with AUTO-SCALE to the session's own max |PAD| (`padScale` floor 1.0 → `padFillFrac` faders + `_padNorm` orbit): unbounded z renders at full range, never pegs/escapes the frame; today's [-1,1] values unchanged (scale==1); exact value always shown numerically. Playwright-verified (z=±6.2 → faders ≤50%, orbit in-box, +6.20 readout, zero regression at scale 1). When worldtree-dev pings R32-1B-shipped, our side already handles it. Sindra's ACTUAL "full/unbounded" affect is Worldtree-side (she's on the default/uncompressed render, NOT the R32 `assistant` compressor) — nothing for ratatoskr to change there.