memory: /snapshot — #368 silo test passed; two-tier migration; stores scrubbed

- #368 (user,character) memory silo test DONE + PASSED live (WT b127):
  write-side conjunctive {end_user,agent_self} scoping + read-side cross-
  character isolation both proven end-to-end; betty (throwaway) deleted,
  Sindra intact. Full record in persistent-memory.d/2026-07-18-368-silo-
  test-passed.md. Retired the stale "silo test in progress" in-flight blocks.
- Two-tier migration: split 152 over-threshold dated entries into
  persistent-memory.d/ detail files, leaving one-line pointers in the index
  (startup load ~196KB -> ~53KB; bodies now load on demand).
- Tier-3 stores scrubbed clean (memory 0 / affect 0, provider restarted
  empty); persistent-memory + detail file updated to reflect the scrub.
This commit is contained in:
vh
2026-07-17 21:39:18 -07:00
parent 29481fbcf0
commit ff1f9ca1e8
154 changed files with 393 additions and 178 deletions
@@ -0,0 +1,70 @@
`[2026-07-18]` **#368 (user, character) memory silo test — DONE, PASSED live against Worldtree b127.**
The immediate post-clear action (the reason for the 2026-07-18 clear). Closed with a decisive PASS on both the
write and read halves of the silo. See sibling arcs [[2026-07-16-wt364-r39-name-recall]] (name recall) and the
#368 enforcement-read history captured in the Recent-decisions log.
## The bug (recap)
WT wrote Tier-3 memory scoped `{end_user}`-only; `agent_id` rode as record metadata, NOT isolation scope. So every
character under one end_user shared the memory pool (Lofn recited Sindra's facts; the operator caught it). NOT a
ratatoskr defect — our lattice `{end_user,group,tenant,agent_self}` + `_matches_scope` already supported the
agent axis; WT's write AND retrieval both omitted it. Operator ruled Option 1: silo per (user,character) +
backfill. ratatoskr was the ENFORCEMENT half (consumer-side read conformance), never a code-change target.
## The fix (what b127 ships)
Every Tier-3 write is stamped with the conjunctive `{end_user, agent_self}` scope. On each session bind WT probes
our store for the migration sentinel `worldtree:migration:agent-scope-v1`; if absent it runs a store-wide backfill
(single-page scan per our finding F2; sentinel carries a zero-vector at embedding_dim per F3) and only FLIPS reads
to conjunctive once the sentinel is present. Un-bound end_user reads on a migrated store drop fail-closed, never
sent as an empty (match-all) filter (F1). All three enforcement finds folded into WT contract rev 1.3.
## Test design (why it's valid)
Held **end_user CONSTANT** (`silo-368`) across BOTH agents — that isolates the `agent_self` axis under test.
Different end_users would let the pre-existing end_user axis explain any separation and prove nothing about #368.
Drove via the **CLI combined bind** (`--bifrost-url http://10.100.10.50:8392`), not the web path — same canonical
combined bind onto the same :8392 provider, but scriptable/deterministic (captures session ids + turn output).
Sindra/Betty are consumer-defined agents with NO server-side chroma → our Tier-3 store is their SOLE memory source,
so cold-recall is a clean test of our store's conjunctive retrieval (no channel-2 confound; that's foundational-only).
## Results — silo holds end to end
- **Migration verified:** first bind wrote the sentinel (`worldtree:migration:agent-scope-v1`, scope
`{"tenant":"worldtree:migration"}`, content "agent-scope-v1 backfill complete"); backfill a no-op on the
born-empty store. Reads flipped conjunctive.
- **WRITE fix (structural core):** Sindra's 3 promoted chunks each carry `{"end_user":"silo-368",
"agent_self":"ratatoskr:sindra"}` (verbatim texts: "Strongly prefers coffee over tea;", "Morning ritual is always
a strong black coffee.", plus a meta-reinforcement chunk). Real WT→provider promotion, not a synthetic write.
- **READ isolation (behavioral, decisive):** cold-recall in FRESH sessions (no in-session context), same end_user
`silo-368`. Sindra pulled her own chunks via person-prime and answered "Coffee. Strong black, no sugar." Betty,
sharing the exact same end_user, got ZERO hits → "I don't know. I don't have memory across chats." Betty did NOT
bleed Sindra's coffee chunks — her conjunctive `{silo-368, betty}` filter matched none of Sindra's
`{silo-368, sindra}`. That's the precise pre→post flip of the Lofn-recites-Sindra bug.
- Bind identical for both (admin `GET /admin/sessions/{id}/bifrost`: connected, caps `[affect, memory]`).
## The Betty asymmetry (NOT a #368 issue — banked as a data point)
Betty never got her OWN tea fact promoted (18+ min idle, while Sindra promoted in ~8.5 min). Cause: WT's memory
extractor leans on the ASSISTANT's prose, and Betty's terse "brief replies" persona never restated the fact, so
nothing was extracted. Sindra's verbose persona restated "your favorite drink is strong black coffee" out loud →
that got promoted. So the persona shaped the outcome INDIRECTLY (reply verbosity), but the gate is WT-side
extraction, not persona disposition. Known #296/#369 family (extractor favors assistant prose over the raw user
statement). Did NOT weaken the verdict — the decisive cross-bleed direction passed cleanly, and Sindra demonstrated
the positive-recall direction. Operator declined a worldtree-dev FYI on it (tangential to #368).
## Name clarification (operator asked)
The operator's name (Vuong) was NOT part of this test — zero mentions in any sent message or either agent's recall,
no name chunk in the store. This test was beverage-only under a FRESH, scrubbed end_user (`silo-368`). The
name-recall behavior from the earlier contamination saga was under a DIFFERENT partition (`ratatoskr-tui`), scrubbed.
## Cleanup + tooling notes
- Throwaway `ratatoskr:betty` DELETED: `python -m ratatoskr.tier3 delete ratatoskr:betty` → WT 404 + local-registry
removal; Sindra intact (`GET /agents/ratatoskr:sindra` = 200). The auto-mode classifier denied the first attempt
(irreversible remote deletion); operator explicitly authorized.
- **tier3 CLI define/patch adapted to the b127 `role` schema** (was `model`), commit `860e0d5`, v0.21.1. The
`delete` verb works (used above).
- **Store SCRUBBED clean afterward (2026-07-18, operator-directed):** `reset-sindra-stores.sh` (rolling backup
`db-reset-backup/`) → memory 0 / affect 0, combined provider restarted empty (pid 3856058). The next bind
re-runs the no-op migration + sentinel write. Silo-test evidence lives in this file, not the store.
## Still QUEUED
The **backfill live-verify** (synthetic legacy corpus): when exercising the backfill machinery matters, drive a
synthetic pre-migration (`{end_user}`-only) corpus into our store, run WT's backfill against it, confirm 100%
stamped + sentinel writes + reads flip clean — a born-fresh store skips the backfill entirely (this test did).