memory: /snapshot — condense in-flight to milestone-close; person-prime shipped+verified, R39 arc closed; first persistent-memory.d/ detail files

This commit is contained in:
2026-07-15 22:54:20 -07:00
parent 5e3e88d26d
commit e365b24339
4 changed files with 217 additions and 50 deletions
@@ -0,0 +1,41 @@
# [2026-07-16] bifrost scan/cursor conformance gap → snapshot-cursor ruled NORMATIVE
## The gap (I surfaced it; operator's catch that it was bifrost's to fix)
The heid-bug-hunt flagged our offset cursor's cross-page dup/drop under mutation. On
verifying before routing, it turned out sharper than "robustness": bifrost's PROTOCOL
already mandates snapshot cursors — the dispatch engine (`bifrost/memory.py:311-314`)
drops `sort` on a cursor continuation with the comment *"the cursor's snapshotted order is
authoritative"*, and maps `ScanCursorExpired → 410`. The reference `InMemoryMemoryStore`
implements it (frozen ordered id-list per opaque token + TTL). But bifrost's **conformance
suite had ZERO scan/cursor coverage** — so our non-snapshot offset cursor passed
conformance while violating the protocol contract. That coverage hole is the real
completeness concern. Routed to bifrost-dev (thread `01KXK7MDTY…`).
## bifrost-dev's ruling
1. **Snapshot-cursor is NORMATIVE, not opaque/per-store.** Operator ruled: cursor
snapshots a frozen ordered id-list, continuation ignores `sort`, stale/unknown cursor →
`ScanCursorExpired` → 410. **Offset-with-documented-limits is NOT blessed.**
2. **Conformance coverage added** in **bifrost 1.1.3** (`bifrost.conformance.
memory_store_conformance`, 4 probes: `scan_snapshot_order_authoritative`,
`scan_no_dup_or_drop`, `scan_unknown_cursor_expired`, `scan_snapshot_stable_under_write`
opt-in). Our offset bug ships as their negative-canary fixture
(`fixtures.v0_7.offset_cursor_store.OffsetCursorMemoryStore`). `main()` grades import
failure as exit 2 (setup error) vs exit 1 (conformance FAIL).
## Our status + the TODO (operator-sequenced, NOT urgent)
Single-page person-prime (`cursor=None`) is already conformant — nothing shipped is broken;
it's only multi-page continuation that's non-conformant. Our contract INV-010 currently
marks the offset cursor **v1-provisional / KNOWN DEVIATION** (commit `8199774`).
TODO when sequenced:
1. Bump pin bifrost `1.1.1 → 1.1.3`.
2. Replace the offset cursor with the reference snapshot semantics (frozen id-list per
opaque token + `ScanCursorExpired` → 410).
3. Run `run_all(store_factory=..., include_optional=True)` against our SQLite store —
expect P1/P2/P3 RED → green (the before/after IS the validation). Report to bifrost-dev
(we're their canary — first non-reference scan implementer).
4. Flip INV-010 from v1-provisional to snapshot semantics.
TTL-duration expiry NOT asserted by the harness (no portable clock hook via store_factory);
bifrost-dev offered a `clock_control` opt-in if we ever need time-based expiry certified —
parked, not needed yet.
@@ -0,0 +1,65 @@
# [2026-07-15/16] person-prime `scan` build — SHIPPED + DEPLOYED + LIVE-VERIFIED (v0.20.14)
**The "last push": fix Sindra not remembering Vuong's name across sessions.**
## Root cause (worldtree-dev)
WT injects a recalled fact only if combined score (sim×salience) ≥ **0.45**
(`auto_inject_combined_score_threshold`, `core/memory/context_promotion/config.py`),
and recall is per-turn **query-gated** → moderate-sim durable facts (name hit ~0.40)
never inject. Designed turn-0 fix = WT **#349 person-prime**: a query-LESS
top-N-by-recency durable-fact injection, capability-gated on the store advertising
`updated_at` in `sort_fields` at the Bifrost handshake — DARK for our provider until now.
Fix (ZERO Worldtree change): implement the sorted `scan` verb + advertise the cap.
## What shipped (6 commits, v0.20.11 → v0.20.14; suite 639 green throughout)
- `8fc757a` **v0.20.11**`scan` verb + `sortable_chunk_fields` cap. Query-LESS,
LIVE-only (INV-009: superseded/tombstoned excluded server-side), globally-ordered-
before-pagination by indexed `updated_at` (INV-010), sort dispatch-gated. Offset
cursor with a `has_more` peek (no empty trailing page = reference-parity). TDD 7/7
incl. `parity_vs_reference` #195. NUANCE resolved: bifrost's `InMemoryMemoryStore`
READS `updated_at` (never stamps it) — identical to ours; ref does NOT lifecycle-
filter so parity is over the live set only.
- `a9c521a` **v0.20.12** — Sindra holodesk first-message preset (split out of the
soong-lab redefine as its own concern).
- `25ccb5c` **v0.20.13** — heid-bug-hunt fix: a truthy non-dict `sort`
(`"updated_at"`/`["updated_at"]`/int) hit `(sort or {}).get(...)` → AttributeError
instead of InvalidArguments. Added isinstance guard (Gróa#1/Hulda#2 confirmed).
- `8199774` — contract: marked the offset cursor **v1-provisional / KNOWN DEVIATION**
(see [[2026-07-16-bifrost-cursor-conformance]]).
- `f46ccba` **v0.20.14****THE DEPLOY-BREAKER** (see Tried/abandoned): advertised
`sortable_chunk_fields=[{"name":"updated_at"}]` WITHOUT `type` → bifrost
`handshake_response` `SortableChunkField` requires BOTH name+type
(`additionalProperties:false`) → whole bind broke. Fixed → add `"type":"timestamp"`
+ regression guard in the caps test.
## heid-bug-hunt triage (panel Gróa/Hulda/Regin, thread `01KXK5XTYHV8TGEDRAZV8GRXWC`)
FIXED: non-dict sort (v0.20.13). SURFACED→operator: offset-cursor cross-page dup/drop
(→ became the bifrost cursor arc). NOTED (reference-parity/accept-known-risk, no fork):
full-table materialize-then-Python-filter (matches ref's O(store) iteration), unbound
limit, falsy scope coercion, lexical timestamp ordering. REFUTED: Regin's mixed-type
ORDER BY "crash" (SQLite orders by storage class, doesn't raise) + its self-retracted
None item.
## Deploy + live verify (2026-07-15, operator-authorized)
Restarted BOTH :8392 combined provider + :8765 web on the new code via the env-
preserving `scratchpad/relaunch_by_pid.py <pid>` (captures /proc cmdline+environ+cwd →
byte-identical config; self-daemonizes). Added the missing REQUIRED
`RATATOSKR_MEMORY_EMBEDDING_DIM=1024` to `env.sh`.
Drove a bound Sindra turn (`ratatoskr --new --agent ratatoskr:sindra --send … --bifrost-url
http://10.100.10.50:8392 --end-user-id ratatoskr-tui`):
- **GATE LIT + scan fired ONCE at turn 1** with worldtree-dev's exact args
(`scope_all={end_user:ratatoskr-tui} cursor=null limit=3 sort={updated_at,desc}`) →
3 records in **~5ms** (no 500ms fail-open). Injection confirmed in Sindra's CoT.
- **Cross-session recognition WORKS** — she recalls Vuong as a distinct person + his
patterns. The blank-slate problem is SOLVED.
- **BUT name-recall FAILS** — `"Name is Vuong."` is the OLDEST chunk (07:09) → excluded
from top-3-by-recency AND scores 0.354 (sub-0.45) on the query path → injects via
NEITHER; meanwhile a stale contradictory `"user has not yet provided their name"`
(07:58, 0.46) IS injected → she concludes she lacks the name. **Root cause = WORLDTREE
ranking/hygiene** (WT #364), not our wire. See [[2026-07-16-wt364-r39-name-recall]].
## Status
Technical path GREEN end-to-end (logged by worldtree-dev as the person-prime live
milestone). Name-recall waits on WT #364 + R39-designed identity-class pinning. Keeping
the live 10-chunk store as the #364 re-verify target.
@@ -0,0 +1,65 @@
# [2026-07-16] Name-recall gap → WT #364 + brokkr R39 re-drive (DECISIVE) + subject-provenance catch
Downstream of the person-prime live verify ([[2026-07-16-person-prime-scan-shipped]]),
which proved the wire is green but the *name* still misses. Root cause is Worldtree-side.
## WT #364 (worldtree-dev filed; our live specimen = the evidence base)
The name-recall miss is TWO defect classes, both at promotion:
1. **Contradiction-reconciliation missing.** Promotion wrote a NEGATIVE-knowledge fact
("user has not yet provided their name") that was already false; promotion does NO
contradiction check against the store, so both the true and stale facts sit live and
the WRONG one wins both recall paths (newer → recency top-3; 0.46 → above the 0.45
query gate, where the true name sits at 0.354).
2. **Subject-attribution leak (operator-caught, NEW class).** Fact `75aa3110` ("prefers
clear parameters Intensity/Mood/Willingness…") is NOT a user fact — Vuong never said
it; it's **Sindra's OWN system-prompt scripted behavior mis-extracted into the USER
memory partition**. `e35b9dfe` (closeness) maybe the same. So the extractor leaks
CHARACTER-self facts into user memory — a subject-correctness axis orthogonal to the
recency/threshold/stale-negative story. worldtree-dev folded it into #364 as a
**subject-attribution gate at promotion** (a third reconciliation dimension).
**The #364 fix that ships = `(subject,relation)` slot-supersession + identity-tier
surfacing** — NOT a threshold tweak. Our harness data directly shaped it. It lands with a
re-verify request to us. **#349 ranking decision already RULED by operator (2026-07-15 via
brokkr's R39 thread): no top-N recency band-aid; straight to R39 identity-class pinning.**
## brokkr R39 Phase-1 Arm-0 fusion bake-off — DECISIVE
brokkr replayed our frozen 7-fact specimen. **HEADLINE: no (similarity, salience) fusion
can fix #364.** The stale negative PARETO-DOMINATES the true name — more similar
(0.463 > 0.354) AND equal salience (1.0 = 1.0) — so any monotone f(sim,sal) puts stale
above true: S0 product / S1 weighted-sum / S2 RRF all fail. Only **S3 bounded-boost** lands
true-in ∧ stale-out, and ONLY via the identity-class/source signal + negative-validity
retirement, NOT the sim/sal fusion. **Threshold-tuning is a dead end; the fix is the signal
FAMILY** (hard confirmation of Phase-0). Write-up: brokkr
`research/R39-memory-salience-dreams-surfacing/phase-1/re-drive-results.md`.
My 3 findings all confirmed + folded: (1) salience blind — both name facts salience 1.0;
(2) shared `(user,name)` supersession slot (Phase-0 Q3); (3) char-self-leak = "genuine NEW
class" that RAISED the VoI of R39's dream/offline-hygiene facet (offline consolidation
re-partitioning mis-attributed facts).
## Data structure findings (for the export)
Our provider persists ONLY `salience` (+ the embedding). `similarity`/`combined` are
WT-side query-time (`bifrost_memory_store.py:702`, combined = sim×salience) — NOT in our
store. **Person-prime is query-LESS → carries NO similarity** (brokkr's Arm-1 finding).
Recovered per-fact similarity from the verify SEARCH log (query "Hi Sindra — do you
remember me?"): name 0.354, stale 0.463; ×salience-1.0 = combined, matching brokkr's
0.354/0.46 grounding. `salience_word` (granite categorical) is a WT-extraction-time
artifact, not persisted.
## The export + the specimen
- Operator approved **VERBATIM** export ("nothing there is really a concern"). The
classifier had blocked writing PII to shared `/mnt/smithy`; routed to operator → he chose
full → delivered INLINE (scoped) in brokkr thread `01KXMN7NR54…` as 7-row JSONL.
- **⚠️ My verify drive CONTAMINATED the specimen**: it wrote 3 new chunks (re-extractions
incl. a 3rd "name unknown" negative) → store is now **10, not 7**. Original 7 intact.
- brokkr ACCEPTED the 3 verify-adds as the **Phase-2 Arm-2 seed** (domain-contradiction
set). I froze a WAL-consistent snapshot of the full 10-chunk store at
`r39-frozen-specimen/sindra-10chunk-specimen.db` (gitignored, `VACUUM INTO`) so it
SURVIVES #364 reconciliation. Export on brokkr's Phase-2 signal.
- **R39 Arm-0 hold LIFTED** (worldtree-dev). Live store kept UNTOUCHED as the #364
re-verify target; frozen snapshot carries the research seed forward independently.
## Peer threads
worldtree-dev verify `01KXK86PZ9…` + hold `01KXMK1C44…`; brokkr R39 `01KXMN7NR54…`;
bifrost cursor `01KXK7MDTY…` (see [[2026-07-16-bifrost-cursor-conformance]]).
+46 -50
View File
File diff suppressed because one or more lines are too long