From 46d6efa96220a64f74b3f1102022fe9424815784 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Wed, 15 Jul 2026 09:14:09 -0700 Subject: [PATCH] =?UTF-8?q?memory:=20snapshot=20=E2=80=94=20person-prime?= =?UTF-8?q?=20LIVE-VERIFIED=20(v0.20.14=20handshake=20fix);=20name-recall?= =?UTF-8?q?=20gap=20is=20WT=20ranking/hygiene?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- persistent-memory.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/persistent-memory.md b/persistent-memory.md index 974b789..77b38f8 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -46,7 +46,11 @@ upstream API key stays server-side (INV-003). _As of 2026-07-15:_ -**✅ person-prime `scan` build (THE "last push"): DEPLOYED (v0.20.13); AWAITING worldtree-dev turn-1 verify.** WHY: Sindra didn't remember Vuong's name across sessions; worldtree-dev root-caused it — WT injects a recalled fact only if combined score (sim×salience) ≥ **0.45** (`auto_inject_combined_score_threshold`, `core/memory/context_promotion/config.py`), and recall is per-turn **query-gated**, so moderate-sim durable facts (name hit ~0.40) never inject. The designed **turn-0** fix is WT **#349 person-prime** — a query-LESS top-N-by-recency durable-fact injection — capability-gated on the store advertising **`updated_at` in `sort_fields`** at the Bifrost handshake, DARK for our provider until now. Fix (ZERO Worldtree change): implement the sorted `scan` verb + advertise the capability. **✅ CODE COMPLETE + COMMITTED (2026-07-15):** contract amended + validated; `scan` impl in `src/ratatoskr/provider/memory_store.py` (globally-ordered by indexed `updated_at`, live-only via `_is_live`, offset-cursor; `has_more` peek so an exact-fill page emits no empty trailing cursor = reference-parity; `describe_store` advertises `sortable_chunk_fields=[{updated_at}]`; expression index `idx_chunks_updated_at`); **TDD 7/7 GREEN** (scan_recency tracer + live_only + scope_isolation + unadvertised_sort + person_prime_record_shape + cursor pagination + **`parity_vs_reference` #195**); NUANCE resolved — the bifrost ref `InMemoryMemoryStore` **READS** `updated_at` from the record (never stamps it; its own comment confirms), identical to ours, so order-parity holds over the live set (ref does NOT lifecycle-filter, so parity is defined over live chunks only). **Full suite 638 green.** Committed as **two patches**: `8fc757a` scan (**v0.20.11**) + `a9c521a` Sindra holodesk first-message preset (**v0.20.12**, split out as a separate concern from the soong-lab redefine). **heid-bug-hunt RETURNED + TRIAGED** (panel Gróa/Hulda/Regin, thread `01KXK5XTYHV8TGEDRAZV8GRXWC`; base-ref 39050c3, code-only diff). **FIXED (v0.20.13, `25ccb5c`):** the non-dict `sort` crash — a truthy non-dict `sort` (`"updated_at"`/`["updated_at"]`/int) hit `(sort or {}).get(...)` → AttributeError instead of InvalidArguments; added isinstance guard (Gróa#1/Hulda#2, confirmed-from-code). **SURFACED (operator design decision, NOT auto-adopted):** offset-cursor cross-page duplicate/drop under concurrent mutation (all 3 arms) — real divergence from bifrost `InMemoryMemoryStore` (snapshots the ordered id-list per opaque cursor + `ScanCursorExpired` TTL); BUT person-prime uses a SINGLE page (no cursor) so the actual use isn't exercised → operator picks: adopt reference snapshot-cursors vs keyset vs accept + honestly scope INV-010 to a static store. **NOTED (reference-parity / accept-known-risk, no fork):** full-table materialize-then-Python-filter (matches ref's own O(store) iteration), unbound limit, falsy scope coercion `[]`→`{}` (parity w/ search + ref), lexical timestamp ordering (WT writes consistent ISO-8601 UTC); `_is_live` dead-marker completeness (Gróa#6 needs-repro) → confirm canonical marker set w/ worldtree-dev, don't guess. **REFUTED:** Regin's mixed-type ORDER BY "crash" (SQLite orders by storage class, doesn't raise) + its self-retracted `_is_live`/None item. **✅ DEPLOYED (2026-07-15, operator-authorized):** (5) **DONE** — restarted BOTH `:8392` combined provider (new pid, old 982284 gone) AND `:8765` web on **v0.20.13** via the env-preserving `scratchpad/relaunch_by_pid.py ` (captures /proc cmdline+environ+cwd → byte-identical config, only code changes; self-daemonizes). Verified: `:8392` up (uvicorn startup complete; `/`→404 = no root route = serving), deployed `describe_store` advertises `sortable_chunk_fields=[{updated_at}]`; `:8765` up HTTP 200 (log "ratatoskr-web v0.20.13"). **Added the missing REQUIRED `RATATOSKR_MEMORY_EMBEDDING_DIM=1024` to `env.sh`** (gitignored; was absent → bare `source env.sh` restart would crash). (6) **DONE** — pinged **worldtree-dev** (thread `01KXK86PZ9M8J53A4SFXREEMW7`) to verify turn-1 recall against personal :8081. Store has **7 REAL live Sindra durable facts** ready (agent_id `ratatoskr:sindra`, worldtree_scope `end_user`, scope `{end_user:ratatoskr-tui}`, all carry `updated_at`, accumulated across 3 sessions today) → top-N-by-recency has real data to inject. AWAITING their turn-1 verdict (monitor armed). (7) **F3 cursor RESOLVED — operator ACCEPTED offset-for-v1** (keep offset cursor; person-prime single-page is correct/conformant). INV-010 amended to mark the cursor **v1-provisional / KNOWN DEVIATION** (multi-page continuation diverges from bifrost's protocol snapshot-cursor contract — the `bifrost.memory` dispatch drops `sort` on continuation citing "snapshotted order is authoritative" + maps `ScanCursorExpired→410`; our offset cursor doesn't snapshot → dup/drop under concurrent write, and never expires). **Durable fix = a BIFROST completeness concern (operator's catch), ROUTED to bifrost-dev** (thread `01KXK7MDTY0B6VEDF3SVGSKY9R`): bifrost's conformance suite has ZERO scan/cursor coverage, so a non-snapshot cursor passes silently — asked bifrost-dev to (1) confirm snapshot-cursor+ScanCursorExpired is the NORMATIVE store contract, (2) add scan/cursor conformance coverage; we'll adopt the reference snapshot-cursor semantics IF they rule it normative (offered to be their canary as first non-reference scan implementer). AWAITING bifrost-dev ruling; monitor armed. +**✅ person-prime `scan` build (THE "last push"): DEPLOYED + LIVE-VERIFIED (v0.20.14). Technical path GREEN; name-recall gap is now WORLDTREE-side (ranking/hygiene).** + +**⚠️ DEPLOY-BREAKER caught by driving the live bind (v0.20.14, `f46ccba`):** I first advertised `sortable_chunk_fields=[{"name":"updated_at"}]` WITHOUT `type` — bifrost's `handshake_response` `SortableChunkField` schema requires BOTH name+type (`additionalProperties:false`), so the handshake_response failed wire-schema validation → `bifrost.schema_validation_failed` → the ENTIRE bind (memory+affect) broke, not just sort. Unit tests + worldtree-dev's name-only service parser + the heid-bug-hunt ALL missed it; only DRIVING the real handshake (`/verify` discipline) caught it. Fixed → `{"name":"updated_at","type":"timestamp"}` (matches reference; type advisory-only) + regression guard in the caps test (asserts required name+type, no extra keys). LESSON: validate `describe_store` against the bifrost wire schema, not just our own caps assertions. + +**LIVE VERIFY (2026-07-15, drove a bound Sindra turn via `ratatoskr --new --agent ratatoskr:sindra --send … --bifrost-url http://10.100.10.50:8392 --end-user-id ratatoskr-tui`):** GATE LIT + scan fired ONCE at turn 1 with worldtree-dev's exact args (`scope_all={end_user:ratatoskr-tui} cursor=null limit=3 sort={updated_at,desc}`) → 3 records in ~5ms (no 500ms fail-open); injection confirmed in Sindra's CoT; **cross-session recognition WORKS** (she recalls him as a distinct person + patterns — blank-slate GONE). **BUT name-recall FAILS** — `"Name is Vuong."` is the OLDEST chunk (07:09) → excluded from top-3-by-recency AND scores 0.354 (sub-0.45) on the query path, so injects via NEITHER path; meanwhile a STALE contradictory `"user has not yet provided their name"` (07:58, 0.46) IS injected → Sindra concludes she lacks the name. **ROOT CAUSE = WORLDTREE ranking/hygiene** (not our wire): recency-top-3 under-weights early identity facts + the "no name" fact was never `mark_superseded` when "Name is Vuong" landed → both live+contradictory. Reported to worldtree-dev (thread `01KXK86PZ9…`) with 3 options: memory hygiene (supersede), person-prime salience-weighting, or higher N. AWAITING their call; our harness + 7 real facts sit ready for a re-drive. WHY: Sindra didn't remember Vuong's name across sessions; worldtree-dev root-caused it — WT injects a recalled fact only if combined score (sim×salience) ≥ **0.45** (`auto_inject_combined_score_threshold`, `core/memory/context_promotion/config.py`), and recall is per-turn **query-gated**, so moderate-sim durable facts (name hit ~0.40) never inject. The designed **turn-0** fix is WT **#349 person-prime** — a query-LESS top-N-by-recency durable-fact injection — capability-gated on the store advertising **`updated_at` in `sort_fields`** at the Bifrost handshake, DARK for our provider until now. Fix (ZERO Worldtree change): implement the sorted `scan` verb + advertise the capability. **✅ CODE COMPLETE + COMMITTED (2026-07-15):** contract amended + validated; `scan` impl in `src/ratatoskr/provider/memory_store.py` (globally-ordered by indexed `updated_at`, live-only via `_is_live`, offset-cursor; `has_more` peek so an exact-fill page emits no empty trailing cursor = reference-parity; `describe_store` advertises `sortable_chunk_fields=[{updated_at}]`; expression index `idx_chunks_updated_at`); **TDD 7/7 GREEN** (scan_recency tracer + live_only + scope_isolation + unadvertised_sort + person_prime_record_shape + cursor pagination + **`parity_vs_reference` #195**); NUANCE resolved — the bifrost ref `InMemoryMemoryStore` **READS** `updated_at` from the record (never stamps it; its own comment confirms), identical to ours, so order-parity holds over the live set (ref does NOT lifecycle-filter, so parity is defined over live chunks only). **Full suite 638 green.** Committed as **two patches**: `8fc757a` scan (**v0.20.11**) + `a9c521a` Sindra holodesk first-message preset (**v0.20.12**, split out as a separate concern from the soong-lab redefine). **heid-bug-hunt RETURNED + TRIAGED** (panel Gróa/Hulda/Regin, thread `01KXK5XTYHV8TGEDRAZV8GRXWC`; base-ref 39050c3, code-only diff). **FIXED (v0.20.13, `25ccb5c`):** the non-dict `sort` crash — a truthy non-dict `sort` (`"updated_at"`/`["updated_at"]`/int) hit `(sort or {}).get(...)` → AttributeError instead of InvalidArguments; added isinstance guard (Gróa#1/Hulda#2, confirmed-from-code). **SURFACED (operator design decision, NOT auto-adopted):** offset-cursor cross-page duplicate/drop under concurrent mutation (all 3 arms) — real divergence from bifrost `InMemoryMemoryStore` (snapshots the ordered id-list per opaque cursor + `ScanCursorExpired` TTL); BUT person-prime uses a SINGLE page (no cursor) so the actual use isn't exercised → operator picks: adopt reference snapshot-cursors vs keyset vs accept + honestly scope INV-010 to a static store. **NOTED (reference-parity / accept-known-risk, no fork):** full-table materialize-then-Python-filter (matches ref's own O(store) iteration), unbound limit, falsy scope coercion `[]`→`{}` (parity w/ search + ref), lexical timestamp ordering (WT writes consistent ISO-8601 UTC); `_is_live` dead-marker completeness (Gróa#6 needs-repro) → confirm canonical marker set w/ worldtree-dev, don't guess. **REFUTED:** Regin's mixed-type ORDER BY "crash" (SQLite orders by storage class, doesn't raise) + its self-retracted `_is_live`/None item. **✅ DEPLOYED (2026-07-15, operator-authorized):** (5) **DONE** — restarted BOTH `:8392` combined provider (new pid, old 982284 gone) AND `:8765` web on **v0.20.13** via the env-preserving `scratchpad/relaunch_by_pid.py ` (captures /proc cmdline+environ+cwd → byte-identical config, only code changes; self-daemonizes). Verified: `:8392` up (uvicorn startup complete; `/`→404 = no root route = serving), deployed `describe_store` advertises `sortable_chunk_fields=[{updated_at}]`; `:8765` up HTTP 200 (log "ratatoskr-web v0.20.13"). **Added the missing REQUIRED `RATATOSKR_MEMORY_EMBEDDING_DIM=1024` to `env.sh`** (gitignored; was absent → bare `source env.sh` restart would crash). (6) **DONE** — pinged **worldtree-dev** (thread `01KXK86PZ9M8J53A4SFXREEMW7`) to verify turn-1 recall against personal :8081. Store has **7 REAL live Sindra durable facts** ready (agent_id `ratatoskr:sindra`, worldtree_scope `end_user`, scope `{end_user:ratatoskr-tui}`, all carry `updated_at`, accumulated across 3 sessions today) → top-N-by-recency has real data to inject. AWAITING their turn-1 verdict (monitor armed). (7) **F3 cursor RESOLVED — operator ACCEPTED offset-for-v1** (keep offset cursor; person-prime single-page is correct/conformant). INV-010 amended to mark the cursor **v1-provisional / KNOWN DEVIATION** (multi-page continuation diverges from bifrost's protocol snapshot-cursor contract — the `bifrost.memory` dispatch drops `sort` on continuation citing "snapshotted order is authoritative" + maps `ScanCursorExpired→410`; our offset cursor doesn't snapshot → dup/drop under concurrent write, and never expires). **Durable fix = a BIFROST completeness concern (operator's catch), ROUTED to bifrost-dev** (thread `01KXK7MDTY0B6VEDF3SVGSKY9R`): bifrost's conformance suite has ZERO scan/cursor coverage, so a non-snapshot cursor passes silently — asked bifrost-dev to (1) confirm snapshot-cursor+ScanCursorExpired is the NORMATIVE store contract, (2) add scan/cursor conformance coverage; we'll adopt the reference snapshot-cursor semantics IF they rule it normative (offered to be their canary as first non-reference scan implementer). AWAITING bifrost-dev ruling; monitor armed. **✅ soong-lab interop ESTABLISHED (this session) — soong-lab is now our Tier-3 agent-authoring studio.** ratatoskr consumes soong-lab bundles → WT `agents.define`, and ships agents back as bundles. Sindra round-trip PROVEN (soong imported her `resume` half through real `import_bundle`). soong-lab export/importer contracts PINNED via canonical-sync (`canonical_source=soong-lab` @ f434016, commit `39050c3`). Role-grounding (worldtree-dev): the 4 soong-lab ROLE_CHOICES = WT model-role slugs 1:1 BY NAME — `character`/`thoughtful-character` need the `character` grant (HELD via Sindra → define-able now); `assistant`/`thoughtful-assistant` need `foundational` (confirm/mint routed to infra-ops, in flight; only for assistant-type bundles). `character-rp` ≡ `thoughtful-character` (same Deckard seat + grant). **Sindra REDEFINED from the bundle** (`/tmp/sindra.json`, operator "update to match"): persona immutable → DELETE+redefine (payload validated via a throwaway `sindra-probe2`→201 BEFORE deleting the real one), `memory:{}` preserved, motivational string→WT-object mapped (Tried/abandoned), role→`thoughtful-character`, first_message preset updated. Verified live (thoughtful-character, 25574-char prompt).