From 4f74645a0053c6991e664d18b560329776a39d06 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Sun, 19 Jul 2026 10:24:35 -0700 Subject: [PATCH] =?UTF-8?q?memory:=20/snapshot=20=E2=80=94=20worldtree-sdk?= =?UTF-8?q?=20cutover=20slice-4=20complete=20(c62b4ee=E2=86=92477d98f)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...-worldtree-sdk-cutover-slice-4-complete.md | 66 +++++++++++++++++++ ...6-07-19-wyrd-368-silo-consult-delivered.md | 31 +++++++++ persistent-memory.md | 62 +++++++++-------- 3 files changed, 130 insertions(+), 29 deletions(-) create mode 100644 persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-4-complete.md create mode 100644 persistent-memory.d/2026-07-19-wyrd-368-silo-consult-delivered.md diff --git a/persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-4-complete.md b/persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-4-complete.md new file mode 100644 index 0000000..eef1a95 --- /dev/null +++ b/persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-4-complete.md @@ -0,0 +1,66 @@ +`[2026-07-19]` **worldtree-sdk cutover SLICE-4 COMPLETE + committed (`c62b4ee`+`aed9429`+`477d98f`, v0.21.13–.15, 475 green).** + +Agents/Tier-3 family onto the `ratatoskr.wt` adapter + the `model`→`role` fold (scope B), +full House Code Discipline end-to-end (contract error-map → TDD → rewire → delete → live +smoke → coverage re-anchor → both heid gates → fixups). + +**What moved onto the SDK** (`client.agents.*`, open-world dicts, errors mapped by +route+(status,error_code) per INV-CUT-2): `list_agents` (`agents.list`), +`get_persona_state` (`agents.persona_state`; 404 persona_not_configured / +404 agent_not_available / 403 auth_scope_denied dual-key), `define_agent` +(`agents.define`; 429→Tier3QuotaExceeded(retry_after=0) status-only, 403 +tier3_user_id_unsupported, 422 layer_deferred), `patch_agent` (`agents.patch`; bare-404→ +Tier3AgentNotFound, 422 field_not_mutable), `delete_agent` (`agents.delete`; bare-404). +Rewired the `python -m ratatoskr.tier3` CLI + the web `_agents_endpoint` / +`_persona_state_endpoint`. DELETED the hand-rolled `sessions.list_agents` / +`get_persona_state` / `AgentInfo` and `tier3.define/patch/delete_agent` / +`Tier3AgentInfo` / parse+extract helpers. + +**model→role fold:** define/patch response echoes `role` (spec 1.2 / b128), read off the +open-world dict; `LocalAgentEntry.model`→`.role`, local-index `_SCHEMA_VERSION` 1→2 (old +index discarded, no-compat). `project_tier3_agents_model_to_role_pending` is RESOLVED by +this — the deferred scope-B work landed here (doing it standalone would have been throwaway). + +**KEY SLICE-4 FOOT-GUNS (for slices 5-7 + any tier3 work):** +- **`python -m` double-module exception identity.** Running tier3 as `__main__` while + `wt` imports `ratatoskr.tier3` bound TWO copies of each `Tier3*` class → a raised + `Tier3AgentNotFound` escaped the CLI's `except` as an uncaught traceback (exit 1, not + the mapped exit 20). Unit tests call `main()` in-process (module is `ratatoskr.tier3`, + not `__main__`) so they NEVER hit the split — the LIVE SMOKE caught it. Fix: the `Tier3*` + exceptions live in `sessions.py` (never run as `__main__`) → single class identity, and + it removed the `wt→tier3` import edge. Rule: caller-semantic exceptions the adapter + raises + a `-m` CLI catches must NOT live in the `-m` module. +- **Open-world presenters must degrade, never crash.** The heid-bug-hunt panel (5/5/5, two + 3/3 crash sites) caught the CLI (`_run_define`/`_run_patch`) and web (`_agents_endpoint`) + HARD-INDEXING the open-world dicts (`info["agent_id"]`, `{a["agent_id"] for a in + upstream}`) → KeyError/TypeError on a partial/drifted 2xx response (incl. `system_prompt: + null` → `None.splitlines()` AttributeError). The wt tests + live smoke used FULL server + dicts, so it never surfaced. Fix: `_str_field` (absent/null/non-str → default) in the + CLI; well-formed-mapping filter in the web endpoint; a no-agent_id 2xx → controlled exit + 20. The invariant "open-world reads degrade, never crash the presenter" must hold at + EVERY presenter, not just the adapter. + +**Both heid gates cleared:** +- **code-review (panel, zero adapter/error-map/model→role drift):** 3 fixups (v0.21.14, + `aed9429`) — persona endpoint now catches `wt.SessionApiFailed` (parity with 3 sibling + endpoints; a latent PRE-cutover gap, NOT a slice-4 regression); dual-key NEGATIVE tests + for define/patch + flat-`field` test; contract documents the `":" in agent_id` PRE. +- **bug-hunt (panel 5/5/5, no false positives):** 3 fixups (v0.21.15, `477d98f`) — the two + open-world presenter crash sites above + `_error_field_from_body` type-checks `field` is + str. HELD (contract-intended, Heid-confirmed): the 429→quota / bare-404→not-found + status-only maps (the arms flagged them spec-free; the § Error map specifies them; the + SDK's ApiError floor drops Retry-After so retry_after=0 is canonical). Dual-keying + define's 429 for full row consistency is an available tightening (contract amendment), + surfaced not applied. + +**LIVE-SMOKE on personal :8081 (b128):** valid agent roles are `thoughtful-character` / +`character` / `assistant` (NOT the `/models/available-for-characters` `char-rp` — that's a +character-model, a different vocab; define 422s on it). define→patch→list(6 agents: +forseti/lofn/mask/mimir/vili/…)→persona_state(→PersonaNotConfigured mapped)→delete→index +empty; non-existent-id patch via `-m` → `[agent_not_found]` exit 20 (double-module fix +proven). Throwaway `ratatoskr:slice4-smoke` deleted, server left clean. + +**NEXT = slice-5** (characters + me/capabilities/models — the remaining consumer reads); +then slice-6 (admin: `stream_admin_events` + `get_session_bifrost`, admin_auth), slice-7 +(teardown: delete residual hand-rolled, drop `httpx-sse`, retire contracts #2/#15, MINOR +bump per DEC-6 w/ operator approval). Consumer layer ONLY; Bifrost provider planes untouched. diff --git a/persistent-memory.d/2026-07-19-wyrd-368-silo-consult-delivered.md b/persistent-memory.d/2026-07-19-wyrd-368-silo-consult-delivered.md new file mode 100644 index 0000000..2da89f1 --- /dev/null +++ b/persistent-memory.d/2026-07-19-wyrd-368-silo-consult-delivered.md @@ -0,0 +1,31 @@ +`[2026-07-19]` **wyrd-dev #368 silo-enforcement consult DELIVERED (althing thread `01KXXNDH3JE4`) — ratatoskr's store-side memory silo is CONVENTIONAL, wyrd going STRUCTURAL off my framing.** + +wyrd-dev is standing up their `bifrost-memory-store-server` (unit-4) and asked for +ratatoskr's hands-on read of the #368 `(end_user, agent)` silo mechanics before writing +their contract. I answered artifact-only from `src/ratatoskr/provider/memory_store.py` +(reference-impl posture: verified against code, not memory). + +**The load-bearing finding:** ratatoskr's silo is **conventional (query-time scope +filter), NOT structural.** Scope axes live inside `record_json` (+ a mirror `scope_json` +column), and every read path (`search`, `scan`, `list_chunks`) calls `_matches_scope` in +Python — "an `if` a caller can forget." Inherited byte-faithfully from bifrost's reference +`InMemoryMemoryStore`. Sharpest leak surfaces: `get`/`get_many` do ZERO scope check +(trust caller-knows-entitled-ids); the vec-search must over-fetch-ALL-then-Python-filter +(a naive `LIMIT top_k` in SQL silently under-recalls or leaks). ratatoskr has NO +`clear_partition` verb — forget is by-id `delete_many` (two-table chunk+vec atomicity in +one txn); the idempotency table is a hidden replay channel a total-forget must purge. + +**Outcome — wyrd committed to STRUCTURAL** (msg 3, `01KXXNQYGX0K`): one-DB-file-per-campaign +(campaign_id silo structurally unrepresentable across files) + `(scope_end_user, +scope_agent_self)` first-class NOT NULL composite-indexed columns, every query incl. the +vec JOIN carrying the partition in SQL `WHERE`. Dissolves my two sharpest leak surfaces +(by-id reads enforce the predicate; the vec JOIN filters+ranks+caps in ONE statement). +Folding my foot-guns as contract STEPS/invariants (two-table+producer-candidate delete +atomicity; forget purges idempotency + reaches superseded; `scope_any` = list of +whole-element-conjunctive dicts never flatten axes #297; `check_same_thread=False`; +`SortableChunkField` needs name+type or handshake dies; advertise⟹implement; +revision+idempotency-check before writes in one txn). worldtree-dev already pushed wyrd +structural in the record-shape re-read, so aligned; wyrd will flag the +reference-`InMemoryMemoryStore`-divergence to bifrost-dev (internal enforcement, wire +unchanged). **OPEN LOOP:** I offered to eyeball wyrd's data model once the unit-4 contract +is cut — they took it; they'll ping. No action pending on ratatoskr now. diff --git a/persistent-memory.md b/persistent-memory.md index 77652cd..0dcbb53 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -46,34 +46,34 @@ upstream API key stays server-side (INV-003). _As of 2026-07-19:_ -**🔨 ACTIVE MIGRATION — worldtree-sdk cutover (issue #20): SLICE-1 + SLICE-2 + SLICE-3 COMPLETE, slice-4 next.** +**🔨 ACTIVE MIGRATION — worldtree-sdk cutover (issue #20): SLICE-1–4 COMPLETE, slice-5 next.** Operator ruled ADOPT (2026-07-18): ratatoskr cuts its CONSUMER client layer over to **worldtree-sdk (Python) 1.0.0**, retiring the hand-rolled httpx wrappers behind a thin `ratatoskr.wt` adapter. Design locked (6 DECs, vor-cross'd, heid-panel-reviewed); contract `docs/contracts/worldtree_sdk_cutover.contract.md`. **SLICE-1+2 -(foundation + sessions/turn) ✅ PUSHED** origin `aba1730` (arc `b1fbadd`→`aba1730`, tags v0.21.3–.10). **SLICE-3 -(persona + authored-history + first-message) ✅ DONE** — `ca9a339` (feat) + `fc256bb` (heid-bug-hunt fixups), -tags v0.21.11–.12; full House Code Discipline (TDD → heid-code-review CLEAN/zero-drift → heid-bug-hunt). -Suite **469 green**. Slice-3 migrated `set_persona_state` (SDK `PadState`), `write_authored_history` -(`write_history`, 404→AuthoredHistoryUnavailable hide-existence), `get_session_messages`, + routed -`first_message` seed through the adapter; DELETED the last hand-rolled `sessions.py` paths (`create_session`+ -`SessionInfo`, `set_persona_state`, `write_authored_history`, `get_session_messages`, `_bifrost_error_from`). -LIVE-SMOKE on :8081 (b128): seed→201(seq0)→read-back; persona→204; create-path preset seed. **KEY ADAPTER FACTS -(foot-guns for slices 4-7):** SDK returns **open-world dicts** for reads → `info["session_id"]`; `TurnEvent` -carries `sse_id` + a **`turn_id` ABSENT on text/thinking frames** → parse cancel-target from `sse_id`; the SDK -**normalizes ANY transport failure to `ConnectFailed(status=0)`** (`request.py:196`, NOT raw httpx) — every -caller through the adapter must `except ConnectFailed` (the slice-3 bug-hunt caught both probes missing it); -`consumer_key` is BOUND-create-only; the SDK's envelope parser **prefers nested `detail`** (why bound-502 isn't -error_code-gated). **NEXT = slice-4** (agents/tier3 — list/get/define/patch/delete/persona_state, FOLDS the -`model`→`role` cutover); then slice-5 (characters/me/caps), slice-6 (admin — `stream_admin_events` + -`get_session_bifrost` still hand-rolled), slice-7 (teardown: retire contracts #2/#15, drop `httpx-sse`, minor -bump per DEC-6 w/ operator approval). Scope: consumer layer ONLY; Bifrost provider planes untouched. Full -design → auto-memory `project_worldtree_sdk_cutover`. +(foundation + sessions/turn) ✅ PUSHED** origin `aba1730`. **SLICE-3 (persona + authored-history + first-message) +✅ DONE** `ca9a339`+`fc256bb`. **SLICE-4 (agents/Tier-3 + `model`→`role` fold) ✅ DONE** — `c62b4ee` (feat) + +`aed9429` (heid-code-review fixups) + `477d98f` (heid-bug-hunt fixups), tags v0.21.13–.15; full House Code +Discipline, both heid panels cleared. Suite **475 green**. Slice-4 migrated +list_agents/get_persona_state/define/patch/delete onto `client.agents.*` (open-world dicts, errors mapped by +route+(status,error_code)), folded `model`→`role` (`LocalAgentEntry.role`, index schema v2), and DELETED the +hand-rolled `sessions.list_agents`/`get_persona_state`/`AgentInfo` + `tier3.define/patch/delete_agent`. Full arc ++ the two slice-4 foot-guns → `persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-4-complete.md`. +**KEY ADAPTER FACTS (foot-guns, cumulative for slices 5-7):** SDK reads = **open-world dicts** — +presenters MUST degrade not crash (slice-4 bug-hunt: hard-indexing `info["x"]` crashes on partial wire; use +`.get`/type-guards; `system_prompt: null` → `None.splitlines()`); the SDK **normalizes ANY transport failure to +`ConnectFailed(status=0)`** (NOT raw httpx) — every adapter caller must `except ConnectFailed`; +**caller-semantic exceptions the adapter raises + a `-m` CLI catches must NOT live in the `-m` module** (the +`python -m ratatoskr.tier3` double-module split gave two `Tier3*` class identities → uncaught traceback; fixed +by homing them in `sessions.py`; unit tests can't catch this, the live smoke did); `TurnEvent` `turn_id` ABSENT +on text/thinking frames; `consumer_key` is BOUND-create-only; envelope parser prefers nested `detail`. +**NEXT = slice-5** (characters + me/capabilities/models — remaining consumer reads); then slice-6 (admin: +`stream_admin_events` + `get_session_bifrost`, admin_auth), slice-7 (teardown: retire contracts #2/#15, drop +`httpx-sse`, MINOR bump per DEC-6 w/ operator approval). Scope: consumer layer ONLY; Bifrost provider untouched. +Full design → auto-memory `project_worldtree_sdk_cutover`. -**⏸️ DEFERRED — tier3 agents `model`→`role` (scope B), folds into cutover slice-4.** WT renamed the -agents-RESPONSE selector `model`→`role` (spec 1.2). **DEPLOY NOW LIVE** on :8080/:8081 (v1.0.0b128, -worldtree-dev confirmed 2026-07-19 — was the deploy-flag gate; acked). Operator chose scope B (full tier3 -`model`→`role` incl. contract #15 + CLI `--model`→`--role`); lands in cutover slice-4 where tier3.py routes -through the SDK (doing it standalone now = throwaway). Auto-memory `project_tier3_agents_model_to_role_pending`. +**✅ RESOLVED — tier3 agents `model`→`role` (scope B) folded into cutover slice-4** (`c62b4ee`, v0.21.13). The +deferred deploy-gated scope-B work (response `model`→`role` per spec 1.2 / b128, `LocalAgentEntry`, index schema +v2) landed with the agents-family SDK cutover — no longer pending. See the slice-4 detail file + Recent decisions. **✅ RESOLVED — the "app product" workstreams leave Rata entirely (operator 2026-07-18).** **No arbo fork, no SillyTavern-on-Rata** — a NEW repo (template-dev standing up) takes over BOTH @@ -117,13 +117,13 @@ Full record → `persistent-memory.d/2026-07-18-368-silo-test-passed.md`. Siblin (2) R39 Phase-2 **matched-quartets rebuild** (confirmatory, "whenever"); (3) bifrost **snapshot-cursor adoption** (ruled normative, not blocking → `persistent-memory.d/2026-07-16-bifrost-cursor-conformance.md`). -**Substrate / environment:** branch `main` at **v0.21.12**, **PUSHED to origin** (slice-3 arc `ca9a339`+ -`fc256bb` + this snapshot, tags v0.21.11–.12, pushed 2026-07-19; slice-1+2 arc `b1fbadd`→`aba1730` + -v0.21.3–.10 earlier). origin `git@gitea.phasefinal.com:vh/ratatoskr.git`. **NEW core dep: +**Substrate / environment:** branch `main` at **v0.21.15** — slice-4 arc `c62b4ee`→`477d98f` + this snapshot +**COMMITTED, not-yet-pushed** (tags v0.21.13–.15; push is the operator's call); slice-1–3 (v0.21.3–.12, +`b1fbadd`→`4f92a21`) PUSHED to origin earlier. origin `git@gitea.phasefinal.com:vh/ratatoskr.git`. **NEW core dep: `worldtree-sdk==1.0.0`** (gitea PyPI, `[tool.uv.sources]`; `httpx-sse` retires at slice-7). bifrost **`==1.1.4`** / wire v0.7; WT openapi vendored 2.3.0, **conversation-api-spec re-synced to v1.1** (`b4a278c`); -**suite 469 green** (was 497 post-slice-2; net delta = slice-3 adapter/probe tests added, ~50 deleted -hand-rolled sessions tests). Personal WT on **b128** +**suite 475 green** (slice-4 added the agents-family adapter tests + heid-gate fixup tests, ~offset by the +deleted hand-rolled agent/persona tests). Personal WT on **b128** (`http://10.250.50.152:8081`; #368 silo + #364 promotion-hygiene live both instances). The combined **:8392** provider (memory+affect) + **:8765** web are THE surfaces, dev-box BACKGROUND SHELLS — restart via `scratchpad/relaunch_by_pid.py ` (pid via `ss -ltnp | grep `). `env.sh` sets @@ -274,6 +274,10 @@ decision. Captures rationale that won't be obvious from code alone. - `[2026-07-19]` **worldtree-sdk cutover SLICE-1 + SLICE-2 COMPLETE + PUSHED (origin `aba1730`, v0.21.10, 497 green).** The biggest, riskiest cutover slice done end-to-end through the full House Code Discipline (adapter→cli→web→delete→live-smoke→both heid gates); the bug-hunt caught 4 real confirmed bugs the conformance lens couldn't, and a convergent code-review finding was correctly REJECTED as category-5 (SDK envelope-parser behavior). Slice-3 next. → `persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-1-2-complete.md` - `[2026-07-19]` **worldtree-sdk cutover SLICE-3 COMPLETE + pushed (`ca9a339`+`fc256bb`, v0.21.11–.12, 469 green).** Persona/authored-history/first-message onto the wt adapter; last hand-rolled `sessions.py` paths deleted; both heid gates cleared — code-review ZERO drift, bug-hunt caught + fixed a real regression I shipped (ConnectFailed escaping both rewired probes → uncaught crash; the SDK normalizes ALL transport failures to `ConnectFailed`, not raw httpx) plus a finite-PAD chokepoint gap. Slice-4 (agents/tier3 + model→role) next. → `persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-3-complete.md` +- `[2026-07-19]` **worldtree-sdk cutover SLICE-4 COMPLETE (agents/Tier-3 + `model`→`role` fold, `c62b4ee`+`aed9429`+`477d98f`, v0.21.13–.15, 475 green).** Full House Code Discipline; the LIVE SMOKE caught a `python -m` double-module exception-class-identity bug unit tests structurally can't; both heid panels cleared (code-review zero-drift + 3 fixups; bug-hunt 5/5/5 → open-world-presenter degrade-not-crash fixes). Resolves the deferred scope-B model→role. Slice-5 next. → `persistent-memory.d/2026-07-19-worldtree-sdk-cutover-slice-4-complete.md` + +- `[2026-07-19]` **wyrd-dev #368 silo-enforcement consult delivered — ratatoskr's store-side silo is CONVENTIONAL (query-time filter); wyrd going STRUCTURAL off the framing.** Answered artifact-only from the provider memory-store code; wyrd folded my foot-guns into their unit-4 contract + committed to one-DB-file-per-campaign + first-class partition columns. OPEN LOOP: I'll eyeball their data model once the contract's cut (they'll ping). → `persistent-memory.d/2026-07-19-wyrd-368-silo-consult-delivered.md` + _67 older entries (2026-05-* debug-TUI/web era + the 2026-06-14 → 06-18 Bifrost-provider build / #17+#18 / #295-296 era) archived to archival-memory.md._ _For per-issue TDD implementation notes, Volva findings, and contract amendments, see the git log — every per-issue commit carries a structured message capturing the trail._