diff --git a/persistent-memory.md b/persistent-memory.md index f894650..05a6d49 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -127,7 +127,7 @@ _As of 2026-09-30 ~0120 PT._ - **PERSONAL flipped at 0120 PT** on 64f79b38a7fb, the same way. ⚠ Personal has **no /metrics**, because the #308 metrics block is demo-only, so the proof was the no-metrics one: an in-container `get_default→load_cutover_config→validate_cutover` gave off/writer/reader True, /health was 200, and the boot log matched the pre-flip one. Whether to add metrics to personal is still worldtree-dev's open call. Both instances are in sync with the repo. - Watch after each flip: any `legacy executor invoked with memory.legacy.mode=off; ignored` line goes to worldtree-dev with its stamp. A sweep at 0154 of both logs since their last start found **0 watched lines, but there was ~no traffic** (demo 0 real requests, personal 1), so that proves little. **TODO: re-sweep after the first real traffic** with `docker logs --since `, grepping `legacy executor|Traceback|ERROR|will not be remembered`. A CI recreate drops the old container's log. - **Daily gate batches at off: LIVE, infra-hermes from 2026-10-01** (Prime's 2340 ruling; the plan was revised to `--legacy-mode off --label window-off` by worldtree-dev, thread `01M3RRW7WZ04GNR871HC1VKY4K`). The wrapper is `scripts/wt-memory-gate-batch`. It tests demo's deployed sha, from the detached worktree `~/development/Worldtree-gate` when worldtree-dev's tree has moved. Exit codes: 0 PASS, 1 FAIL, 2 error, 3 refused, 4 busy. **PASS and FAIL both go to worldtree-dev the same day**, because they build the n. **The U11b DATA deletion is gated on 3 consecutive PASS at off, or a Prime waiver**; code retirement is not gated. The controls were 080927Z FAIL and 082829Z FAIL (one flip each; the filing floor held; worldtree-dev calls it undecided). **Mine, 20260930T090608Z, PASSED (user median 0.83)**, so the count is 1 of 3 if the FAILs reset it. ⚠ It started by accident, from a test meant to be `--dry-run`, and matched the plan anyway. -- **⚠ U11b STEP 5 IS MINE, AUTO-TRIGGERED (Prime ruled 0320 via worldtree-dev, thread `01M3SGEQDRQD7DWBVT4K73FAHP`):** once the THIRD CONSECUTIVE PASS at off lands (infra-hermes copies me on every verdict; 20260930T090608Z was 1 of 3; a FAIL restarts the count), I re-run the H2 global agent-self=0 check (asked worldtree-dev for the exact procedure; do not improvise it), then delete on BOTH instances, using literal paths only: `agents/{forseti,lofn,mimir}/memory/.chroma` + `memory/context_promotion`. Default: stop the api, delete, start it (asked worldtree-dev whether b192 re-creates the dir). Then send worldtree-dev the stamp, because b193 ships after it. The 377 parked rows go with it. The archive stays under its rev 1.2 rules. After b193: remove the retired config keys at my pace. +- **⚠ U11b STEP 5 IS MINE, AUTO-TRIGGERED (Prime ruled 0320 via worldtree-dev, thread `01M3SGEQDRQD7DWBVT4K73FAHP`):** once the THIRD CONSECUTIVE PASS at off lands (infra-hermes copies me on every verdict; 20260930T090608Z was 1 of 3; a FAIL restarts the count), I run the H2 check VERBATIM, per instance, right before its deletion: `docker exec -i python - < scripts/wt-h2-count.py`. It is worldtree-dev's script, and exit 2 means STOP and send them the output. A rehearsal on `cp -a` copies on 09-30 read 0 on both (demo 4/12/5 rows, personal 0/6/2200). Then I delete on BOTH instances, using literal paths only: `agents/{forseti,lofn,mimir}/memory/.chroma` + `memory/context_promotion`. **I delete LIVE with the api running** (worldtree-dev, read at source on b192: at off, nothing holds those files open). ⚠ Any b192 restart re-creates an empty, schema-only `context_promotion/ledger.db` (+wal/shm). That is residue, not memory data; note it in the stamp and rm it after b193 is deployed. Then send worldtree-dev the stamp, because b193 ships after it. The 377 parked rows go with it. The archive stays under its rev 1.2 rules. After b193: remove the retired config keys at my pace. - **U11b prep (worldtree-dev asks, read-only, answered 0215):** /embed usage from the Skuld ledger (`phase_name='embed'`, which carries no caller identity): demo 436 calls, 07-29..08-31, none since; personal 5 calls, 07-30..08-05. ⚠ Demo's whole Skuld ledger has been idle since 09-14. Legacy data: demo 3 chroma ≈2.1M plus context_promotion 224K; personal ≈26M plus 6.5M (1,413 JSONL + ledger.db). Both are in the `*_worldtree-state` volumes. **ARCHIVE DONE 2026-09-30 0957 PT (worldtree-dev GO), restore drill passed.** The copies: - corviduo-dev `/var/lib/wt-legacy-archive/` (root 0700, UNENCRYPTED): demo 51 files and personal 1,426 files, each as tar.zst plus per-file sha256 plus MANIFEST.txt. - A DEDICATED restic repo, rest-server-nh3 `/nh3-dev/wt-legacy-archive/`, snapshot `98dc64e0`, with its password vaulted at `nh3-dev/wt-legacy-archive/restic-password`. Its URL is the vaulted `nh3-dev/etc/restic/repository` plus `wt-legacy-archive/`. It is mirrored to ana-nas at 05:00. diff --git a/scripts/wt-h2-count.py b/scripts/wt-h2-count.py new file mode 100644 index 0000000..17640cf --- /dev/null +++ b/scripts/wt-h2-count.py @@ -0,0 +1,46 @@ +"""H2 re-check: count legacy chunks with scope == agent_self per agent store. + +Provenance: written by worldtree-dev against the v1.0.0b192 tag and delivered +2026-09-30 (thread 01M3SGEQDRQD7DWBVT4K73FAHP) for the U11b step-5 gate. Run it +VERBATIM, right before each instance's legacy-data deletion, inside that +instance's worldtree-api container (it pins the chromadb that wrote the stores): + + docker exec -i python - < scripts/wt-h2-count.py ; echo "exit=$?" + +Exit 0 = global count 0, go. Exit 2 = non-zero: STOP, send the output to +worldtree-dev before deleting anything. Attach the full stdout to the stamp. + +⚠ chromadb rewrites HNSW sidecars and the sqlite file ON OPEN: the store's +bytes change even though no row does. Compare live sha256s against the archive +manifest BEFORE running this, or run it against a `cp -a` copy (point +WORLDTREE_DATA_PATH at the copy's root). + +Opens each /agents//memory/.chroma with +chromadb and counts rows whose metadata scope is "agent_self" (the legacy +reader's own where-clause). A missing store dir or collection counts as 0 +and says so. Exit 0 when the global total is 0, exit 2 otherwise. +""" +import os, sys +from pathlib import Path +import chromadb + +root = Path(os.environ["WORLDTREE_DATA_PATH"]) +agents = sys.argv[1:] or ["forseti", "lofn", "mimir"] +total = 0 +for agent in agents: + d = root / "agents" / agent / "memory" / f"{agent}.chroma" + if not d.is_dir(): + print(f"{agent}: store absent ({d}) -> agent_self=0") + continue + client = chromadb.PersistentClient(path=str(d)) + try: + col = client.get_collection("memory_chunks") + except Exception as exc: + print(f"{agent}: no memory_chunks collection ({type(exc).__name__}) -> agent_self=0") + continue + n_all = col.count() + n_self = len(col.get(where={"scope": "agent_self"}, include=[])["ids"]) + total += n_self + print(f"{agent}: rows={n_all} agent_self={n_self}") +print(f"H2 global agent_self count: {total}") +sys.exit(0 if total == 0 else 2)