scripts: wt-h2-count.py — worldtree-dev's U11b H2 gate check (verbatim); rehearsed on copies, 0 on both

persistent-memory.md

@@ -127,7 +127,7 @@ _As of 2026-09-30 ~0120 PT._
   - **PERSONAL flipped at 0120 PT** on 64f79b38a7fb, the same way. ⚠ Personal has **no /metrics**, because the #308 metrics block is demo-only, so the proof was the no-metrics one: an in-container `get_default→load_cutover_config→validate_cutover` gave off/writer/reader True, /health was 200, and the boot log matched the pre-flip one. Whether to add metrics to personal is still worldtree-dev's open call. Both instances are in sync with the repo.
   - Watch after each flip: any `legacy executor <kind> invoked with memory.legacy.mode=off; ignored` line goes to worldtree-dev with its stamp. A sweep at 0154 of both logs since their last start found **0 watched lines, but there was ~no traffic** (demo 0 real requests, personal 1), so that proves little. **TODO: re-sweep after the first real traffic** with `docker logs --since <StartedAt>`, grepping `legacy executor|Traceback|ERROR|will not be remembered`. A CI recreate drops the old container's log.
 - **Daily gate batches at off: LIVE, infra-hermes from 2026-10-01** (Prime's 2340 ruling; the plan was revised to `--legacy-mode off --label window-off` by worldtree-dev, thread `01M3RRW7WZ04GNR871HC1VKY4K`). The wrapper is `scripts/wt-memory-gate-batch`. It tests demo's deployed sha, from the detached worktree `~/development/Worldtree-gate` when worldtree-dev's tree has moved. Exit codes: 0 PASS, 1 FAIL, 2 error, 3 refused, 4 busy. **PASS and FAIL both go to worldtree-dev the same day**, because they build the n. **The U11b DATA deletion is gated on 3 consecutive PASS at off, or a Prime waiver**; code retirement is not gated. The controls were 080927Z FAIL and 082829Z FAIL (one flip each; the filing floor held; worldtree-dev calls it undecided). **Mine, 20260930T090608Z, PASSED (user median 0.83)**, so the count is 1 of 3 if the FAILs reset it. ⚠ It started by accident, from a test meant to be `--dry-run`, and matched the plan anyway.
-- **⚠ U11b STEP 5 IS MINE, AUTO-TRIGGERED (Prime ruled 0320 via worldtree-dev, thread `01M3SGEQDRQD7DWBVT4K73FAHP`):** once the THIRD CONSECUTIVE PASS at off lands (infra-hermes copies me on every verdict; 20260930T090608Z was 1 of 3; a FAIL restarts the count), I re-run the H2 global agent-self=0 check (asked worldtree-dev for the exact procedure; do not improvise it), then delete on BOTH instances, using literal paths only: `agents/{forseti,lofn,mimir}/memory/<agent>.chroma` + `memory/context_promotion`. Default: stop the api, delete, start it (asked worldtree-dev whether b192 re-creates the dir). Then send worldtree-dev the stamp, because b193 ships after it. The 377 parked rows go with it. The archive stays under its rev 1.2 rules. After b193: remove the retired config keys at my pace.
+- **⚠ U11b STEP 5 IS MINE, AUTO-TRIGGERED (Prime ruled 0320 via worldtree-dev, thread `01M3SGEQDRQD7DWBVT4K73FAHP`):** once the THIRD CONSECUTIVE PASS at off lands (infra-hermes copies me on every verdict; 20260930T090608Z was 1 of 3; a FAIL restarts the count), I run the H2 check VERBATIM, per instance, right before its deletion: `docker exec -i <api> python - < scripts/wt-h2-count.py`. It is worldtree-dev's script, and exit 2 means STOP and send them the output. A rehearsal on `cp -a` copies on 09-30 read 0 on both (demo 4/12/5 rows, personal 0/6/2200). Then I delete on BOTH instances, using literal paths only: `agents/{forseti,lofn,mimir}/memory/<agent>.chroma` + `memory/context_promotion`. **I delete LIVE with the api running** (worldtree-dev, read at source on b192: at off, nothing holds those files open). ⚠ Any b192 restart re-creates an empty, schema-only `context_promotion/ledger.db` (+wal/shm). That is residue, not memory data; note it in the stamp and rm it after b193 is deployed. Then send worldtree-dev the stamp, because b193 ships after it. The 377 parked rows go with it. The archive stays under its rev 1.2 rules. After b193: remove the retired config keys at my pace.
 - **U11b prep (worldtree-dev asks, read-only, answered 0215):** /embed usage from the Skuld ledger (`phase_name='embed'`, which carries no caller identity): demo 436 calls, 07-29..08-31, none since; personal 5 calls, 07-30..08-05. ⚠ Demo's whole Skuld ledger has been idle since 09-14. Legacy data: demo 3 chroma ≈2.1M plus context_promotion 224K; personal ≈26M plus 6.5M (1,413 JSONL + ledger.db). Both are in the `*_worldtree-state` volumes. **ARCHIVE DONE 2026-09-30 0957 PT (worldtree-dev GO), restore drill passed.** The copies:
   - corviduo-dev `/var/lib/wt-legacy-archive/` (root 0700, UNENCRYPTED): demo 51 files and personal 1,426 files, each as tar.zst plus per-file sha256 plus MANIFEST.txt.
   - A DEDICATED restic repo, rest-server-nh3 `/nh3-dev/wt-legacy-archive/`, snapshot `98dc64e0`, with its password vaulted at `nh3-dev/wt-legacy-archive/restic-password`. Its URL is the vaulted `nh3-dev/etc/restic/repository` plus `wt-legacy-archive/`. It is mirrored to ana-nas at 05:00.

scripts/wt-h2-count.py

@@ -0,0 +1,46 @@
+"""H2 re-check: count legacy chunks with scope == agent_self per agent store.
+
+Provenance: written by worldtree-dev against the v1.0.0b192 tag and delivered
+2026-09-30 (thread 01M3SGEQDRQD7DWBVT4K73FAHP) for the U11b step-5 gate. Run it
+VERBATIM, right before each instance's legacy-data deletion, inside that
+instance's worldtree-api container (it pins the chromadb that wrote the stores):
+
+    docker exec -i <worldtree-api container> python - < scripts/wt-h2-count.py ; echo "exit=$?"
+
+Exit 0 = global count 0, go. Exit 2 = non-zero: STOP, send the output to
+worldtree-dev before deleting anything. Attach the full stdout to the stamp.
+
+⚠ chromadb rewrites HNSW sidecars and the sqlite file ON OPEN: the store's
+bytes change even though no row does. Compare live sha256s against the archive
+manifest BEFORE running this, or run it against a `cp -a` copy (point
+WORLDTREE_DATA_PATH at the copy's root).
+
+Opens each <WORLDTREE_DATA_PATH>/agents/<agent>/memory/<agent>.chroma with
+chromadb and counts rows whose metadata scope is "agent_self" (the legacy
+reader's own where-clause). A missing store dir or collection counts as 0
+and says so. Exit 0 when the global total is 0, exit 2 otherwise.
+"""
+import os, sys
+from pathlib import Path
+import chromadb
+
+root = Path(os.environ["WORLDTREE_DATA_PATH"])
+agents = sys.argv[1:] or ["forseti", "lofn", "mimir"]
+total = 0
+for agent in agents:
+    d = root / "agents" / agent / "memory" / f"{agent}.chroma"
+    if not d.is_dir():
+        print(f"{agent}: store absent ({d}) -> agent_self=0")
+        continue
+    client = chromadb.PersistentClient(path=str(d))
+    try:
+        col = client.get_collection("memory_chunks")
+    except Exception as exc:
+        print(f"{agent}: no memory_chunks collection ({type(exc).__name__}) -> agent_self=0")
+        continue
+    n_all = col.count()
+    n_self = len(col.get(where={"scope": "agent_self"}, include=[])["ids"])
+    total += n_self
+    print(f"{agent}: rows={n_all} agent_self={n_self}")
+print(f"H2 global agent_self count: {total}")
+sys.exit(0 if total == 0 else 2)