memory: snapshot — althing v3 fleet-wide at 3.1.1, sec on GPU0

The in-flight section was a day stale: it still described run 3c as the
live subject on a box where nothing had moved. Rewritten around what is
actually true now -- v3 deployed fleet-wide, the post office relocated
to nh3-docker, sec serving on GPU0, run 3c still held on power.

Six new decision entries, three of which carry findings that outlive
their incident: the inbound half of the handle-resolution bug (a stale
ALTHING_HANDLE reads another agent's mailbox and reports it empty,
which is a second route into the failure v3 exists to prevent), the OOM
attribution to Claude Code sessions, and the operator's two explicit
belays recorded so a later session does not re-raise them as new.

Auto-archival fired at 301 lines and moved exactly one entry. Three
others were old enough and every one carries a still-open deferred
pointer -- the parked CI flip, muninn-gate's submit path, and the triton
backend deferred to the Ada refresh. Held back per the guards; an
over-cap file that keeps live decisions beats a scannable one that lost
them. The entry that did move had its deferred item closed today:
nh3-extdev's staged v2.1.2 wheel is moot now that the box runs 3.1.1.

persistent-memory.d/2026-08-28-althing-v3-cutover.md

@@ -148,3 +148,71 @@ re-verified end to end.
 means repeating that by hand. The fleet pattern (skaldsong, soong-lab) is a gitea registry pull;
 this should join it. The old nh3-dev volume is left in place untouched — not a rollback path
 (the operator ruled that out), just not deleting the only other copy on the day of the move.
+
+---
+
+## `[2026-08-28, later]` The release train: 3.0.1 → 3.1.1 in one afternoon, and the registry
+
+Six releases landed the same day as the cutover. **The container was touched exactly once** (the
+nh3-docker move); every other release was herald- or client-side, established each time by
+forseti's **import-graph argument** — asking what `althing/post_office/*` IMPORTS rather than
+reading the diff. A diff tells you what moved; an import graph tells you what can be affected.
+
+    3.0.1  pane routes (a pane agent can register its own route)
+    3.0.2  the docs are now checked against the CLI, not against each other  — NO redeploy
+    3.0.3  PANE_SETTLE_S=0.3, the write/submit race
+    3.1.0  herald writes $ALTHING_ROOT/post-office; dev-launch reads it  — BOTH binaries
+    3.1.1  warn when ALTHING_HANDLE disagrees with launch-history
+
+**Deploy recipe per release:** `uv build --wheel` on nh3-dev → `uv tool install --force .`
+locally → stage the wheel to nh3-extdev and install under `UV_TOOL_DIR=/opt/uv-tools` with
+`/home/infra-ops/.local/bin/uv` → restart both `althing-po-herald` units.
+`playbooks/nh3-extdev-althing-v3.yaml` does the extdev half.
+
+## ⚠ THE REPEATED DEFECT — a verify that half-passes, four costumes in one day
+
+Every one of these was written correctly for its first run and silently wrong on the next:
+
+    1. install step gated on `postbox` not existing   -> right for the cutover, would SKIP
+                                                          every release after and report success
+    2. content check pinned to the PREVIOUS release's markers -> passes forever, asserts nothing
+    3. ONE file:marker pair against a TWO-file release  -> asserts half a release
+    4. a Go template `{{ }}` inside elway, whose own substitution ate it -> FAILED on a green deploy
+
+**The check now matches on PRESENCE (`grep -q`), never a count**, and takes a LIST of
+`file:marker` pairs bumped per release. 3.1.1's own release note said
+`grep -c handles_launched_at dev_launch.py # 2+`; the real count there is 1 (the definition,
+with 2 in postbox.py), so a count assertion would have reported FAILED on a byte-perfect install.
+
+## The registry, and why the namespace is `claude-bot`
+
+    gitea.phasefinal.com/claude-bot/althing-post-office:3.0.0@sha256:410fed41...
+
+Digest-pinned, not tag-floating — a tag is a mutable pointer on a registry anyone can re-push and
+this container is the whole bus. ⚠ **claude-bot's token carries `write:package` and `docker
+login` SUCCEEDS, but package namespaces are owned**: pushing to `vh/` returns
+`unauthorized: authentication required` AFTER a successful login — an ownership refusal wearing a
+credential error's clothes. Publishing under claude-bot's own namespace also satisfies the
+standing directive to stop reusing the operator's personal credentials.
+
+## The deployed CC plugin copies are a release step NOBODY owns
+
+`sync_skill.sh` covers the SKILL, not the plugin. Nothing in the repo reaches
+`~/.local/share/althing-plugin/` or `~/.claude/plugins/cache/althing/althing/0.0.1/`. Both must be
+`rsync -a --delete`'d from the repo's `plugin/` **on every release**, by hand, or they carry the
+previous release's bugs into the live surface — which happened: forseti's new docs-vs-CLI check
+found `postbox reply --to` twice in `plugin/commands/inbox.md`, **the file a CC session reads
+every time it drains its inbox**, and both my deployed copies had it.
+
+⚠ **A running CC session keeps the plugin text it loaded at startup.** Files being right is
+necessary and not sufficient — the session has to restart. I synced the copies at 09:30 and was
+handed the v2 `/althing:monitor` text an hour later, calling three deleted binaries.
+
+## Peer skill copies: I stopped hand-syncing, deliberately
+
+I wrote into eitri/dvalin/bil's repos three times (operator-authorised, and right while they were
+dark and could not pull). **Once they were awake and pulling, it became a race I was losing** —
+canonical moved three times in an hour and I was chasing a one-line version-banner lag. It also
+cost provenance: dvalin had to correct their own account of their file because my write looked
+like a pre-existing partial. Two writers, no lock. `sync_skill.sh` deliberately does not write
+into peer trees; althing owns canonical and peers pull. Respect that boundary once they can.

persistent-memory.d/2026-08-28-handle-resolution-wrong-inbox.md

@@ -0,0 +1,60 @@
+# `[2026-08-28]` A stale `ALTHING_HANDLE` silently reads ANOTHER agent's inbox and reports it empty
+
+Found by pewpew-dev, whose session posted as **forseti** all day. Mechanism:
+
+    ALTHING_HANDLE inherited from the environment
+    + nothing binding a shell to the handle it may query
+    = the env var wins, silently, with no warning and no error
+
+Measured, same shell, same second, no credential, no complaint:
+
+    ALTHING_HANDLE=infra-ops    postbox status  -> infra-ops' mailbox
+    ALTHING_HANDLE=forseti      postbox status  -> forseti's mailbox
+    ALTHING_HANDLE=pewpew-dev   postbox status  -> pewpew-dev's mailbox
+
+## ⚠ THE INBOUND HALF IS THE SERIOUS ONE, AND IT IS INVISIBLE
+
+**Outbound** mis-signing sometimes gets caught: a peer notices the sender cannot hold that
+context — which is exactly how this surfaced, when forseti was asked a pewpewstudio question.
+
+**Inbound never does.** A session with a stale handle runs `postbox peek`, reads SOMEONE ELSE'S
+mailbox, and is told — confidently, correctly, nothing broken anywhere — that it has nothing
+unread. Measured cost: my reply sat unread until pewpew-dev's operator asked whether they were
+blocked on me.
+
+**This is a SECOND route into the failure v3 exists to prevent.** The guarantee — *"an
+unreachable post office is an OUTAGE, never an empty inbox"* — holds, and does not cover this:
+the post office is reachable and answers correctly, **about someone else**. Nothing is down, so
+the outage semantics never fire. forseti has stopped describing that guarantee as though it
+closes the empty-inbox class; it closes one route into it.
+
+## ⚠ MY GROUNDING WAS WRONG, AND THE CORRECTION MADE IT WORSE
+
+I measured "41 of 79 project dirs mapped in `~/.althing/session_handles.json`" and called the 46
+unmapped ones exposed. **`session_handles.json` is a v2 artifact that v3 never opens** —
+`grep -rn session_handles althing/` is empty, and `postbox.resolve_config` takes `--handle` then
+`ALTHING_HANDLE` and nothing else. So the exposure is LARGER than my number implied: every
+directory is in the same position, because the map is consulted for none of them.
+
+**The same stale data source had survived inside my statusline rewrite that morning.** I updated
+the v2 block's COMMANDS and kept its DATA SOURCE — the commands were the visible half of the
+cutover and the source was not. Fixed (`8a04d6f`): it reads `launch-history.json` now (written by
+`dev_launch`, shape `{cwd: {command: {at, handle}}}`, take the most recent by `at`).
+
+## The fix that shipped, and why my proposal was the worse one
+
+I proposed printing the resolution source (`handle: forseti (from ALTHING_HANDLE)`). forseti
+killed it with one observation: **`postbox status` already prints the handle, first field, every
+call.** pewpew-dev had `forseti` on screen and it did not register. **The information was never
+missing; the salience was.** That generalised into the rule that chose the design: a line that is
+always there teaches the reader to skip it, so a warning beats a field.
+
+Shipped as **3.1.1** — warn (never refuse; a legitimate cross-project send is real) when the cwd's
+`launch-history` names a different handle, on stderr, BEFORE the output it is about. Keyed by
+directory across commands, not `(directory, command)` — one repo hosting a claude and a codex
+session under different handles is normal, and a warning that fires on a legitimate case is one
+people learn to ignore.
+
+⚠ **Its silence is not an all-clear:** 8 of 79 directories are in `launch-history.json`, so the
+quiet case is ~90% of the box. forseti put that caveat in a TEST NAME
+(`test_an_unlaunched_directory_says_nothing`) because prose gets skimmed and a test goes red.

persistent-memory.d/2026-08-28-sec-seat-gpu0.md

@@ -0,0 +1,35 @@
+# `[2026-08-28]` The `sec` pen-test seat moved GPU1 → GPU0 and came up — on a circuit that tripped 36h earlier
+
+Operator-directed. `sec` = M.O.G.-SEC-27B (`stacks/mog-sec/`, LiteLLM aliases `sec` /
+`sec-reasoning`, ana-ml2 `:8019`, 262K native ctx, NVFP4+FP8 mixed with a grafted MTP head).
+
+## Why it had to move
+
+    GPU1  69,895 MiB used of 97,887 — gen 46 GB + embed 9.8 + coder 8.4 + rerank 3.5 + reward 2.1
+          -> ~28 GB free, against the ~51 GB this seat reserves at MOG_GPU_MEM_UTIL=0.52
+          -> it could not start on GPU1 AT ALL
+    GPU0  empty since run 3c was stopped 2026-08-26
+
+One line: `MOG_GPU_ID=1 -> 0` in `/opt/docker/compose/mog-sec/.env`. The compose default, the
+header comment and the **homepage label** all named GPU 1 and were corrected in the same change —
+a label naming the wrong card is a record that lies about where the work runs.
+
+## Landed state
+
+    container   vllm-mog-sec, healthy after ~400s load (22 GB model off the DEGRADED /tank)
+    GPU0        51,532 MiB, idle draw 16.34 W
+    GPU1        69,895 MiB, idle draw  6.42 W
+
+⚠ **This re-arms the two-GPU load condition that tripped the Anaheim rack breaker on 08-26.**
+Idle draw is negligible — ~23 W across both cards. **The risk materialises only under concurrent
+load**, when both seats work at once and the box approaches the ~600 W that tripped it. The
+operator accepted that with the constraint stated. See
+[[2026-08-27-anaheim-breaker-and-onboot-gap]] — one circuit feeds the whole rack including ana-gw
+and ana-wg, so a trip costs the site AND the way back in.
+
+## Deploy gotchas worth keeping
+
+- `up -d`, never `restart` — **labels bind at container creation**, so a restart keeps the old
+  homepage label and the dashboard silently keeps showing the old GPU number.
+- Diff deployed-vs-canonical BEFORE pushing. There was no drift here, which is the only reason
+  the push was safe to make blind.