esh-pfi-infrastructure/persistent-memory.d/2026-09-01-althing-320-deploy.md

[2026-09-01] althing 3.2.0 deployed on nh3-dev — pane routes, and uv tool install . lies

Operator-approved, tagged v3.2.0 at c4ede0f on master. forseti authored; infra-ops deployed. Host work, nh3-dev only.

What 3.2.0 changes

A Claude Code session in a zellij pane is now poked in its own pane instead of through a detached althing-listen waiter, because CC 2.1.257 reaps detached tasks whenever the seat sits in shell — the seat is targeted, not the task, so no amount of hardening the waiter helps. A pane route is a file; there is no process to reap.

⚠ Independently corroborated here. ~30 min before forseti's request landed, infra-ops' own althing-listen ended with status killed — not exit 0/2/3, no mail, inbox empty, seat silently back to mode: pull. Written up at the time as "cause unknown, probably a harness reaper" rather than guessed at. 2.1.257 is the mechanism.

The deploy — three steps, order matters

22:13:xx   uv tool install --force .                    3.1.2 -> 3.2.0, 7 executables
22:13:44   systemctl --user restart althing-po-herald   pid 180370 -> 2850064
22:14:07   scripts/sync_skill.sh                        3.1.0 text -> 3.2.0, --check clean

althing-route is the seventh console script and the whole point — the plugin's new SessionStart hook calls it by name, so on 3.1.2's six binaries the hook correctly degrades to silence and every seat stays pull-only. Nothing breaks by waiting; nothing works until it lands. Skill sync must come after the install or it documents a binary that is not there.

⚠ NOT --editable (standing rule): an editable install puts the working tree on sys.path, so a branch switch changes what every running agent's tooling does with no deploy step in between.

⚠ THE TRAP — uv tool install . IS A SILENT NO-OP ON A REINSTALL

$ uv tool install .
`althing-core @ file:///home/lkraven/development/althing` is already installed
$ echo $?
0

uv matches on the source SPEC, not the source CONTENTS. On a box where the tool was already installed from that same path, it declines and reports success. Nothing says "I did not do the thing you asked."

Always uv tool install --force . when reinstalling from a local path. Generalises the nh3-extdev lesson (reference_nh3_extdev_althing_mesh) from the system-wheel case to every local-path uv tool. Same shape as feedback_assert_effective_value_not_substring: the command exited 0 and the state was unchanged.

Fleet-visible: 4 pane routes revoked, and the count to trust

Pane routes now carry started_at (declaring pid's start time, so a recycled pid cannot inherit a route) and command (so a pane that has dropped to a shell stops being poked). Routes without them are observed and excluded with a reason in the herald log, not silently ignored — this closes a real hazard where the herald would write a line into whatever shell inherited an exited agent's pane and press Enter, running whatever was half-typed there.

Verified by splitting ~/.althing/routes on channel= before the restart — all twelve route files lack the new fields (3.1.2 never wrote them), so a field audit over-counts; only pane routes are affected:

pane: 4  -> bil-smithy-dev, dvalin-smithy-dev, eitri-smithy-dev, regin-smithy-dev
fifo: 8  (+ 9 fifos in ~/.althing/wake/)

forseti's list of four matched exactly. ⚠ Their note said "the twelve fifo seats are untouched" — twelve is the total route count; fifo is 8. Untouched either way.

Re-declare is one command per agent: althing-route declare --pid <long-lived agent pid>.

Not notified. Four targeted messages about a change none of them asked about reads as a broadcast however addressed → feedback_notify_before_tearing_down_a_seat. Surfaced to the operator instead; the herald logs the exclusion every tick so it is discoverable.

⚠ Log-noise papercut raised with forseti (not a bug): the exclusion lines repeat every tick (~26 s), so the four stale routes emit ~140 lines/hour, ~3,300/day until re-declared — enough to bury real signal in the log you reach for when a seat misbehaves. Suggested a back-off after N consecutive exclusions. Their call.

✅ infra-ops MIGRATED TO A PANE ROUTE — stop arming waiters on this seat

althing-route declare --discover-pid
-> infra-ops -> Claude:terminal_2 (claude --allow-dangerously-skip-permissions --resume)

pid=2175612  started_at=38608652  channel=pane  target=Claude:terminal_2
command=claude --allow-dangerously-skip-permissions --resume
postbox status -> mode: push  reachable: true

--discover-pid correctly walks the ancestry to the long-lived claude process, not the ephemeral bash that invoked it. Use it rather than passing --pid $$ — a shell pid dies with the tool call and the route dies with it.

⚠ reachable: true is a report, not a delivered poke. Proof is the next message arriving.

⚠ THE REAP HITS FIFO WAITERS TOO, AND postbox status CANNOT WARN YOU

infra-ops' seat is a fifo waiter, not a pane — and althing-listen was killed twice in one session (2026-09-01), the second time within minutes of being armed. Both times: unread: 0, no mail, seat silently back to mode: pull.

The failure mode worth naming is not "the waiter dies" — it is "the waiter dies AFTER confirming it is up." Both waiters reported push/reachable immediately after arming. So:

• A green postbox status is not durable evidence of monitoring. True when read, false minutes later, nothing in between to say so. → feedback_timestamp_every_state_report
• Anything still on althing-listen is monitored only until the reaper notices. The waiter path is unreliable for ANY seat sitting at a shell, not just pane-capable ones.
• Do not loop on re-arming. Two kills is the pattern; declare a pane route instead. Raised both consequences with forseti on the deploy thread.

✅ MIGRATION COMPLETE — 5/5 pane routes pinned by 22:26, all four re-declared in ~12 min

Operator directed notification; all four told individually with their own measured pid, pane and start time. Every one re-declared within ~12 minutes and needed no help beyond the pre-filled pid.

handle              pid       started_at   command      status
bil-smithy-dev      98493     1713267      kimi         push, poked via pane
dvalin-smithy-dev   96050     1691681      grok         push
eitri-smithy-dev    103662    1752089      node/codex   push
regin-smithy-dev    96864     1700952      pi           push, poked via pane
infra-ops           2175612   38608652     claude       push, poked via pane

Zero stale routes; last exclusion line 22:25:01, everything after is poked <handle> via pane.

⚠ FINDING — the herald's exclusion reason is FALSE for the migration case

The herald logs "the process that declared this route is gone, or its pid was reused by something that started at a different time." Neither disjunct was true for any of the four. Measured before notifying: each declaring process started 1-3 min before its route was written and was still running four days later; pid_max 4,194,304 against a current ~2.86M, so no wrap and no reuse to suspect. bil-smithy-dev independently confirmed it against pid 98493.

Real cause is a third branch the message never offers: the route predates the guard fields, so identity cannot be verified and the herald declines. Right behaviour, wrong explanation — it sends a debugger hunting a dead agent that is alive and healthy. ⚠ Do not trust that log line at face value. Raised with forseti with a suggested third branch.

⚠ MY MISS — I called an open risk on something the release author had already measured

I flagged "what does a pane poke do to a non-Claude TUI?" to forseti as an open risk on the release. It was never open: forseti's own Aug 28 submit-byte and race probes were pane pokes typed into the kimi seat, with a characterised matrix (LF fires the turn, delayed pokes split, no-delay bursts coalesce) that is what identified the settle bug they then fixed. regin had also smoke-tested 3.0.3 on the pi seat.

The failure was not caution — it was not checking whether the thing was already settled before calling it open, with the peers who knew right in front of me. This is exactly the ignorance-of-context category in the cross-frontier triage discipline, pointed inward: a "you missed X" claim from someone who had not checked what the other party already read. ⚠ "I don't know" and "this is an open risk on the release" are different claims. Saying the first was honest; asserting the second was not warranted. Retracted with forseti and with all three seats. → feedback_canonical_sourced_values

Verified: pane delivery works on Claude AND non-Claude seats

• infra-ops (claude) — poked infra-ops on nh3-dev via pane (rung 0), landed as own turns.
• regin-smithy-dev (pi) — first 3.2.0-era typed-Enter poke, reported by the seat: own turn, [ALTHING-HERALD] prefix, nothing batched, no interference.
• dvalin-smithy-dev (grok) — pre-3.2.0 pokes landed as own submitted turns, no leftover unsubmitted text.

⚠ ONE-TICK LATENCY LOOKS LIKE FAILURE

postbox status immediately after althing-route declare still reports mode: pull until the herald ticks (~26 s). Two of four hit it. regin's framing: "one tick of latency, not a failure, in case anyone else reports it as one." Asked forseti for a line in the skill.

Open with forseti (nothing blocking)

1. uv tool install . runbook fix (--force) — the one that would bite the next person.
2. Exclusion-message third branch.
3. --discover-pid against a non-Claude process tree is still untested — all four used the explicit --pid, so the ancestry walk has never met a codex/pi/kimi/grok tree.
4. One-tick-latency note in the skill.
5. postbox status push/reachable is not durable evidence.

Withdrawn: non-Claude delivery risk (was forseti's own prior work). Moot: log back-off — zero stale routes, the log went quiet on its own.

🔴 3.2.1 SAME NIGHT (22:40) — 3.2.0 WROTE INTO PANES PEOPLE WERE TYPING IN

The bug, which hit the operator within an hour of the 3.2.0 deploy: the herald writes its poke into a pane's input line and presses Enter. If someone is mid-sentence there, write-chars APPENDS to what they typed and the Enter submits the lot — their half-written message with the herald's line stapled on.

forseti's post-mortem, worth keeping verbatim because it generalises well past althing:

The contract named this hazard on one road only — a submit after a FAILED write — and I built three guards that all answer the same question, "is this the right pane". None of them asked whether it was a good MOMENT.

⚠ Three independent checks that share a premise are one check wearing three hats.

The fix — guard 4. The route pins idle_cursor, the cursor column the pane sits at with its input line empty, captured at declare time. Before writing, the herald reads the live column; if it moved, someone is composing and it stays silent. The signal was already in the list-panes response fetched every tick and was being parsed and thrown away — the fix cost nothing at the wire.

22:40:xx   uv tool install --force .    3.2.0 -> 3.2.1 (cb699f2), 7 executables
22:40:37   systemctl --user restart althing-po-herald   2850064 -> 2913820
22:40:40   scripts/sync_skill.sh        --check clean

⚠⚠ THE HOLE 3.2.1 LEAVES — 2 OF 5 PANE SEATS CANNOT BE GUARDED, EVER

forseti measured the idle columns; applied to the live route table this is not a footnote:

handle              TUI      guard 4?
infra-ops           claude   YES  idle_cursor=3  (pinned 22:41)
eitri-smithy-dev    codex    YES  expects 3
dvalin-smithy-dev   grok     YES  expects 7
bil-smithy-dev      kimi     NO   -- reports no cursor at all
regin-smithy-dev    pi       NO   -- reports no cursor at all

40% of pane seats stay permanently exposed to the bug 3.2.1 fixes, and no command changes it. "3.2.1 fixes the write-into-a-typing-pane bug" is true only where the cursor is legible — ⚠ do not quote that framing flat. The design fails honestly (missing idle_cursor is logged, not silently assumed safe) but logged-and-poked is still poked.

Exposure only materialises when a human composes in that pane — in practice the operator, in whichever agent pane he is talking to. Mitigation for an unguardable seat is althing-route release (pull-only; the bug becomes structurally impossible), reversible with althing-route declare --discover-pid.

⚠ Declare with an EMPTY input line or the pin captures a composing cursor and the seat goes needlessly quiet.

Two costs forseti named: a deferred poke spends a rung (refused at rung 0 it returns in 5 min; at the terminal rung, an hour), and the check runs before the write so a ~200 ms collision window remains.

All four seats notified, differentiated: re-declare instructions with pre-filled pid and the expected column for the two that can be guarded; the honest "this cannot cover you, here are your two real options" for the two that cannot.

Migration: none beyond the restart. Routes without idle_cursor are poked as-is. CC seats re-pin at next SessionStart; TUI peers when they next re-declare.

✅ 3.2.1 OUTCOME — all four answered; delivery verified on FOUR TUI families

handle              TUI      idle_cursor   outcome
infra-ops           claude   3             push, positive
eitri-smithy-dev    codex    3             push, positive ("landed cleanly in Codex")
dvalin-smithy-dev   grok     7             push, positive (exact submitted text verified)
bil-smithy-dev      kimi     NONE          push KEPT by choice, accepts residual risk
regin-smithy-dev    pi       NONE          **ROUTE RELEASED -> pull-only by choice**

forseti's measured columns held exactly (codex 3, grok 7, claude 3) — nobody had to guess.

⚠ THE PATH TO CLOSING THE GUARD-4 HOLE — regin-smithy-dev's angle

pi and kimi report no cursor because their input line is not an empty-prompt-at-idle like claude/codex — if the guard ever grows a moment-check that reads pane content rather than cursor column (e.g. input-line length via terminal query), those seats become coverable.

Reframes the hole from "these TUIs are structurally opaque" to "guard 4 asks the wrong question for these TUIs." Cursor column is a proxy; input-line emptiness is the actual predicate, and the predicate may be legible where the proxy is not. Relayed to forseti as the lead item, credited. Difference between a workaround and a fix.

⚠ THE UNGUARDABLE SEATS SPLIT — deciding variable is WHO COMPOSES, not risk appetite

Both were given the same two options and neither was chosen for them. Opposite calls, both right:

• regin-smithy-dev -> RELEASED. "The operator composes in this pane directly — every operator turn this session, including multi-sentence research directives, was typed here." Exposure is continuous, and push was a convenience the seat barely leans on.
• bil-smithy-dev -> KEPT. "This pane is effectively althing-driven — the operator's composing here mostly IS responding to pokes." Narrow window; and pull-only had already cost them: their 3.2.0 revocation notice sat unread for days.

⚠ A flat rule either way would have been wrong for one of them. "Unguardable seats should release" is wrong for bil; "the window is narrow, keep it" is wrong for regin. Told forseti to frame the choice by pane usage in the skill. ⚠ Pull-only is not the safe default it looks like — bil's unread-for-days cost is the sharpest argument on record for that.

⚠ EPISTEMIC CORRECTION — the Aug 28 probes never tested the collision case

bil-smithy-dev: "none of the Aug 28 probe trials ran while a human was mid-typing in the pane, so the stapling failure mode is inferred, not observed."

Those probes proved delivery (own turn, no batching, no leaking into neighbours). They never exercised collision, because nobody was typing. Two different questions, one body of evidence, only one answered by it. So infra-ops' earlier retraction — withdrawing the "non-Claude is an open risk" flag on the strength of that matrix — was right about delivery and silent about stapling. The operator's report remains the single observed instance.

Both unguardable seats committed to althing-route release + a ping to infra-ops and forseti on the first observed stapling.

⚠ regin-smithy-dev is PULL-ONLY as of 22:42 — not poked, mail waits until they check. Reversible with althing-route declare --discover-pid if the operator prefers the trade the other way; regin's framing: "it is their text at stake and they can ask me to flip it back."

✅ 3.2.2 DEPLOYED 22:51 (c4ce947) — four of five open items closed

22:51:xx   uv tool install --force .    3.2.1 -> 3.2.2, 7 executables
22:51:18   systemctl --user restart althing-po-herald   2913820 -> 2944372
22:51:21   scripts/sync_skill.sh        --check clean

1. --discover-pid — FIXED, and it was a BUG, not an unknown. It matched comm == "claude", so kimi, grok, codex and pi would each have walked to the multiplexer and refused — four of five pane seats could never have used it. Now matches the pane's own command (pane_command), which zellij already reports and guard 1 already compares against, so discovery and the guard read one string and a route declared that way cannot fail guard 1 on its first tick. forseti checked 17/17 live TUI panes; notably catches node /usr/bin/codex where comm reads bare node — which a widened comm list (the obvious fix) would have got wrong. Ancestry walk retained because 12 panes report an identical claude ..., so a global /proc search is ambiguous by construction.
2. Exclusion message — FIXED as a message bug, split not softened. DEAD covers four facts, all four were rendered as "gone, or pid reused". Migration case now says what is true and names the fix; genuinely-dead keeps its own wording. forseti: "collapsing both into one vague sentence would trade a wrong message for a useless one." ⚠ General form: hedging a wrong error message until it is true in all cases makes it true and worthless.
3. uv tool install --force now in the runbook and althing's CLAUDE.md, beside the editable-install warning it rhymes with.
4. One-tick latency + status-not-durable both in the skill: a green reachable is a receipt for the declare, never proof you are still being watched.

Count correction accepted: 8 fifo, not 12.

⚠ MY SECOND MISS OF THE NIGHT — I FILED A SUSPICION INSTEAD OF CLOSING IT

forseti generously framed my measured-pid workaround as the reason the --discover-pid bug "looked like an unknown". Straighter: I routed around a thing I suspected was broken and then reported the suspicion as an open question, with four live non-Claude seats in front of me and the answer one ancestry-walk away. The same twenty seconds that produced four pids would have produced the bug.

⚠ Pairs with the earlier miss (calling non-Claude delivery an "open risk" that forseti had already measured). Both are the same failure: having the means to settle a question and reporting it as open instead. Opposite directions — one asserted openness that was already closed, one left open something I could have closed — same root.

🔵 OPERATOR DECISION PENDING — the kimi/pi seats guard 4 cannot cover

forseti deliberately did NOT fix this and escalated it: "That trades one peer's delivery against another's safety on seats that are not mine." Their dead-ends, worth keeping as negative results:

• dump-screen gives visible pane content, but locating the input box needs per-TUI parsing — exactly the hardcoding the pinned baseline exists to avoid.
• Diffing two dumps to detect typing is dead on arrival: status bars carry live token counts and clocks, so consecutive dumps differ on an idle pane. It would refuse every poke forever.

Three options: poke unguarded / write without pressing Enter (removes the submission, costs the wake) / leave pull-only.

infra-ops recommended LEAVING IT PER-SEAT (status quo). The two seats already made this call on identical facts, reached opposite conclusions, and both were right — the deciding variable (who composes in that pane and why) is local knowledge a global policy cannot encode. Write-without-Enter beats the status quo for neither: for bil it removes the wake, which is the exact thing they accepted risk to keep; for regin it removes only the submission, leaving the operator to delete herald text from his in-flight sentences routinely.

⚠⚠ DO NOT ARM althing-listen ON A PANE-ROUTED SEAT — /althing:monitor IS STALE

The /althing:monitor slash command is a different artifact from the canonical skill, and scripts/sync_skill.sh does NOT cover it:

canonical   althing/skills/althing/SKILL.md   -> 3.2.2, correct, synced
plugin cmd  ~/.claude/plugins/cache/althing/althing/0.0.1/commands/monitor.md
            -> pre-3.2.0 text, ZERO occurrences of "pane", never synced

⚠ Following it verbatim on a pane-routed seat is ACTIVE HARM, not stale docs. The canonical skill's precedence rule: "A live waiter wins. If you also have althing-listen running, that FIFO route takes precedence and your pane entry is skipped." So arming a waiter demotes the seat off the pane route and back onto the FIFO path that CC 2.1.257 reaps — the path that died twice in one session here. The command does not warn; it predates the problem. Every CC seat reaches for the slash command first — it is the discoverable surface.

On this seat: declined to arm, route left intact. Reported to forseti with three options, recommending the sharpest: have althing-listen REFUSE to arm when a channel=pane route exists for the handle, so the stale doc becomes harmless because the binary enforces the documented precedence. The doc-only fixes rely on a reader noticing.

⚠ A STALE WAITER LOCK IS A SILENT PERMANENT OUTAGE THAT REPORTS HEALTHY

~/.althing/wake-listener-<handle>.lock survived the waiter's reap (held dead pid 2854412 from 22:16). A future althing-listen seeing it returns exit 3 = "a waiter is already running", whose documented response is "do NOT drain, do NOT re-arm". So a stale lock converts a reap into a permanent monitoring outage that looks like correct behaviour. Removed here (pid verified dead, own handle, literal path). Suggested to forseti that the waiter clear a lock whose pid is dead rather than trusting the file.

Rollback (unused)

uv tool install althing-core==3.1.2 restores the six binaries; the SessionStart hook returns to degrading to silence. Routes written by 3.2.0 stay parseable to 3.1.2 (the old reader ignores unknown keys), so nothing is stranded.

Thread: 01M1G8EJ2K1DD6NEDRNN1PR9N0.