From a4a529dd767fb5e28611a1e3aaa665c1f322becb Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Wed, 2 Sep 2026 08:34:43 -0700 Subject: [PATCH] docs(runbook): assert waiter identity, not liveness, in the reachability audit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cross-reference loop filtered live waiters on `kill -0 "$pid"`. A lock file left by a reaped listener names a pid the kernel is free to reissue, so the loop would report an unrelated process as a live waiter — and a phantom waiter is how a false "your seat is unreachable" notice reaches a seat that is fine, with nothing in the output to falsify it. `session_listener.sh --stop` already refuses this standard: it walks /proc//cmdline for the `--_route=` segment before signalling, on the grounds that a live pid proves existence and not identity. The audit now asserts the same thing and prints STRANGER on a mismatch. Verified against the live box: six waiters, all identity-confirmed, all reporting push. --- docs/runbooks/althing-deploy.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/docs/runbooks/althing-deploy.md b/docs/runbooks/althing-deploy.md index f5cc5d5..79c6306 100644 --- a/docs/runbooks/althing-deploy.md +++ b/docs/runbooks/althing-deploy.md @@ -99,16 +99,33 @@ what the post office believes: for f in ~/.althing/wake-listener-*.lock; do h=$(basename "$f" .lock); h=${h#wake-listener-} pid=$(cat "$f" 2>/dev/null) - if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then - printf " %-24s waiter %-8s mode: %s\n" "$h" "$pid" \ - "$(postbox status --handle "$h" 2>/dev/null | grep -oP 'mode: \K\w+')" - fi + [ -n "$pid" ] && [ -r "/proc/$pid/cmdline" ] || continue + # IDENTITY, not liveness: a lock left by a reaped listener names a pid the + # kernel is free to hand to anything, so `kill -0` alone reports a stranger + # as a live waiter. Match the same cmdline segment `althing-listen --stop` + # requires before it will signal anything. + ours=0 + while IFS= read -r -d '' seg; do [ "$seg" = "--_route=$h" ] && ours=1; done \ + < "/proc/$pid/cmdline" + [ "$ours" = 1 ] || { printf " %-24s pid %-8s STRANGER (recycled pid)\n" "$h" "$pid"; continue; } + printf " %-24s waiter %-8s mode: %s\n" "$h" "$pid" \ + "$(postbox status --handle "$h" 2>/dev/null | grep -oP 'mode: \K\w+')" done ``` A live waiter reporting `mode: pull` is a seat that will never be poked. From 3.2.4 onward `$ALTHING_ROOT/listen.log` records failed declarations directly. +⚠ **Liveness is not identity, and this loop is the place that gets it wrong.** +The earlier `kill -0` form would print a phantom waiter for any handle whose +stale lock happens to name a recycled pid — and a phantom waiter is exactly what +sends a false "you are unreachable" notice to a seat that is fine. +`session_listener.sh` refuses to SIGTERM on liveness alone for precisely this +reason; an audit that only *reads* has no excuse for a weaker standard than the +one that *kills*. (Adopted 2026-09-02 after the regin-smithy-dev +cross-reference, where the pid happened to be genuine and the weaker check +happened to be right.) + ⚠ **A stale `wake-listener-*.lock` is NOT a fault.** The gate is `flock -n` on an open fd, which the kernel releases when the holder dies, so a lock file left by a reaped listener is inert and exit 3 only fires against a genuinely live holder.