From d0882fb830cae12921528effb65995d29f835eda Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Wed, 2 Sep 2026 01:04:32 -0700 Subject: [PATCH] feat(althing): four-surface deploy script + runbook Deploying althing touches four independent surfaces on nh3-dev. Three were known. The fourth -- the plugin -- had no step in any runbook and drifted for five days before anyone noticed. The plugin chain is repo plugin/ to the marketplace directory to Claude Code's cache, and neither hop was automated. The marketplace directory was a frozen copy from Aug 28 carrying only the UserPromptSubmit hook, with no SessionStart, no SessionEnd and no pane-route.sh at all. So the claim that CC seats re-declare their pane route automatically at session start was never true on this box, which is why every seat had to be hand-declared with a pid measured by hand. The script backs up the marketplace directory before syncing, re-stamps its marketplace.json from the repo's plugin.json, and uses `claude plugin update` for the cache rather than hand-editing installed_plugins.json -- that is Claude Code's own bookkeeping and a subtle mistake there breaks the plugin in a way that looks like an upstream bug. The runbook also carries the two things most likely to waste someone's afternoon: `uv tool install .` without --force is a silent no-op that exits 0 having done nothing, and a live waiter reporting mode:pull is a seat that will never be poked, with the audit loop for finding them. --- docs/runbooks/althing-deploy.md | 93 +++++++++++++++++++++++++ scripts/deploy-althing.sh | 116 ++++++++++++++++++++++++++++++++ 2 files changed, 209 insertions(+) create mode 100644 docs/runbooks/althing-deploy.md create mode 100755 scripts/deploy-althing.sh diff --git a/docs/runbooks/althing-deploy.md b/docs/runbooks/althing-deploy.md new file mode 100644 index 0000000..962acd0 --- /dev/null +++ b/docs/runbooks/althing-deploy.md @@ -0,0 +1,93 @@ +# althing deploy — four surfaces + +**`scripts/deploy-althing.sh`** does all four. `--check` reports drift without +touching anything. This page is the why. + +Deploying althing means updating **four independent surfaces on nh3-dev**. Three +were known; the fourth had no step in any runbook and drifted for five days +before anyone noticed, on 2026-09-01. + +| # | surface | what it is | how it drifts | +|---|---|---|---| +| 1 | **binaries** | `uv tool install --force .` — the 7 console scripts | silently, see below | +| 2 | **herald** | `systemctl --user restart althing-po-herald` | new guards live here; nothing takes effect until it restarts | +| 3 | **skill** | `scripts/sync_skill.sh` → `~/.agents/skills/althing/SKILL.md` | covered by its own `--check` | +| 4 | **plugin** | repo `plugin/` → marketplace dir → Claude Code cache | **two hops, neither was automated** | + +## ⚠ `uv tool install .` without `--force` is a silent no-op + + $ uv tool install . + `althing-core @ file:///home/lkraven/development/althing` is already installed + $ echo $? + 0 + +uv matches on the source **spec**, not its **contents**. On a box that already +installed from that path it declines and reports success. An operator following +a runbook literally would restart the herald, see everything green, and wonder +why the new binary was missing — with every command exiting 0. + +## ⚠ Surface 4 is the one that bites, and it ate a hook + +The chain is: + + repo plugin/ → ~/.local/share/althing-plugin/ → ~/.claude/plugins/cache/althing/althing// + ^^^ nothing synced this hop + +On 2026-09-01 the marketplace directory was a frozen copy from **Aug 28**: + + deployed 0.0.1 hooks: ['UserPromptSubmit'] + repo 0.1.1 hooks: ['UserPromptSubmit', 'SessionStart', 'SessionEnd'] + + scripts/pane-route.sh (absent from the deployment entirely) + +So **"CC seats re-declare their pane route automatically at next SessionStart" +was never true on this box.** The hook existed and worked upstream; it was never +deployed. That is why every seat — including infra-ops' own Claude Code seat — +had to be hand-declared with a pid someone measured by hand, and why the +`idle_cursor` pin from 3.2.1 would not have self-applied either. + +**Use the supported CLI for the second hop.** `claude plugin update althing` +(also `install` / `uninstall` / `list` / `details` / `validate` / `marketplace`). +⚠ Do **not** hand-edit `~/.claude/plugins/installed_plugins.json` or fabricate a +cache directory — that is Claude Code's own bookkeeping, and a subtle mistake +there breaks the plugin in a way that looks like an upstream bug. + +⚠ **A Claude Code restart is required** for new plugin hooks. They are read at +session start; a running session keeps the old ones. + +## Verifying a seat is actually reachable + +`postbox status` reports `mode: push`, but until 3.2.4 a **failed** push +declaration was silent — `_declare_push` devnulled both streams with +`check=False`, so a missing binary, a non-zero exit and a server-side rejection +rendered identically as nothing. A seat could have been silently pull-only since +3.1.2. + +Audit for "looks armed but is not", cross-referencing local waiter locks against +what the post office believes: + +```bash +for f in ~/.althing/wake-listener-*.lock; do + h=$(basename "$f" .lock); h=${h#wake-listener-} + pid=$(cat "$f" 2>/dev/null) + if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then + printf " %-24s waiter %-8s mode: %s\n" "$h" "$pid" \ + "$(postbox status --handle "$h" 2>/dev/null | grep -oP 'mode: \K\w+')" + fi +done +``` + +A live waiter reporting `mode: pull` is a seat that will never be poked. From +3.2.4 onward `$ALTHING_ROOT/listen.log` records failed declarations directly. + +⚠ **A stale `wake-listener-*.lock` is NOT a fault.** The gate is `flock -n` on an +open fd, which the kernel releases when the holder dies, so a lock file left by a +reaped listener is inert and exit 3 only fires against a genuinely live holder. +The pid in the file is read by `--stop` alone. (Recorded because infra-ops +claimed the opposite, untested, on 2026-09-01; forseti measured it.) + +## Rollback + + uv tool install althing-core==3.1.2 + +Routes written by later versions stay parseable — the old reader ignores unknown +keys — so nothing is stranded. diff --git a/scripts/deploy-althing.sh b/scripts/deploy-althing.sh new file mode 100755 index 0000000..c1f7e90 --- /dev/null +++ b/scripts/deploy-althing.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# Deploy althing on this box. FOUR surfaces, not three — the fourth is the one +# that had no step and drifted for five days without anyone noticing. +# +# scripts/deploy-althing.sh deploy +# scripts/deploy-althing.sh --check report drift, change nothing +# +# ───────────────────────────────────────────────────────────────────────────── +# WHY FOUR +# +# 1. uv tool install --force . the 7 binaries (postbox, althing-listen, +# althing-route, althing-po-herald, ...) +# 2. herald restart guards live in the herald; nothing new takes +# effect until it restarts +# 3. scripts/sync_skill.sh the machine-global participant skill +# 4. PLUGIN repo plugin/ -> marketplace dir -> CC cache +# +# Surface 4 is two hops and NEITHER was automated. The marketplace directory +# was a frozen copy from 2026-08-28 carrying plugin 0.0.1 with ONLY the +# UserPromptSubmit hook — no SessionStart, no SessionEnd, no pane-route.sh. +# Consequence: "CC seats re-declare their pane route automatically at next +# SessionStart" was never true on this box, which is why every seat had to be +# hand-declared with a pid someone measured by hand. +# +# ⚠ `uv tool install .` WITHOUT --force IS A SILENT NO-OP. uv matches the +# source SPEC, not its contents: on a box that already installed from this +# path it prints "already installed" and exits 0 having done nothing. +# +# ⚠ The plugin cache update needs a Claude Code RESTART to take effect. The +# hooks are read at session start; a running session keeps the old ones. +# ───────────────────────────────────────────────────────────────────────────── +set -euo pipefail + +REPO="${ALTHING_REPO:-$HOME/development/althing}" +MARKET="${ALTHING_MARKETPLACE_DIR:-$HOME/.local/share/althing-plugin}" +CHECK=0 +[[ "${1:-}" == "--check" ]] && CHECK=1 + +say() { printf '%s\n' "$*"; } +step() { printf '\n── %s\n' "$*"; } + +[[ -d "$REPO/.git" ]] || { say "✗ no althing checkout at $REPO"; exit 2; } + +repo_plugin_version() { + python3 -c "import json;print(json.load(open('$REPO/plugin/.claude-plugin/plugin.json'))['version'])" +} +market_plugin_version() { + python3 -c "import json;print(json.load(open('$MARKET/.claude-plugin/plugin.json'))['version'])" 2>/dev/null || echo "absent" +} +cache_versions() { + ls -1 "$HOME/.claude/plugins/cache/althing/althing/" 2>/dev/null | tr '\n' ' ' || echo "absent" +} + +step "state" +say " repo $(git -C "$REPO" describe --tags --always) $(git -C "$REPO" diff --quiet && echo clean || echo DIRTY)" +say " installed tool $(uv tool list 2>/dev/null | awk '/^althing-core/{print $2}')" +say " repo plugin $(repo_plugin_version)" +say " marketplace $(market_plugin_version)" +say " CC cache $(cache_versions)" + +if (( CHECK )); then + say "" + say "--check: nothing changed." + exit 0 +fi + +step "1/4 uv tool install --force (NEVER without --force: silent no-op)" +( cd "$REPO" && uv tool install --force . 2>&1 | tail -2 ) + +step "2/4 restart the herald" +systemctl --user restart althing-po-herald +sleep 2 +say " herald: $(systemctl --user is-active althing-po-herald)" + +step "3/4 sync the machine-global skill" +"$REPO/scripts/sync_skill.sh" >/dev/null +"$REPO/scripts/sync_skill.sh" --check | sed 's/^/ /' + +step "4/4 plugin — repo -> marketplace -> Claude Code cache" +if [[ -d "$MARKET" ]]; then + cp -a "$MARKET" "$MARKET.bak-$(date -u +%Y%m%dT%H%M%SZ)" + say " backed up the marketplace dir" +fi +mkdir -p "$MARKET/.claude-plugin" +# marketplace.json lives ONLY in the deployed tree (the repo keeps its own at +# .claude-plugin/marketplace.json pointing at ./plugin). Preserve and re-stamp. +tmp=$(mktemp) +[[ -f "$MARKET/.claude-plugin/marketplace.json" ]] && cp -a "$MARKET/.claude-plugin/marketplace.json" "$tmp" +rsync -a --delete --exclude marketplace.json "$REPO/plugin/" "$MARKET/" +[[ -s "$tmp" ]] && cp -a "$tmp" "$MARKET/.claude-plugin/marketplace.json" +rm -f "$tmp" +python3 - "$REPO" "$MARKET" <<'PY' +import json, pathlib, sys +repo, market = sys.argv[1], sys.argv[2] +src = json.loads(pathlib.Path(f"{repo}/plugin/.claude-plugin/plugin.json").read_text()) +p = pathlib.Path(f"{market}/.claude-plugin/marketplace.json") +d = json.loads(p.read_text()) +d["plugins"][0]["version"] = src["version"] +d["plugins"][0]["description"] = src["description"] +p.write_text(json.dumps(d, indent=2) + "\n") +print(f" marketplace.json stamped {src['version']}") +PY +say " updating the Claude Code plugin cache ..." +# `claude plugin update` is the SUPPORTED path. Do NOT hand-edit +# ~/.claude/plugins/installed_plugins.json or fabricate a cache directory — +# that is Claude Code's own bookkeeping and a subtle mistake there breaks the +# plugin in a way that looks like an upstream bug. +claude plugin update althing 2>&1 | sed 's/^/ /' || say " (update reported nothing to do)" + +step "verify" +say " installed tool $(uv tool list 2>/dev/null | awk '/^althing-core/{print $2}')" +say " marketplace $(market_plugin_version)" +say " CC cache $(cache_versions)" +say " herald $(systemctl --user is-active althing-po-herald)" +say "" +say "⚠ RESTART Claude Code to load the new plugin hooks. A running session keeps the old ones."