The standing link board grew from a flat oldest-first list with a per-row
× into a manageable board: newest links lead, favorites stay on top, and
several dead links can go in one pass.
- Ordering: order_for_display() renders pinned rows first, then newest-first
within each group (the board is an append log, so newest = most recently
posted — the row you usually came to grab).
- Pin/favorite: a per-row ★ toggles pinned state via POST /b/<name>/pin.
State lives in a .pins sidecar dotfile (one content id per line), NOT
inline in links.md — so links.md stays a pure atomic-append log (many
sessions post concurrently) and a row's content id never changes just
because it was pinned. remove_link_entry drops a removed row's pin;
orphaned pins are inert (renderer only stars a live id).
- Multi-select delete: checkboxes feed POST /b/<name>/unlink-many (repeated
'sel' content ids), with a select-all box and a live count. The per-row ×
stays for single removal.
- One <form> with formaction buttons, so checkboxes, ×, ★, and bulk delete
coexist without nested forms AND all work with JS off; JS only adds
select-all and the live count. Per-row × confirm reads desc/url from
data-* attrs, so an arbitrary posted description can't break into the JS.
- Every action is keyed by content id, never row position — same race-safety
the existing × has, extended to the bulk path.
- Fixed pre-existing undefined --fg/--bg CSS refs in the board styles.
Tests: +19 (pins round-trip, ordering, orphan-inert, remove-unpins, /pin
and /unlink-many endpoints, board render + order). Full suite 102 passing.
Deployed to nh3-dev booth.service; verified live (newest-first, pin
round-trip, bulk delete) against the real 31-row board with no data loss.
The A record and the CLAUDE.md fleet row both still named 10.100.79.3,
the wg0 lifeline that went dormant in the headscale cutover — so the
name resolved to a dead door and the row read like an outage waiting to
be misdiagnosed.
- dns/internal.yaml: irv-ml1 10.100.79.3 -> 10.6.110.50 (Irvine's real
LAN, 10.6.110.0/24, advertised fleet-wide as a mesh subnet route).
Resolve to the LAN IP to match the host->LAN-IP model every other
entry uses; mesh node IP 100.64.0.6 recorded in the note. Comment
rewritten: Irvine is no longer numbered out of NH3's 10.100.79.0/24.
- CLAUDE.md: fleet-table row and GPU-placement rule updated to the mesh
reality (wg0 lifeline dormant, access via headscale mesh).
Verified: all three AdGuard resolvers (ana/esh/nh3) return 10.6.110.50;
name pings live ~11ms; reachable from a mesh node and from a colo host
behind a subnet router.
Changes the risk profile of the pending ledger-dev -> svos-dev rename, so it
belongs in memory rather than only in the thread: retire is the correct verb
(delete refuses any handle carrying mail) and it can be undone by declaring the
name again.
Also records that both deploy findings landed in althing's own INSTALL.md,
where they sit beside the journal_mode=WAL setting that causes them rather than
in a procedure doc that only describes working around it.
The tools had been on 3.6.0 while the container sat on 3.0.0 for seven days, so
the operator verbs failed with "no tool named delete_handle" — they live in the
post office, not the client.
Image built on nh3-dev from a clean tree at 4d26226 and pushed under the
claude-bot namespace; the compose pin moves to the new digest rather than
floating on the tag, since this container is the fleet's whole message bus.
The backup procedure this file documents earned itself again: at stop time the
database was 23.8 MB with a 5.9 MB WAL beside it, so a plain copy would have
produced a database that opens cleanly, smokes green, and is missing the day's
mail. Stop, explicit checkpoint to a zero-byte WAL, copy, then verify counts on
both sides — 76 handles, 995 messages, 1022 recipients, integrity ok.
Post-deploy the same counts came back with handles.retired_at present, and the
memory cap and OOM guard were confirmed by `docker inspect` rather than by
reading the yaml, which is what that file asks for and the only check that can
tell a working cap from a decorative one.
Bus down about four minutes.
Records the fifo-relay fix and — more usefully — the two shapes that were tried
and rejected, so the next person does not re-derive them: a pipeline breaks cwd
persistence because every stage runs in a subshell, and a plain file is
invisible until the command ends. The absent `wait` is deliberate and is pinned
by a test, because it looks exactly like an oversight.
Corrects this file's own claim that infra-ops is sudo-less on nh3-extdev. It is
not: `sudo -n whoami` returns root. The wrong row would send a session looking
for a privilege path it already has.
Also flags that the althing tools are at 3.6.0 while the post office container
is still on 3.0.0, so the new handle verbs cannot work yet.
Found by using the tool for real work rather than by smoke-testing it. A bare
`sudo -n whoami` through ssh_run returns `running` with no output and wedges the
session permanently; the same command piped through `cat` returns everything.
Reproduced on both macOS and Debian, so it is the harness rather than a platform
quirk, and it matters because infra work is sudo work.
The workaround goes in CLAUDE.md where a session will actually hit it. The
proper fix is a protocol change — run the command through a pipe and read
PIPESTATUS — and is deliberately not attempted here.
Also records where the dsh work stopped: the package is updated but the running
web process is still on the old code, left alive on purpose because there is no
LaunchAgent to restart it into.
The alias requirement is gone for anything inside 10.0.0.0/8, so the tooling
note no longer tells sessions to add an alias first — that instruction was
about to send people back to raw ssh, which is what it was written to prevent.
Records the SureFire carve-out and why it is host-specific rather than a /24:
pfi-pve shares 10.250.250.0/24 with two tenant machines, so a subnet deny would
have taken our own hypervisor with it.
Records the operator's ruling that esh-macbook-air is regenerable surface and is
not to be backed up, so a future session finds a settled decision rather than
re-discovering an unbacked-up machine and re-raising it.
Also records the real blocker on the MCP server, found by trying to use it: it
addresses hosts only by exact ssh_config alias, and 15 of the 28 hosts in
servers/ have no alias at all. The allowlist is not what limits coverage, so
widening it would not have helped.
Records what the fork now contains and, more usefully, the two things that will
waste a future session's time: .mcp.json points at built output so an unbuilt
edit serves stale code, and a finite stdin pipe kills the server mid-handshake
while reporting what looks exactly like a remote-side SSH fault.
Also records the disproven hypothesis — irv-ml1's zsh login shell was not the
cause, because the server invokes bash explicitly — so nobody re-derives it.
elway is a convergent deploy runner and it is good at that: 64 playbooks, sudo
handled once, upload, and three tiers of idempotency. It is the wrong shape for
"what is running over there", which is why this session answered every
reconnaissance question with a raw `ssh host '...'` despite CLAUDE.md saying to
use elway — a playbook for a read-only `docker ps` is ceremony, and each ad-hoc
command is a fresh handshake and a fresh permission decision.
The MCP server fills that half. It holds a persistent shell, so cwd and exported
variables survive between calls and a command costs ~6 ms rather than a full
connect. Measured against nh3-extdev and irv-ml1: state persisted across calls
on both, including the host reached over WireGuard.
CLAUDE.md now routes by intent — changing versus looking — rather than naming
one tool, because the previous single-tool instruction was being ignored in
practice for exactly the cases it did not fit.
Registered project-scoped rather than user-scoped: this workspace is where SSH
infra work happens, and a server that can open shells on the fleet should be
opted into per project rather than inherited everywhere.
SSH_MCP_STRICT_ALLOWLIST=1 is set in .mcp.json deliberately and documented as
load-bearing. Without it the allowlist is every Host in ~/.ssh/config — 18
aliases including both backup servers, database VMs and a tenant-managed host.
Verified: corviduo-dev, present in ssh_config but not in our allowlist, is
refused with host_not_allowed.
Step 8 executed on the operator's direct authorization with preconditions
checked before firing and the post-state read back rather than inferred. The
rollback window is closed and the arc is finished except the handle rename.
Records the remote-ssh-mcp fork and, more usefully, why it was chosen over the
far more popular alternative: it never handles key material, delegating to the
system OpenSSH client, and its dependency surface is two packages. Also records
that its denylist is four bypassable regexes and must not be counted as
security — the host allowlist is the real boundary — plus the two things to
settle before the tool is used in anger.
The delete was verified from outside rather than taken on report: an admin key
cannot see consumer agents, but the key roster shows the old key authenticating
once after 47 minutes of silence with the new key following 35 seconds later,
which is the signature of delete-then-confirm.
Step 8 is deliberately not done. ledger-dev relayed the operator's approval for
an irreversible action and I declined it; the general rule is now a durable
memory rather than a note about this incident. Both keys stay live.
Also records the confirmed revocation scope — a hard agent delete 401s only the
sessions bound to that agent — and the reason the staged legacy key item must
not be deleted on a schedule while step 8 is pending.
Cutover landed clean — a 201 define rather than 409/PATCH, which is what proves
the new identity is actually in use. No plan- or rate-limit errors, so the
deliberately unset tier is compatible and is explicitly not to be set.
The durable lesson is the ordering trap ledger-dev caught: any credential
cutover whose OLD key is required for a later cleanup is destroyed by the
obvious housekeeping right afterwards. Re-vaulting the post-cutover env.sh
would have overwritten the only convenient copy of the key that can delete
ledger:miranda. Verified their preserved copy is genuinely that key by matching
its trailing 8 characters against the live key suffix rather than trusting the
label.
Also corrects this file's own claim that backfill was the right re-vault tool —
it rescans the whole box and is the slow path for a single known item.
They pulled it from the vault and verified the sha independently, so delivery is
established rather than assumed. The remaining step — swapping the live
credential in env.sh, flipping worldtree.user_id, registering svos:miranda and
restarting the service — is the operator's, not mine and not theirs off a peer
message.
Also records that the global CLAUDE.md Heimdall routing line was corrected in
place on operator instruction, so the ruling is not carried by this file alone.
user_id=svos, key_id=eab3cdbe, wt_live_ format, on worldtree-personal
(10.250.50.152:8081). Deployment established by locating the ledger key there
rather than assumed: created 2026-07-13, last used today, matching ledger-dev's
account exactly.
Value vaulted at nh3-dev/development/svos/worldtree-api-key and verified against
the mint response before the scratch copy was removed. Delivered by vault path
rather than over althing — ledger-dev runs on the same box under the same uid,
so the bus never carried the credential.
The ledger key was read back from /admin/keys AFTER the mint and is untouched,
which is what keeps their deletion ordering possible.
Plan tier left unset deliberately and disclosed: the API offers no way to read a
user's current tier, so matching it would have been a guess.
Also records the operator ruling that worldtree-dev owns code only and no ops,
which supersedes the global CLAUDE.md routing of Heimdall auth to worldtree-dev.
Rename executed on the operator's direct authorization. Repo id 70 unchanged.
Redirect verified by measurement rather than by citing upstream #807: web and
API paths both 301, and `git ls-remote` against the old URL warns, follows, and
returns HEAD b48a11ca5183. Existing clones keep working.
Recorded as a standing hazard that the old path is now burned — a repo created
at corviduo/ledger kills the redirect silently.
Also opens the Heimdall item ledger-dev had been holding as "awaiting infra-ops"
without ever having sent it, including the deletion-ordering trap that makes
retiring the old key first unrecoverable.
The `secret` CLI has no rename, so this was re-put + rm. Stored
nh3-dev/development/svos/env.sh from the live file (sha 7253633d4155, verified
on read-back) and retired nh3-dev/development/ledger/env.sh (sha feb418634e10,
id 3a2af37c-c5aa-4f46-9178-f4fb6008a753, soft-deleted to trash and recoverable).
The two shas differ: the vaulted copy was a 2026-08-11 snapshot and the live
file had drifted since without ever being re-vaulted. Recorded because the
lesson generalises — the vault goes stale unless `secret backfill` is re-run.
Gitea corviduo/ledger deliberately NOT renamed. It is ledger-dev's repo; the
answer they needed is that 1.26.1 writes a repo_redirect on a same-org repo
rename, org/user renames do not redirect, the redirect dies if anything
re-creates the old path, and both the repo and the org carry zero webhooks.
Operator asked for it explicitly so he can test in Open WebUI in parallel with
brokkr's batteries. Named `trial` rather than erp-tune-v3c: the run-3 note in
this file records why a tier name arriving before its evidence is a mistake,
and 3c has no more evidence than run 3 had.
Points at the pfi-gx10 seat (http://10.100.50.60:8098/v1, vLLM 0.28.0 on
GB10/sm_121), so this is a cross-site route -- NH3 seat, Anaheim gateway.
⚠ Recorded in the entry, not just here: 3c's safety axes are unmeasured, and
run 3 -- same recipe, differing only in learning rate -- was withdrawn from
this gateway 40 minutes after being added for selfharm/methods -44.0pp and
meta/sysprompt -28.0pp against its own base. This gateway is the shared-key
surface. When brokkr's batteries read out, the entry should be re-decided on
evidence rather than left standing by default.
Applied with a restart of the litellm service only (~60 s of gateway
downtime); the compose .env is root-owned so the restart needs sudo.
vh