Files
esh-pfi-infrastructure/persistent-memory.d/2026-09-21-ops-log-and-the-instruments-that-lied.md
T
vh e52def115c memory: snapshot — the ops log, and a day spent on instruments that report without looking
Archived 15 entries (Recent decisions 14, Tried and abandoned 1) oldest-first
to archival-memory.md; 5 held back on the open-deferred-work guard and 164 on
the 14-day guard, so the index stays over the soft cap at 477 lines. An
over-cap file that keeps live decisions beats a scannable one that lost a
belayed item.

Four new detail files cover the day: the ops log and its four self-inflicted
failure modes, the Booth's two dead controls and the four-iteration layout
probe, the Gitea org grant plus the dead claude-bot token that had been
misreporting permissions, and the disk triage that rescued a LoRA adapter from
a directory this box sweeps at three days.

lv-mccarthy's run outcome remains unverified after two days and is the first
line of the in-flight section and step 1 of the handoff.
2026-09-21 14:26:55 -07:00

45 lines
2.7 KiB
Markdown

# `[2026-09-21]` The ops log shipped, and the day's real subject was instruments that report without looking
**Built** `scripts/ops-log` (ffe7b24) to close the fingerprint-less-change gap:
infra-ops and infra-hermes act as one OS identity, dockerd logs no per-caller
exec, and every commit here is attributed to Vuong Hoang by convention. One
appended line per host-changing action, a `mkdir`-atomic claim `deploy-stack.sh`
refuses (exit 3), automatic writers in `deploy-stack.sh` + `elway`, and
`ops-log audit` as the detector for the raw-`ssh` path the writers cannot see.
136-stack baseline laid so the detector starts from that day.
⚠ **The instrument then failed FOUR ways in its first hours, and every one
recorded something — just nothing findable.** Documented as a table in
`docs/pfi/ops-log.md` § "How this instrument has failed", which is the durable
artifact:
1. **Claim released by a sub-tool** (3e7d3a3) — a 45-min operation claim was
refreshed then released by `deploy-stack.sh`'s exit trap, mid-rollout.
`claim` now exits **10** when already yours and leaves the holder file
untouched, so a refresh cannot overwrite the reason and TTL the original
claimant chose.
2. **Wrong order in the hook chain** (9141a41) — the commit hook was APPENDED
behind graphify's **eight `exit 0` paths**, so a `graphify-out/`-only or
empty commit could never be recorded. Prepend; attribution must never be a
subordinate clause of another hook's interestingness filter.
3. **No handle in the environment** (4e778ae) — `ALTHING_HANDLE` lived only in
`althing-infra-hermes-seat-run.sh`, not the gateway unit. Fallback now says
`unattributed(login)` rather than a bare login that reads like an answer.
4. **Wrong host key on write** (f3b68e2) — elway passed its ssh TARGET through
as the host, so five records of a real esh-pve change landed under
`infra-ops@esh-pve` and were invisible to `--host esh-pve`. infra-hermes
correctly reported the change as unattributed. **A log you cannot query
under the obvious name is not a log.**
⚠ **The general lesson, and it outlived the tool:** twelve instruments reported
confidently and wrongly across 2026-09-19→21, five of them mine. The recurring
shape is **configured ≠ effective** — `systemctl show -p Environment` reporting
a drop-in while `/proc/<pid>/environ` lacked it; a grep proving presence while
evaluation proved absence; a green test suite over a control the browser never
received. What broke the pattern every time was asking a *different* instrument
the same question.
See [[2026-09-21-booth-two-dead-controls]] for the same failure in a UI, and
`feedback_control_flow_before_concurrency` in auto-memory for the triage rule
that came out of it.