docs(ops-log): record how this instrument has failed, not just the fixes
Four attribution failures shipped on the day it was built, and every one of them RECORDED SOMETHING -- just not something findable or attributable. The individual fixes are in the commit log; the PATTERN was not written down anywhere, so a future session would meet each one as an isolated bug rather than as the fifth instance of a known shape. infra-hermes asked for this after the fourth one: 'worth an audit line item so the pattern is visible, not just the fixes.' He was right. The operative warning is at the bottom: absence of an entry is not evidence of absence of the event until you have checked this tool could have recorded it.
This commit is contained in:
@@ -128,6 +128,38 @@ claim does.
|
||||
|
||||
---
|
||||
|
||||
## How this instrument has failed — the pattern, not just the fixes
|
||||
|
||||
**All four failures below shipped on the day it was built, 2026-09-19, and
|
||||
every one of them RECORDED SOMETHING.** None of them recorded something
|
||||
*findable* or *attributable*. That is the shape of the failure class here: this
|
||||
tool does not fall over loudly, it answers the wrong question quietly. Read
|
||||
this table before assuming a missing entry means the event did not happen.
|
||||
|
||||
| # | Failure | Symptom | Cause | Fixed |
|
||||
|---|---|---|---|---|
|
||||
| 1 | **Claim released by a sub-tool** | A 45-min operation claim vanished mid-rollout; nothing refused anything | `deploy-stack.sh` claims and releases around its own work, and released a claim it had only refreshed | `3e7d3a3` — `claim` exits 10 when already yours and leaves the holder file untouched |
|
||||
| 2 | **Wrong order in the hook chain** | Some commits logged, some not; looked flaky | The commit hook was APPENDED behind graphify's eight `exit 0` paths, so a `graphify-out/`-only or empty commit could never reach it | `9141a41` — prepend after the shebang; attribution runs first, unconditionally |
|
||||
| 3 | **No handle in the environment** | A peer's commit logged as `lkraven@nh3-dev` | `ALTHING_HANDLE` was exported only by `althing-infra-hermes-seat-run.sh`, not by `hermes-gateway.service` or the pump unit | Drop-in overrides (infra-hermes) + `4e778ae` — the fallback now says `unattributed(login)` instead of a bare login that reads like an answer |
|
||||
| 4 | **Wrong host key on write** | Five records of a real `jobs.cfg` change on esh-pve were invisible to `--host esh-pve` | elway passed its ssh TARGET through as the host, so they landed under `infra-ops@esh-pve` | `f3b68e2` — `normalize_host` strips `user@` on write AND on query, so pre-fix records are findable too |
|
||||
|
||||
**The rules that fell out of it, in descending order of how much they cost:**
|
||||
|
||||
1. **A log you cannot query under the obvious name is not a log.** #4 was not
|
||||
cosmetic — it produced a confident "this change is unattributed" report
|
||||
about a change that had five records.
|
||||
2. **Attribution must run FIRST**, never behind another tool's opinion about
|
||||
whether an event is interesting.
|
||||
3. **An instrument that cannot answer must say so.** A bare unix login where a
|
||||
handle belongs is answer-shaped; `unattributed(...)` is not.
|
||||
4. **A sub-tool must not drop the claim wrapping its caller.**
|
||||
5. **Two samples that look flaky deserve a control-flow audit before a
|
||||
concurrency hypothesis.** #2 was diagnosed as an `fcntl` race and was not;
|
||||
it was deterministic and five lines up.
|
||||
|
||||
⚠ **Absence of an entry is not evidence of absence of the event** until you
|
||||
have checked that this tool could have recorded it in the first place.
|
||||
|
||||
## What it deliberately does not do
|
||||
|
||||
- **It does not cover raw `ssh`.** It cannot. `audit` is the backstop.
|
||||
|
||||
Reference in New Issue
Block a user