The record read "power capacity is the open item" next to ana-ml2's
~600 W, which reads as a cause. It is not one. The trip and its timing
are measured; the attribution to the training load is the operator's
working read and the reason for the weekend triage.
The observation that makes the single-load story incomplete on its own
terms: a site-wide blackout is a larger blast radius than one GPU box
accounts for. If ana-ml2's draw were the whole story, ana-nas, ana-wg
and the public address would not have gone dark with it.
Shedding seats may still be the right first move and it is cheap. That
is not the same as having identified what loaded the circuit, and the
distinction matters going into a triage that will act on it.
brokkr-smithy-dev asked how many times 3c was launched rather than
reconstructing it, and their reading was three. It was two.
#1 17:53:33 PDT killed by the power loss at step 80/604
#2 20:58:41 PDT stopped deliberately at 21:07:40, healthy
The phantom third came from a report I wrote at 23:03 narrating the
21:07 kill in the present tense with no timestamp. Every fact in it was
accurate; it was unreadable in sequence against a correctly-observed
22:46 snapshot of an idle GPU.
Evidence is ZFS birth times (a `>` redirect truncates the log but keeps
its birth, so mtime alone cannot separate "rewritten" from "created"),
plus the absence of any mtime under /tank/erp-tune after 21:07:34 — a
relaunch would have rewritten three files there.
Also pins the outage window to 18:14:45-18:17:00 PDT and corrects the
downtime from "~90 minutes" to 1h58m: the last journald entry before a
hard power loss is the last time anything wanted to log, not the moment
of the loss, and here it was 20 minutes early.
Corrects the in-flight header (step 22 -> last-logged step 24, stop
deliberate) and its stale "as of" stamp.
Run 3 trained, gated and dispositioned do-not-serve on a measured 44pp self-harm
guardrail regression that its own preregistered rule passed -- a pooled preserve-list
test cannot see a single-axis collapse. Run 3c (lr 20x cut, single variable) launched,
killed by an Anaheim power-breaker trip at step 80, relaunched, then stopped by the
operator at step 22 pending a weekend power triage.
Also captured: the corpus mix was specified in a unit the optimiser never sees
(45.8% dialogue by context, 24.2% by loss); the dose-response says benefit and damage
are one direction in weight space, so the merge-back measures the problem rather than
fixing it; four guests including the storage SPOF had onboot unset and never came back
from the outage, now fixed with dependency ordering; and a transport failure that
enters a measurement as a value looks like whatever you hoped to find -- which found a
live defect in another agent's instrument an hour after it was reported.
Auto-archived 8 entries to archival-memory.md (Recent decisions: 8, Tried and
abandoned: 0); 4 held back on open deferred-work pointers.
vh