memory: the rack is one circuit — my blast-radius objection was wrong

Operator supplied the topology: "the entire rack is on the same circuit,
public ip is served by firewall on the same circuit. load tripped
breaker, entire rack goes dark."

That inverts the argument I committed one commit ago in 3cc55b4. I held
the load attribution partly because a site-wide blackout looked like too
large a blast radius for one GPU box. It is not — it is exactly the
blast radius a shared circuit predicts. The observation I treated as
unexplained was the signature of the mechanism.

Kept the wrong version visible with a superseded marker rather than
editing it away, per the superseded-claims convention: the reasoning
circulated in two althing messages and a peer adopted it.

Holding the attribution as unmeasured was still correct; the argument
used to hold it was reasoning about wiring neither side had seen and
neither side asked for.

Records the finding that outlives the incident: the same circuit powers
ana-ml2, ana-gw, ana-wg, ana-nas and the BMCs, so the load takes every
recovery path with it. Composed with ana-gw's WAN admin being closed
(tunnel-only) and the tunnel terminating on ana-wg, there is no remote
management path to Anaheim during a rack power event, in-band or out.
Open question for the weekend triage: is any of that rack on UPS or a
second circuit?

persistent-memory.d/2026-08-27-anaheim-breaker-and-onboot-gap.md

@@ -66,20 +66,55 @@ Operator: *"we'll triage this weekend, probably shut down some seats."* ana-ml2
 pulling ~600 W across both GPUs at their 300 W caps during training. `gen` stays up by
 instruction; everything else on that box is idle.
 
-⚠ **Keep the measured and the hypothesised apart here — the load attribution is NOT established:**
-
     MEASURED      a breaker tripped (operator-confirmed)
     MEASURED      power lost between 18:14:45 and 18:17:00 PDT
     MEASURED      the WHOLE Anaheim site went dark from the internet
+    TOPOLOGY      ONE CIRCUIT FEEDS THE ENTIRE RACK, including the firewall serving
+                  the public IP  (operator, 2026-08-27)
 
-    HYPOTHESIS    the training load tripped it — the operator's working read and the reason
-                  for the weekend triage. There is no per-circuit meter on that panel, and
-                  a site-wide blackout is a larger blast radius than one GPU box's ~600 W
-                  accounts for on its own: if ana-ml2's draw were the whole story, why did
-                  ana-nas, ana-wg and the public address go dark with it?
+    OPERATOR      "load tripped breaker, entire rack goes dark." Consistent with every
+      DETERMINATION observation. Not a per-circuit measurement, and no longer needs to be —
+                  the operator has the site knowledge and has ruled.
 
-**Held as the right thing to triage first, not as the cause.** Shedding GPU seats may well be
-correct and is cheap; it is not the same as having identified what loaded the circuit.
+⚠ **SUPERSEDED — the blast-radius objection I raised here was WRONG, and it was committed.**
+An earlier revision of this section (commit `3cc55b4`) argued that *"a site-wide blackout is a
+larger blast radius than one GPU box's ~600 W accounts for on its own: if ana-ml2's draw were
+the whole story, why did ana-nas, ana-wg and the public address go dark with it?"*
+
+**One shared circuit inverts that completely.** Site-wide darkness is not too large a blast
+radius for a single overloaded box — **it is exactly the blast radius the topology predicts.**
+The fact I treated as unexplained was the signature of the mechanism.
+
+Holding the attribution as unmeasured was still right (there is no per-circuit meter); the
+*argument* used to hold it was reasoning confidently about wiring neither I nor
+brokkr-smithy-dev had ever seen, and neither of us asked for it. Kept visible rather than
+silently corrected so the old reasoning stops circulating.
+
+## ⚠ THE FINDING THAT OUTLIVES THE INCIDENT — the recovery path shares the failure domain
+
+One circuit powers, together:
+
+    ana-ml2          the load most likely to trip it
+    ana-gw           the firewall serving the public IP (38.120.12.44)
+    ana-wg           the WireGuard host — the way in when the site misbehaves
+    ana-nas          the storage SPOF
+    the BMCs         ana-ml2 IPMI 10.250.250.50, pfi-pve iDRAC 10.250.250.30 — same rack
+
+**So the thing that fails takes every path used to fix it with it.** Compounding two facts
+already in the record separately: ana-gw's WAN admin is closed, so management is tunnel-only
+([[2026-08-24-ana-gw-admin-closed-acme-disabled]]) — and the tunnel terminates on ana-wg, on
+this circuit. **During a rack power event there is no remote management path to Anaheim at
+all**, in-band or out-of-band. Recovery is: wait for power. Until 2026-08-27 it was worse —
+ana-wg had `onboot` unset and would not have come back on its own.
+
+Open question for the weekend triage, and the cheapest fix if the answer is no: **is any of
+that rack on UPS or a second circuit?** Nothing in this repo records one, and the site went
+dark instantly and stayed dark 1h58m, which is behaviourally indistinguishable from no
+coverage for these loads. Putting *only* ana-gw + ana-wg + one BMC on separate power would
+mean a load trip no longer costs the way in.
+
+Load-shed on ana-ml2 is still the right first move and it is cheap. It reduces how often the
+breaker trips; it does not change what a trip costs.
 
 ## Blast radius beyond us
 

persistent-memory.md

@@ -122,7 +122,7 @@ _State as left 2026-08-26 23:05 PDT (written 2026-08-27 07:27, corrected 10:5x)
 - `[2026-08-27]` **Run 3 gated: the preregistered rule PASSED and a k=25 follow-up found a 44pp self-harm guardrail collapse — DO NOT SERVE.** A pooled preserve-list test structurally cannot see a single-axis collapse. → `persistent-memory.d/2026-08-27-run3-gate-safety-regression.md`
 - `[2026-08-27]` **The corpus mix was specified in a unit the optimiser never sees** — 45.8% dialogue by CONTEXT, 24.2% by LOSS. Harness now leads with loss share and calls context a memory budget (`dd5a12e`). → `persistent-memory.d/2026-08-27-mix-specified-in-the-wrong-unit.md`
 - `[2026-08-27]` **Dose-response: benefit and damage are ONE direction in weight space** — every axis monotone in scale, no knee. The merge-back cannot separate them; vLLM cannot LoRA-serve this MoE at all. → `persistent-memory.d/2026-08-27-dose-response-entanglement.md`
-- `[2026-08-27]` **Anaheim tripped a power breaker; four guests including the NAS had `onboot` unset and never came back.** Fixed with dependency ordering — ana-nas order=1,up=45 ahead of the databases. → `persistent-memory.d/2026-08-27-anaheim-breaker-and-onboot-gap.md`
+- `[2026-08-27]` **Anaheim tripped a power breaker; four guests including the NAS had `onboot` unset and never came back.** Fixed with dependency ordering — ana-nas order=1,up=45 ahead of the databases. ⚠ **ONE CIRCUIT FEEDS THE WHOLE RACK including the firewall serving the public IP** (operator) — so ana-gw, ana-wg and every BMC go down with the load, and there is NO remote management path to Anaheim during a power event. → `persistent-memory.d/2026-08-27-anaheim-breaker-and-onboot-gap.md`
 - `[2026-08-27]` **A transport failure that enters a measurement as a VALUE looks like whatever you hoped to find.** heid's lost panel arms found a live defect in brokkr's `t4_dissect` an hour later. → `persistent-memory.d/2026-08-27-empty-response-as-a-datum.md`
 - `[2026-08-27]` **Run 3c authorised (lr 20x cut, single variable) and then HELD by the operator after the breaker trip.** Config built and validated at `/tank/erp-tune/run-03c.json`; `save_steps` made configurable in the harness (`0a6bd2e`) because the first launch lost 80 steps with no checkpoint. Tracking surface: commit `0a6bd2e` + that config path. **Relaunch is one command once power is triaged.**
 - `[2026-08-27]` **An event report with no timestamp is a claim about "now" — and it manufactured a launch that never happened.** brokkr reconstructed a phantom third 3c launch because my 23:03 report narrated a 21:07 kill in the present tense. Every fact in it was true; it was unreadable in sequence. → `persistent-memory.d/2026-08-27-run3c-launch-count-reconstruction.md`