2.6 KiB
[2026-09-05] A peer's "2.7x stack effect" was a coin flip — the operator caught it and the arithmetic is worth keeping
brokkr-smithy-dev reported that a diversity noise floor was 2.7x tighter on ana-ml2 than on the GX10 and ranked ana-ml2 the better instrument on it. I relayed it. The operator rejected it on instinct — "that makes zero sense. except for speed, serving a model should be identical across servers" — and he was substantially right.
ana-ml2 0.9688, 0.9574 spread 1.14pp mean 0.9631
gx10 0.9583, 0.9895 spread 3.12pp mean 0.9739
between-box LEVEL difference 1.08pp
ana-ml2's own replicate spread 1.14pp <- LARGER than the between-box gap
pooled range ignoring box 3.21pp <- ~= the entire "GX10 floor"
Each "floor" is |block0 - block1| from n=2. A range over two draws is not an estimate of
dispersion; the ratio of two such ranges is a ratio of two half-normals — a half-Cauchy.
brokkr computed the tail himself on retraction: P(ratio >= 2.7) = 1 - (2/pi)*arctan(2.7),
doubled = 0.452, confirmed by 400,000-draw simulation. He had reported a coin flip as a
measured effect and ranked hardware on it. Retracted at 97f73dd.
⚠ The disconfirming evidence was inside his own sentence. He wrote "the base level is nearly identical, it is the spread that shifts" and offered it as reassurance. A real stack effect moves the level. Level agreeing while a two-draw range differs is the signature of a noisy range estimator — he had the refutation in hand and read it as support.
⚠ His own diagnosis of why it got through is the transferable part: he had spent the session triaging my claims hard — reading the encode path, simulating preflight, recomputing shas from bytes — and this one was his, and flattering: it made his earlier work look prescient and produced a clean recommendation. Asymmetric scepticism is one error twice, and the flattering direction needs the extra pass.
What survived, deliberately separated: re-measuring the floor on whatever stack actually serves stays non-negotiable. The mechanism list is sound whether or not those four numbers show it — sm_121 vs sm_120 kernel selection, vLLM 0.28.0 against mixed 0.26/nightly, and separately measured batch-invariance (3.12pp at jobs=8, same order as the whole claimed effect). Retracting the evidence and keeping the discipline are different acts. Settling it properly wants several blocks per box and is its own probe, not a by-product of a gate.