40 lines
2.6 KiB
Markdown
40 lines
2.6 KiB
Markdown
# `[2026-09-05]` A peer's "2.7x stack effect" was a coin flip — the operator caught it and the arithmetic is worth keeping
|
|
|
|
brokkr-smithy-dev reported that a diversity **noise floor** was 2.7x tighter on ana-ml2 than on
|
|
the GX10 and ranked ana-ml2 the better instrument on it. I relayed it. **The operator rejected it
|
|
on instinct** — *"that makes zero sense. except for speed, serving a model should be identical
|
|
across servers"* — and he was substantially right.
|
|
|
|
ana-ml2 0.9688, 0.9574 spread 1.14pp mean 0.9631
|
|
gx10 0.9583, 0.9895 spread 3.12pp mean 0.9739
|
|
|
|
between-box LEVEL difference 1.08pp
|
|
ana-ml2's own replicate spread 1.14pp <- LARGER than the between-box gap
|
|
pooled range ignoring box 3.21pp <- ~= the entire "GX10 floor"
|
|
|
|
**Each "floor" is `|block0 - block1|` from n=2.** A range over two draws is not an estimate of
|
|
dispersion; the ratio of two such ranges is a ratio of two half-normals — a **half-Cauchy**.
|
|
brokkr computed the tail himself on retraction: `P(ratio >= 2.7) = 1 - (2/pi)*arctan(2.7)`,
|
|
doubled = **0.452**, confirmed by 400,000-draw simulation. He had reported a coin flip as a
|
|
measured effect and ranked hardware on it. Retracted at `97f73dd`.
|
|
|
|
⚠ **The disconfirming evidence was inside his own sentence.** He wrote "the base level is nearly
|
|
identical, it is the spread that shifts" and offered it as *reassurance*. A real stack effect
|
|
moves the level. Level agreeing while a two-draw range differs is the signature of a noisy range
|
|
estimator — he had the refutation in hand and read it as support.
|
|
|
|
⚠ **His own diagnosis of why it got through is the transferable part:** he had spent the session
|
|
triaging *my* claims hard — reading the encode path, simulating preflight, recomputing shas from
|
|
bytes — and this one was **his, and flattering**: it made his earlier work look prescient and
|
|
produced a clean recommendation. **Asymmetric scepticism is one error twice, and the flattering
|
|
direction needs the extra pass.**
|
|
|
|
**What survived, deliberately separated:** re-measuring the floor on whatever stack actually
|
|
serves stays non-negotiable. The mechanism list is sound whether or not those four numbers show
|
|
it — sm_121 vs sm_120 kernel selection, vLLM 0.28.0 against mixed 0.26/nightly, and separately
|
|
measured batch-invariance (3.12pp at jobs=8, same order as the whole claimed effect).
|
|
**Retracting the evidence and keeping the discipline are different acts.** Settling it properly
|
|
wants several blocks per box and is its own probe, not a by-product of a gate.
|
|
|
|
See [[2026-09-05-vllm-on-sm121-and-run4]].
|