Commit Graph
2 Commits
Author SHA1 Message Date
Vuong Hoang e8eb1594d9 docs(lv-mccarthy): five-arm ladder -- ckpt300 wins every axis, held at the gate by 0.02
1,200 generations across five arms. ckpt300 (epoch 0.652) is the best arm in the
run on every axis that resolves:

  VOICE          +0.177 at 3.2x its pairwise floor (primary), +0.128 at 2.8x with
                 every punctuation mark stripped. Best point estimate AND best
                 margin of any arm, spread 0.055/0.038 with no outlier seed.
  MEMORISATION   0.12 against real unseen McCarthy's own 0.12 -- identical -- with
                 a longest match of 10 words against the author's coincidental 12.
                 All 31 matches read: stock grammar, names are the renamed
                 inventions, nothing protectable.
  DAMAGE         ran-on +0.12. Clears the operator's ratified v2 floor of 0.200 by
                 40%. FAILS AMENDMENT 3's self-imposed 0.100 bar by 0.02.

NOT SHIPPED, and the reason is the bar rather than the adapter. AMENDMENT 3 fixed
ran-on <= 0.100 before either new arm existed, precisely so a marginal number could
not be talked into a ship, and shipping at 0.12 would make that pre-registration
theatre. But the bar's stated rationale was written against ckpt450's pass by 0.01
-- 5% of the threshold -- and ckpt300 clears by 40%. The number excludes a candidate
the reasoning does not. That is an operator call.

ckpt325/350/375 are on disk and one may sit under 0.100. They were deliberately NOT
gated: searching the checkpoint space until something clears is candidate-shopping,
the same family as threshold-shopping approached from the other side.

THREE CLAIMS FROM EARLIER THIS SESSION ARE REFUTED and are corrected in the record:

  1. "The damage is flat across epochs and only rotates direction" -- FALSE. ran-on
     is non-monotonic (0.38 -> 0.13 -> 0.20 -> 0.28 across epochs 0.49/0.65/0.98/
     1.96) with a real minimum near 0.65, and ckpt225 is 48% out-of-band against
     ckpt300's 35%.
  2. "ckpt300 runs far too short, ckpt225 will clear ran-on by being short" -- FALSE
     on both. ckpt225 runs LONG (median 127, 38% over-band) and is the worst arm in
     the run. I generalised from SIX generations of one arm, which is the exact n=1
     violation the measurement-discipline rule names, committed in the same breath
     as a note about being careful.
  3. The original "gate an earlier checkpoint, the overshoot may not have arrived
     yet" recommendation was RIGHT. Retracting it an hour later on a three-arm read
     was the error, not the recommendation.

What is true and unresolved by any checkpoint choice: 35% of ckpt300's generations
miss the 90-140 band against base's 11%, and in-band is 0.65 against 0.89. An
adapter that buys a voice and costs a third of the length compliance is a trade, not
a defect -- but it is the operator's trade to accept.

Raw artifacts for all five arms at scripts/mccarthy-corpus/gate-results/.
2026-09-21 17:53:43 -07:00
Vuong Hoang 4c3f3896f1 docs(lv-mccarthy): record the gate result -- voice passes, memorisation is the cleanest in the line, NOT shipped
720 generations, 3 arms x 60 held-out beats x 4 seeds, against the design frozen
in GATE-PREREG.md before any arm existed.

AXIS A VOICE -- PASS, both candidates, both reads. Span 0.661 -> 0.370 = 0.291
achievable; ckpt900 closed 59.1% (+0.172, but only 1.2x its floor), ckpt450 52.2%
(+0.152 at 2.9x). The normalised secondary read HOLDS at +0.124 / +0.114, so about
three quarters of the gain survives stripping every punctuation mark -- the voice
is not the cheap win the register made available.

AXIS B NOT COPIED -- ckpt450 is the cleanest result in the line. 0.12 hit-rate
against the author's own held-out 0.12, and its longest match (11 words) is
SHORTER than the author's coincidental longest (12). All 96 matched runs were
read: stock grammar in the commonest words, the name-shaped hits are the RENAMED
inventions, nothing protectable. The amendment is why this reads as clean -- the
defective base control would have shown 0.12 vs 0.00 as a 12x red flag.
Separately measured: the "his register makes collisions inevitable" story that
was FALSE for Hemingway (0.01) is TRUE for McCarthy (0.12). Neither transfers.

AXIS C NO DAMAGE -- FAIL, both, and it survives reading. 20% (ckpt450) / 28%
(ckpt900) of generations overshoot the 90-140 band against base's 1%; p90 171/190
words, max 297/279. The worst case is degenerate looping, not a long McCarthy
sentence. Base is GOOD on this axis here (0.89 in-band vs Hemingway's 0.05), so
the adapter measurably makes instruction-following worse.

NOT SHIPPED. Section 7 rule 3 makes axis C disqualifying outright.

Recorded honestly: my own prereg's axis C transcribed score_beats.py's v1
criteria, including "in-band up on base", which the operator RETIRED on
2026-09-15 for exactly the reason it fails here -- base maxes it, so it is
unsatisfiable on this corpus regardless of adapter quality. Under the operator's
v2 (ran-on only) ckpt450 passes by 0.01 against a 0.200 floor. That reading was
found AFTER the numbers and was NOT used; lv-bronte's floor defect is the in-line
precedent for finding one and declining to exploit it. The prereg gets fixed
prospectively for the next author, not re-read for this one.

And the finding worth more than the adapter: the two-epoch recipe is now 0 for 3,
and this time the loss curve was CONFIDENTLY wrong. On Bronte and Hemingway the
epoch-1/epoch-2 checkpoints were tied, so preferring the earlier one cost nothing.
Here the curve resolved epoch 2 as better at 4.9x the median neighbour jitter --
and epoch 2 lost every axis that resolves: 4.0x wider seed spread, 1.8x the
author's memorisation rate against 1.0x, more ran-on, worse on-beat. Its only win
is a 0.019 voice point estimate, inside the floor, and its spread is one outlier
seed -- the third occurrence of that shape in the later checkpoint after
lv-bronte's ckpt925 and lv-hemingway's ckpt1750.

Raw artifacts committed at scripts/mccarthy-corpus/gate-results/ so the claims can
be re-read without gx10.
2026-09-21 16:31:41 -07:00