vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(lv-mccarthy): five-arm ladder -- ckpt300 wins every axis, held at the gate by 0.02

Browse Source 
1,200 generations across five arms. ckpt300 (epoch 0.652) is the best arm in the
run on every axis that resolves:

  VOICE          +0.177 at 3.2x its pairwise floor (primary), +0.128 at 2.8x with
                 every punctuation mark stripped. Best point estimate AND best
                 margin of any arm, spread 0.055/0.038 with no outlier seed.
  MEMORISATION   0.12 against real unseen McCarthy's own 0.12 -- identical -- with
                 a longest match of 10 words against the author's coincidental 12.
                 All 31 matches read: stock grammar, names are the renamed
                 inventions, nothing protectable.
  DAMAGE         ran-on +0.12. Clears the operator's ratified v2 floor of 0.200 by
                 40%. FAILS AMENDMENT 3's self-imposed 0.100 bar by 0.02.

NOT SHIPPED, and the reason is the bar rather than the adapter. AMENDMENT 3 fixed
ran-on <= 0.100 before either new arm existed, precisely so a marginal number could
not be talked into a ship, and shipping at 0.12 would make that pre-registration
theatre. But the bar's stated rationale was written against ckpt450's pass by 0.01
-- 5% of the threshold -- and ckpt300 clears by 40%. The number excludes a candidate
the reasoning does not. That is an operator call.

ckpt325/350/375 are on disk and one may sit under 0.100. They were deliberately NOT
gated: searching the checkpoint space until something clears is candidate-shopping,
the same family as threshold-shopping approached from the other side.

THREE CLAIMS FROM EARLIER THIS SESSION ARE REFUTED and are corrected in the record:

  1. "The damage is flat across epochs and only rotates direction" -- FALSE. ran-on
     is non-monotonic (0.38 -> 0.13 -> 0.20 -> 0.28 across epochs 0.49/0.65/0.98/
     1.96) with a real minimum near 0.65, and ckpt225 is 48% out-of-band against
     ckpt300's 35%.
  2. "ckpt300 runs far too short, ckpt225 will clear ran-on by being short" -- FALSE
     on both. ckpt225 runs LONG (median 127, 38% over-band) and is the worst arm in
     the run. I generalised from SIX generations of one arm, which is the exact n=1
     violation the measurement-discipline rule names, committed in the same breath
     as a note about being careful.
  3. The original "gate an earlier checkpoint, the overshoot may not have arrived
     yet" recommendation was RIGHT. Retracting it an hour later on a three-arm read
     was the error, not the recommendation.

What is true and unresolved by any checkpoint choice: 35% of ckpt300's generations
miss the 90-140 band against base's 11%, and in-band is 0.65 against 0.89. An
adapter that buys a voice and costs a third of the length compliance is a trade, not
a defect -- but it is the operator's trade to accept.

Raw artifacts for all five arms at scripts/mccarthy-corpus/gate-results/.
This commit is contained in:
Vuong Hoang
2026-09-21 17:53:43 -07:00
parent 3de4d2dfaf
commit e8eb1594d9
13 changed files with 329 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/mccarthy-corpus/gate-results/voice_distance.txt
+12 -2
View File
@@ -3,6 +3,8 @@ same-author target (held-out McCarthy vs itself): delta_cb = 0.370
-> the floor of what any arm could reach; lower is more McCarthy-like, this is the best possible
arm delta_cb per-seed [words]
ckpt300 0.484 (0.561 0.509 0.564 0.519) [26444] spread 0.055
ckpt225 0.485 (0.592 0.493 0.513 0.542) [33153] spread 0.099
ckpt900 0.490 (0.605 0.457 0.531 0.554) [30935] spread 0.148
ckpt450 0.509 (0.562 0.530 0.563 0.567) [28494] spread 0.037
base-unadapted 0.661 (0.711 0.667 0.673 0.659) [26078] spread 0.052
@@ -11,10 +13,12 @@ same-author target (held-out McCarthy vs itself): delta_cb = 0.370
PAIRWISE floor is the verdict: max(spread(candidate), spread(base-unadapted) = 0.052)
vs base-unadapted control (positive gap = moved toward McCarthy):
ckpt300 +0.177 (MOVED toward McCarthy (3.2x the pairwise floor 0.055))
ckpt225 +0.176 (MOVED toward McCarthy (1.8x the pairwise floor 0.099))
ckpt900 +0.172 (MOVED toward McCarthy (1.2x the pairwise floor 0.148))
ckpt450 +0.152 (MOVED toward McCarthy (2.9x the pairwise floor 0.052))
ordering: ckpt900 < ckpt450 < base-unadapted (lower = more McCarthy-like)
ordering: ckpt300 < ckpt225 < ckpt900 < ckpt450 < base-unadapted (lower = more McCarthy-like)
⚠ RELATIVE reading on one harness: 4 seed group(s) per arm, scored against this corpus's own held-out split. It is not an absolute-band claim and corroborates nothing on its own.
PUNCTUATION DENSITY per 10k words -- the confound check, not an axis
@@ -23,6 +27,8 @@ same-author target (held-out McCarthy vs itself): delta_cb = 0.370
arm quote-marks all-apos contraction-apos dashes
held-out reference 0.0 121.1 117.6 7.6
base-unadapted 19.9 204.8 203.6 1.9
ckpt225 0.0 155.3 154.7 0.0
ckpt300 0.8 188.3 188.3 0.0
ckpt450 0.0 167.4 166.4 0.0
ckpt900 0.0 160.7 160.7 0.0
@@ -43,7 +49,9 @@ same-author target (held-out McCarthy vs itself): delta_cb = 0.363
-> the floor of what any arm could reach; lower is more McCarthy-like, this is the best possible
arm delta_cb per-seed [words]
ckpt300 0.460 (0.525 0.510 0.532 0.494) [26956] spread 0.038
ckpt900 0.464 (0.563 0.460 0.493 0.522) [31440] spread 0.104
ckpt225 0.466 (0.561 0.481 0.498 0.516) [33678] spread 0.080
ckpt450 0.474 (0.518 0.506 0.528 0.526) [28975] spread 0.022
base-unadapted 0.588 (0.634 0.611 0.594 0.588) [26645] spread 0.046
@@ -51,8 +59,10 @@ same-author target (held-out McCarthy vs itself): delta_cb = 0.363
PAIRWISE floor is the verdict: max(spread(candidate), spread(base-unadapted) = 0.046)
vs base-unadapted control (positive gap = moved toward McCarthy):
ckpt300 +0.128 (MOVED toward McCarthy (2.8x the pairwise floor 0.046))
ckpt900 +0.124 (MOVED toward McCarthy (1.2x the pairwise floor 0.104))
ckpt225 +0.122 (MOVED toward McCarthy (1.5x the pairwise floor 0.080))
ckpt450 +0.114 (MOVED toward McCarthy (2.5x the pairwise floor 0.046))
ordering: ckpt900 < ckpt450 < base-unadapted (lower = more McCarthy-like)
ordering: ckpt300 < ckpt900 < ckpt225 < ckpt450 < base-unadapted (lower = more McCarthy-like)
⚠ RELATIVE reading on one harness: 4 seed group(s) per arm, scored against this corpus's own held-out split. It is not an absolute-band claim and corroborates nothing on its own.