vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(lv-mccarthy): note the undershoot hole in AMENDMENT 3's bar without editing the bar

Browse Source 
Six generations into ckpt300 its outputs are far too SHORT -- 6, 25, 46, 59 words
against a 90-140 ask. At epoch 0.65 the adapter may not have learned the length
target yet.

That exposes a hole in the bar I set an hour ago: ran-on is defined as words > 140,
so a chronically undershooting arm satisfies "ran-on <= 0.100" trivially while
being useless. Same class as the unsatisfiable criterion AMENDMENT 3 exists to
correct, inverted -- too easy rather than impossible.

The bar is NOT being edited again. Three rule-motions in one session is already too
many and a bar patched once per observation is not a bar. Instead the judgment is
stated in the open: a candidate that clears the ran-on term by being short has
cleared it vacuously and will not be shipped, and the full in-band / on-beat /
median / p90 table is reported for every arm so a reader can see which kind of pass
they are looking at.

Gate design for this corpus is evidently not settled. That is an operator call, and
the complete five-arm ladder is being measured so it can be made once rather than
iteratively.
This commit is contained in:
Vuong Hoang
2026-09-21 16:37:25 -07:00
parent 43a8d7409e
commit 3de4d2dfaf
1 changed files with 23 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/mccarthy-corpus/GATE-PREREG.md
+23
View File
@@ -420,3 +420,26 @@ arm is generated:**
If no candidate clears all three, lv-mccarthy is **parked pending a retrain**, and that is the
report — not a softer bar.
### AMENDMENT 3a — noted 6 generations into ckpt300, before any of its rates exist
ckpt300's first outputs are **far too SHORT** — 6, 25, 46, 59 words against a 90–140 ask, with
keyword hits of 0/12 and 3/12. At epoch 0.65 the adapter may simply not have learned the length
target yet.
⚠ **That exposes a hole in AMENDMENT 3's bar, and the hole is mine.** `ran-on` is defined as
`words > 140`, so **a chronically undershooting arm satisfies `ran-on ≤ 0.100` trivially while
being useless.** It is the same class of error as the unsatisfiable criterion this amendment
exists to correct, inverted: a criterion too easy rather than impossible.
**The written bar is NOT being edited again.** Three rule-motions in one session is already too
many, and a bar patched once per observation is not a bar. Instead:
> **A candidate that clears AMENDMENT 3's ran-on term by being SHORT has cleared it vacuously
> and will not be shipped.** The full in-band / on-beat / words-median / p90 table is reported
> for every arm so the reader can see which kind of pass it is, and the judgment is stated in
> the open rather than buried in a threshold.
This is a judgment applied in the conservative direction, not a threshold moved. **Gate design
for this corpus is evidently not settled — that is an operator call, and the complete five-arm
ladder is being measured so it can be made once rather than iteratively.**