3e73374c6e47d948d8af5bbe61a8a7ab0298bb25
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vuong Hoang
|
3de4d2dfaf |
docs(lv-mccarthy): note the undershoot hole in AMENDMENT 3's bar without editing the bar
Six generations into ckpt300 its outputs are far too SHORT -- 6, 25, 46, 59 words against a 90-140 ask. At epoch 0.65 the adapter may not have learned the length target yet. That exposes a hole in the bar I set an hour ago: ran-on is defined as words > 140, so a chronically undershooting arm satisfies "ran-on <= 0.100" trivially while being useless. Same class as the unsatisfiable criterion AMENDMENT 3 exists to correct, inverted -- too easy rather than impossible. The bar is NOT being edited again. Three rule-motions in one session is already too many and a bar patched once per observation is not a bar. Instead the judgment is stated in the open: a candidate that clears the ran-on term by being short has cleared it vacuously and will not be shipped, and the full in-band / on-beat / median / p90 table is reported for every arm so a reader can see which kind of pass they are looking at. Gate design for this corpus is evidently not settled. That is an operator call, and the complete five-arm ladder is being measured so it can be made once rather than iteratively. |
||
|
Vuong Hoang
|
43a8d7409e |
fix(lv-mccarthy): the axis C I froze is UNSATISFIABLE, so it never adjudicated anything
Arithmetic, not a re-reading of results. Criterion 1 needs the candidate's in-band rate to beat base's 0.89 by more than the 0.200 floor -- so it needs in_band > 1.09, and a rate cannot exceed 1. A PERFECT adapter, every one of 240 generations inside the 90-140 band, scores +0.11 and fails by 0.09. So "both candidates failed axis C" carries zero information about either candidate. That verdict was determined before a single generation existed. It is my error. The cause is specific: section 6 transcribed score_beats.py's v1 criteria, which the operator AMENDED on 2026-09-15 -- three weeks before this corpus was built -- for exactly this reason. Its own docstring: "NOT carried into v2: in-band (unresolvable -- base maxes it) and on-beat." The governing rule reverts to the operator's v2: axis C is ran-on only. That is the rule they ratified and the rule lv-hemingway shipped under. Reverting a transcription error to its source is not threshold-shopping; the source is older than the data. It does not rescue ckpt450 either. Under the corrected axis C its ran-on is +0.19 against a 0.200 floor -- a pass by 0.01, inside the noise the floor exists to describe, and not a pass worth shipping on an author in copyright with a living estate. AMENDMENT 3 therefore gates two more arms on the SAME frozen fixture, seeds and rule -- ckpt300 (epoch 0.65) and ckpt225 (epoch 0.49), both on disk because --save-total-limit 60 kept all 56 checkpoints -- since the damage grows monotonically with epoch and the voice may arrive before it does. And it sets the ship bar BEFORE those arms exist: voice clearing the pairwise floor on both reads, memorisation at or near the held-out author's 0.12 with every match read, and ran-on delta AT MOST 0.100 -- half the measured floor, not merely inside it, because a rate that clears a noise threshold by 5% of that threshold has not been shown to clear it. If nothing clears all three, lv-mccarthy is parked pending a retrain, and that is the report rather than a softer bar. |
||
|
Vuong Hoang
|
0d80e493a8 |
fix(r49): the quote-mark counter was counting apostrophes, and I saw it fire before I saw the bug
voice_distance.py's quote class shipped this morning as "'‘’“”«»‹›‚„` -- with the
apostrophe characters in it. On a corpus whose defining tic is dont/aint/wont and
whose possessives are everywhere, that made it an apostrophe counter wearing a
quote-mark label.
as implemented TRUE quotes all apostrophes
held-out McCarthy ref 121.1 0.0 121.1
base-unadapted 224.7 19.9 204.8
held-out Hemingway ref 1112.6 694.7 351.7
The corrected column is the one the pre-registration names: 0.0 for McCarthy,
which is exactly what build_corpus_mccarthy.py ASSERTS, and 694.7 for Hemingway,
the documented ~838 scale the 100-per-10k trigger line was anchored to. The
as-implemented column matched neither.
ORDER OF EVENTS, because it is the material fact: the base arm finished first, so
the trigger became evaluable while the adapted arms were still generating. I
evaluated it, saw it FIRE at 224.7, and only then -- reading the reference row of
my own table against a corpus I knew asserts 0.0 -- found the bug. No delta_cb,
memorisation rate or damage number had been read at any point.
Fixing a detector to measure the quantity the frozen rule names is not moving the
rule, but the fix un-fires the trigger and no reader should have to take my word
about my motives. So GATE-PREREG.md AMENDMENT 2 makes the trigger MOOT instead of
adjudicating it: the normalised secondary read is load-bearing UNCONDITIONALLY for
this gate, whichever reading you accept, both columns reported. The fix therefore
has no effect on the verdict.
There is a better reason than the bug anyway: base's true quote density is 19.9
against the reference's 0.0, so it did not fully comply. A small residual cheap win
IS available to the adapter, and the normalised read is what prices it. A threshold
is a blunt instrument for a residual that size.
Apostrophes now get their own column and are never folded into quotes again.
Default path stays byte-identical to the shipped lv-hemingway artifact.
The durable lesson is the one this line keeps relearning in new places: I controlled
strip_punct (2500 -> 0) and the byte-identity of the default path, but never asked
the quote counter for a value whose answer I already knew. The corpus asserts 0.0.
That check cost one line and was available before the gate ever launched.
|
||
|
Vuong Hoang
|
b4ba731b26 |
fix(r49): commit the held-out-author memorisation control, and amend the mccarthy prereg to use it
The lv-hemingway gate established that memorization_check.py's negative control is
defective: base-unadapted writes SUMMARY while the adapted arms write PASTICHE, and
text that does not imitate a register cannot collide with that register's n-grams.
Base's 0.00 partly measures "different register", not "did not memorise". A negative
control that differs from the candidate in a way correlated with the metric is not a
control.
The correct innocent sample is the author himself -- held-out text no arm trained on,
in the same register by construction. That control was computed BY HAND during the
lv-hemingway gate and never committed, so the finding it produced was not
reproducible. This commits it as --heldout-reference, with --train-only to build the
n-gram set from split=="train" records only (the default includes val, so a collision
with held-out text was being counted as memorisation of training text -- an overcount
Bronte's and Hemingway's numbers both carry).
--heldout-reference without --train-only REFUSES, because the held-out text would be
scored against a gram set containing itself.
Instrument controls:
- default path reproduces the shipped lv-hemingway memorization.txt BYTE FOR BYTE
- the new held-out row reproduces the lv-hemingway record's hand-computed numbers
to the digit: 370 samples, hit-rate 0.01, mean-longest 0.1, max 10, at the same
101-word median chunk length -- an independently written instrument reproducing a
documented finding is what makes its zeroes mean absent rather than blind
GATE-PREREG.md AMENDMENT 1 adopts it for lv-mccarthy, appended rather than folded in
so the order of events stays legible. Written while the base arm was still generating
and before any McCarthy number had been read. It moves the reference a candidate is
compared against from a known-broken one to a known-correct one, in the direction
that makes the axis HARDER to pass, on an argument published four days before this
gate existed. Section 7's tiebreak is amended to match, and the amendment records
that an elevated rate is not by itself a no-ship -- rate and exposure are different
questions, every matched run gets READ, and McCarthy being in copyright with a living
estate makes a distinctive match disqualifying in a way a rate number alone is not.
|
||
|
Vuong Hoang
|
9c8a4e959c |
feat(lv-mccarthy): pre-register the v2 gate before any arm is generated
Freezes the lv-mccarthy gate design while no McCarthy generation exists, per the
lv-hemingway precedent: a threshold chosen after seeing the numbers is not a
threshold. Three arms (base-unadapted, ckpt900, ckpt450), 60 beats, 4 seeds.
Settles the voice-axis question D1 deferred. McCarthy's corpus measures 0.0 quote
marks per 10k against Hemingway's 838, so "emit no quotation marks" is the cheapest
way to move a char-bigram Delta without learning a sentence. Three parts:
PRIMARY unchanged -- the mccarthy register names the punctuation and
--system-from drives the base control with the same prompt, so the
cheap win is handed to both sides.
SECONDARY voice_distance.py --secondary-normalised re-runs the whole analysis
with punctuation stripped from the reference and every arm. A
conservative lower bound; reported, never the verdict.
TRIGGER voice_distance.py --punct-report evaluates a pre-registered line --
base-unadapted quote density above 100 per 10k means the control did
not take the win it was handed, and the normalised read is promoted
to load-bearing. 100 is the order-of-magnitude line between this
corpus's 0.0 and Hemingway's 838, chosen now rather than after the
table prints.
ckpt450 is NOT tied with the minimum (+4.9x the 0.00393 median neighbour jitter)
and the pre-registration says so. It is generated to test a stated prior and to
price the memorisation headroom an earlier checkpoint buys on an in-copyright
author, with a decision rule that says exactly what result would let it displace
ckpt900.
Instrument controls, run before this landed:
- the voice_distance.py refactor reproduces the shipped lv-hemingway
voice_distance.txt BYTE FOR BYTE on the default path
- strip_punct drops a quote-bearing sample from 2500.0 to 0.0 marks per 10k
- the secondary read still resolves a gap on the Hemingway arms (+0.410 at
7.3x floor), so a null on McCarthy would be a finding, not a blind detector
Pre-flight re-run at gate time rather than quoted from 2026-09-17:
- leak gate: 0 of 75 renameable, 0 of 37 sub-threshold, 0 separator-split,
four controls green
- beat-contamination audit: 0 of 3942 beats AND 0 of 3942 responses, against
Hemingway's 70 of 7094 -- build_sft_pairs.py --source-entities earned its
mandatory flag
Also records three provenance defects found on first read of the run and their
disposition: the hardcoded "r49-babyyarros-pairs-pilot" run label (cosmetic, same
literal on all three runs), the empty harness_commit (all three runs), and a
pairs_sha256_16 that is not a sha256sum of the file (consistent across runs, so a
cache key rather than a fault). The run is bound to McCarthy's pairs by record
count, not by the label.
|