2bf418c510bf415fae2e3e93f81d4174ee18c3bd
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vuong Hoang
|
0d80e493a8 |
fix(r49): the quote-mark counter was counting apostrophes, and I saw it fire before I saw the bug
voice_distance.py's quote class shipped this morning as "'‘’“”«»‹›‚„` -- with the
apostrophe characters in it. On a corpus whose defining tic is dont/aint/wont and
whose possessives are everywhere, that made it an apostrophe counter wearing a
quote-mark label.
as implemented TRUE quotes all apostrophes
held-out McCarthy ref 121.1 0.0 121.1
base-unadapted 224.7 19.9 204.8
held-out Hemingway ref 1112.6 694.7 351.7
The corrected column is the one the pre-registration names: 0.0 for McCarthy,
which is exactly what build_corpus_mccarthy.py ASSERTS, and 694.7 for Hemingway,
the documented ~838 scale the 100-per-10k trigger line was anchored to. The
as-implemented column matched neither.
ORDER OF EVENTS, because it is the material fact: the base arm finished first, so
the trigger became evaluable while the adapted arms were still generating. I
evaluated it, saw it FIRE at 224.7, and only then -- reading the reference row of
my own table against a corpus I knew asserts 0.0 -- found the bug. No delta_cb,
memorisation rate or damage number had been read at any point.
Fixing a detector to measure the quantity the frozen rule names is not moving the
rule, but the fix un-fires the trigger and no reader should have to take my word
about my motives. So GATE-PREREG.md AMENDMENT 2 makes the trigger MOOT instead of
adjudicating it: the normalised secondary read is load-bearing UNCONDITIONALLY for
this gate, whichever reading you accept, both columns reported. The fix therefore
has no effect on the verdict.
There is a better reason than the bug anyway: base's true quote density is 19.9
against the reference's 0.0, so it did not fully comply. A small residual cheap win
IS available to the adapter, and the normalised read is what prices it. A threshold
is a blunt instrument for a residual that size.
Apostrophes now get their own column and are never folded into quotes again.
Default path stays byte-identical to the shipped lv-hemingway artifact.
The durable lesson is the one this line keeps relearning in new places: I controlled
strip_punct (2500 -> 0) and the byte-identity of the default path, but never asked
the quote counter for a value whose answer I already knew. The corpus asserts 0.0.
That check cost one line and was available before the gate ever launched.
|
||
|
Vuong Hoang
|
9c8a4e959c |
feat(lv-mccarthy): pre-register the v2 gate before any arm is generated
Freezes the lv-mccarthy gate design while no McCarthy generation exists, per the
lv-hemingway precedent: a threshold chosen after seeing the numbers is not a
threshold. Three arms (base-unadapted, ckpt900, ckpt450), 60 beats, 4 seeds.
Settles the voice-axis question D1 deferred. McCarthy's corpus measures 0.0 quote
marks per 10k against Hemingway's 838, so "emit no quotation marks" is the cheapest
way to move a char-bigram Delta without learning a sentence. Three parts:
PRIMARY unchanged -- the mccarthy register names the punctuation and
--system-from drives the base control with the same prompt, so the
cheap win is handed to both sides.
SECONDARY voice_distance.py --secondary-normalised re-runs the whole analysis
with punctuation stripped from the reference and every arm. A
conservative lower bound; reported, never the verdict.
TRIGGER voice_distance.py --punct-report evaluates a pre-registered line --
base-unadapted quote density above 100 per 10k means the control did
not take the win it was handed, and the normalised read is promoted
to load-bearing. 100 is the order-of-magnitude line between this
corpus's 0.0 and Hemingway's 838, chosen now rather than after the
table prints.
ckpt450 is NOT tied with the minimum (+4.9x the 0.00393 median neighbour jitter)
and the pre-registration says so. It is generated to test a stated prior and to
price the memorisation headroom an earlier checkpoint buys on an in-copyright
author, with a decision rule that says exactly what result would let it displace
ckpt900.
Instrument controls, run before this landed:
- the voice_distance.py refactor reproduces the shipped lv-hemingway
voice_distance.txt BYTE FOR BYTE on the default path
- strip_punct drops a quote-bearing sample from 2500.0 to 0.0 marks per 10k
- the secondary read still resolves a gap on the Hemingway arms (+0.410 at
7.3x floor), so a null on McCarthy would be a finding, not a blind detector
Pre-flight re-run at gate time rather than quoted from 2026-09-17:
- leak gate: 0 of 75 renameable, 0 of 37 sub-threshold, 0 separator-split,
four controls green
- beat-contamination audit: 0 of 3942 beats AND 0 of 3942 responses, against
Hemingway's 70 of 7094 -- build_sft_pairs.py --source-entities earned its
mandatory flag
Also records three provenance defects found on first read of the run and their
disposition: the hardcoded "r49-babyyarros-pairs-pilot" run label (cosmetic, same
literal on all three runs), the empty harness_commit (all three runs), and a
pairs_sha256_16 that is not a sha256sum of the file (consistent across runs, so a
cache key rather than a fault). The run is bound to McCarthy's pairs by record
count, not by the label.
|
||
 vh
|
0bb4938518 |
lv-hemingway: pre-register the v2 gate, and fix the floor rule that decided lv-bronte
The gate design is written before any generation exists, because lv-bronte's
verdict turned on a choice that was only visible after the numbers printed.
THE FLOOR RULE IS NOW PAIRWISE. lv-bronte computed the noise floor as the largest
within-arm seed spread across ALL arms present. Its ckpt475 shipped at +0.193
against a 0.251 floor set entirely by ckpt925 -- a third arm nobody was shipping,
on one outlier seed. Scored against the arm it was actually compared to, the floor
is 0.092 and the same gap clears at 2.1x. A candidate's verdict must not depend on
which other arms happened to be generated. voice_distance.py now prints both floors
and flags any disagreement, so the lv-bronte record stays comparable.
audit_pairs_sourcenames.py closes the blind spot leak_gate.py has by construction:
it reads the corpus and the renamed copies, never the generated beats, so it cannot
see a beat-writing model restoring the author's real character names. Run over the
Hemingway pairs, which predate build_sft_pairs.py --source-entities:
val 0 of 200 -- the eval fixture is clean, the gate is unconfounded
train 70 of 7,094 (0.96%) -- Santiago x16, Catherine x7, Rinaldi x3, Brett,
Harry, Jake, Pablo, Nick, Maria ...
responses 0 of 7,294 -- the lv-bronte beat-only signature exactly
A matched surface is only counted when the rename actually removed it, verified
against the renamed copies, so a beat naming a held real-world place is not a leak.
Controls run every time: 941/941 surfaces found in the unrenamed source, nonce
absent from both trees, and 6 planted canonical names detected 6/6.
voice_distance.py --author is now REQUIRED. It was hardcoded "Yarros" and printed
"reference: held-out Yarros" over Brontë's numbers into a committed artifact. A
default would have moved the silent-wrong-label failure rather than removed it. The
stale "one seed-pair per arm / corroborates Base < Instruct" footer is replaced with
what the run actually carries.
Gate design: three arms (base-unadapted, ckpt1750, ckpt850), 60 beats, 4 seeds.
ckpt850 is present because the loss curve cannot separate it from ckpt1750 -- +0.0040
against a 0.0044 median neighbour jitter, with three checkpoints inside one jitter of
the minimum. adapter/ is excluded: +0.0762 is 17.4x the jitter and is resolved without
a gate.
|
||
|
vh
|
5558d9c7d3 |
R49/BabyYarros: voice + beat evaluation tooling and the seat-free delta_cb
Adds the Yarros-side evaluation the training exists to justify: does the adapter move arbitrary prose toward Yarros, and can the instruct arm still expand a beat to a paragraph on direction. Yarros-flavoured voice prompts (modern/neutral/ romantasy tiers so any Yarros voice in the modern tier is adapter-attributable, not prompt-supplied) and a Yarros-register beat SYS on the chat generator. voice_distance.py is the honest slice of adjudication that needs no seat: Burrows's Delta over character bigrams against held-out Yarros. Its first cut mis-framed the noise floor — it used the same-author distance (held-out vs itself) as the between-arm significance threshold, which is the target, not the threshold. Fixed to the measured floor: the within-arm seed spread, which is this metric's sampling variance at this sample size, computed from the two seeds already generated rather than assumed. Result on the built corpus, ordering base-125-tuned < instruct-tuned < base-unadapted, both adapters clearing the 0.046 measured floor (base +0.157, instruct +0.076), and the ordering corroborating the independent held-out loss ordering (Base below Instruct). One seed-pair per arm, so it corroborates rather than settles; the full frozen adjudication still needs a romantasy control panel, a second seed, and the gen seat for the beat-incumbent leg. |