esh-pfi-infrastructure/scripts/mccarthy-corpus/RUNBOOK.md

lv-mccarthy — corpus → leak gate → pairs → train → v2 gate

Six works, 167 units, 584,684 words. Built on pfi-gx10 under ~/lv-mccarthy/ as infra-ops, except D1, which must run on nh3-dev: the builder reads the kvasir catalogue at /home/lkraven/development/kvasir/data/library/catalog.sqlite and that path exists only there.

⚠ This file exists because the 2026-09-17 D1→D3 session recorded nothing. There was no runbook, the commands were issued over non-interactive ssh so no shell history survived, and a later session had to recover the whole chain by rebuilding candidates and matching sha256 against the artifacts on disk. Every deviation below is now pinned by a reproducibility control; keep it that way.

The chain

R=~/development/eshpfi-management/scripts        # nh3-dev for D1, ~/lv-mccarthy/scripts on gx10

# D1 — build. ON nh3-dev (needs the kvasir catalogue), then rsync corpus-clean/ to gx10.
python3 $R/mccarthy-corpus/build_corpus_mccarthy.py --out corpus-clean

# D2 — entity map. --min-mid-ratio is what keeps `Yeah`/`Buenas`/`Shh` out; see below.
python3 $R/r49-corpus/entities.py corpus-clean --out corpus-clean/entities.json \
  --stoplist $R/mccarthy-corpus/stoplist_mccarthy.json \
  --min-count 5 --fold-clitics --drop-acronyms --min-mid-ratio 0.2 --min-mid 2

# D2c — the four hand-verified genders.
python3 $R/r49-corpus/apply_gender_overrides.py --entities corpus-clean/entities.json \
  --overrides $R/mccarthy-corpus/gender_overrides_mccarthy.json \
  --out corpus-clean/entities-final.json

# D3 — rename. --scope corpus, --preset mccarthy and --min-cap 5 are all deviations.
python3 $R/r49-corpus/rename.py corpus-clean --entities corpus-clean/entities-final.json \
  --dictionary ~/r49-prep/name_dictionary.json --out corpus-renamed \
  --preset mccarthy --scope corpus --min-cap 5 --copies 6 --seed 4919 \
  --holdout-chapter 7 17

# GATE — must pass before anything is trained. --min-cap 8 here is deliberate; see below.
python3 $R/r49-corpus/leak_gate.py corpus-clean --entities corpus-clean/entities-final.json \
  --renamed corpus-renamed --min-cap 8 --report corpus-renamed/leak_gate_report.json

# D4 — instruction pairs. ON gx10; ~24 min against the `gen` seat.
./run-pairs.sh

Gate result, 2026-09-17 (after the separator fix): 0 of 75 renameable, 0 of 37 sub-threshold, 0 separator-split surfaces, all four controls green. Sensitivity floor: a name under 8 capitals per work is never detected, a phrase under 5 recurrences never audited, and no phrase map exists for this corpus, so the phrase audit does not run at all — the Yarros and Brontë runs both had one.

The deviations, and what forced each

deviation	why
D1 runs on nh3-dev	The builder reads the kvasir catalogue by absolute path. gx10 has no copy.
--min-count 5 (was 8)	The entity map's floor has to match rename's, or entities between the two floors sit in the map, are never renamed, and count as leaks. The first D3 gate run failed with 45 survivors for exactly this.
--min-mid-ratio 0.2 --min-mid 2	Without it the map admits Yeah, Buenas, Shh, Mande, Sí, Git — words that are only ever capitalised at a sentence start. ⚠ The map is INSENSITIVE to the exact value: any ratio in [0.05, 0.3] with --min-mid 1 or 2 reproduces it byte-for-byte. The original run's values are unrecoverable and it does not matter. --min-mid 3 does NOT reproduce it.
--fold-clitics --drop-acronyms	Both are needed to reproduce the map; --rescue-honorific is inert here (0 rescued).
--scope corpus (was work)	Nine surfaces appear in more than one work — Parham, Grady, Cole, Socorro, Héctor. A per-work map gives John Grady a different invented name in each Border Trilogy novel.
--preset mccarthy	Hemingway's romance pool carries it_IT/fr_FR for his Italian and French casts; McCarthy writes neither. en_US + es_MX/es_ES at an even share.
--holdout-chapter 7 17 (a LIST)	The val split is one chapter per work, so its size scales with WORK COUNT, not corpus size. Six works would have given a Brontë-class ~18,000-word val reference; two indices give 11 units and 40,653 words per copy, larger than Hemingway's, for 7% of the corpus.
gate at --min-cap 8, rename at --min-cap 5	Not a mistake. The gate reports the 37 entities rename never touched separately, which is strictly more informative than running both at 5 (where the sub-threshold bucket is empty).
D4 --reflow-hard-wraps	The Road is hard-wrapped at ~76 characters and the other five works are not. See DEFECT 4 in build_sft_pairs.py.
D4 --source-entities	Mandatory. The beat generator reads the passage and will supply canonical names from its own memory of the book; the corpus gate never reads the generated beats.

The defect the gate could not see, and now can

⚠⚠ The 2026-09-17 tree passed this gate at "0 of 75 renameable and 0 of 37 sub-threshold" while carrying 13 occurrences of five protagonist names in all six copies. leak_gate.py scanned \b(Surface)\b, and a character inserted inside a name defeats that pattern outright — so the occurrence was unrenameable by rename.py and unreportable by the gate. Two mechanisms, both in the source extraction:

B ell  C higurh  M oss  T oadvine     a small-caps drop cap survived as its own token
Toad-vine  Glan-ton                   a print line-break hyphen survived the extraction

Every visible occurrence had been renamed correctly, which is what made the residue invisible to any spot-read: the names are gone everywhere you look.

Fixed in three places, and all three must stay:

1. build_corpus_mccarthy.py rules 4 and 5 repair the text at its source — 32 split initials with a lowercase remainder, 5 hyphen-split names, both with expected counts so a master change fails the build. ⚠ Rule 4's letter class is consonants only: I opens 1,966 paragraphs, A opens 143 and Y opens 32 (Spanish y), and folding any of them would corrupt 2,141 lines to fix 32.
2. leak_gate.py --split-frag-max runs a separator-tolerant scan every time, with its own positive and negative controls, and fails the gate. Verified against the pre-fix tree: it reports all five surfaces and exits 1.
3. The exact-match passes are unchanged, so the old verdict is reproduced alongside.

Cross-checked on the two shipped corpora: lv-bronte is clean of this class; lv-hemingway carries 3 (Primi tivo, Pasionar ia, Chi cote) and is live.

Corpus properties worth knowing before you change anything

• repair_typography.py MUST NOT be run on this corpus. It normalises "toward what the text does" and would put quotation marks back into a corpus that measures 0.0 per 10k against Hemingway's 838. The builder runs no normalisation and asserts the density.
• Back matter is stripped BEFORE the split, inverting the Hemingway order: Blood Meridian and The Crossing end with a dumped table of contents made of bare roman numerals, which is the exact shape of a chapter marker.
• Blood Meridian sets a dash-separated chapter argument under each roman numeral. 131 paragraphs; --drop-leading-heading now consumes them.
• Two truncated catalogue rows are excluded in favour of complete siblings.

D5 — train the adapter

# ON gx10, ~/lv-mccarthy. 1,380 steps, 2h30m47s on the GB10.
./launch-train.sh          # scripts/mccarthy-corpus/launch-train.sh

Qwen3-4B-Instruct, rank 32 / alpha 64, lr 1e-4, seq 1536, batch 1 × accum 8, 3 epochs, seed 4919 → ~/r49-runs/mccarthy-4b-pairs-3ep/. The two deliberate deviations from the script defaults (--save-total-limit 60, --eval-steps/--save-steps 25) are argued in the launch script's own header and exist to keep the epoch-1 and epoch-2 checkpoints alive for the tiebreak; the default limit of 12 would have deleted both.

Result, 2026-09-18 00:49 PT: train_loss 2.172, end-of-run eval_loss 2.4594, 56 eval points, median neighbour jitter 0.00393.

ckpt900   2.38706  ep1.958   the minimum          875 +0.4x · 850 +1.1x · 925 +1.5x = tied
ckpt450   2.4063   ep0.980   +4.9x jitter         the epoch-1 local minimum, NOT tied
adapter/  2.4594   ep3.000   +18.4x jitter        the epoch-3 collapse, resolved by the curve

⚠ The curve does not drift into the epoch-3 collapse, it STEPS: 2.393 at step 925, 2.457 at 950, then flat for the remaining 430 steps. Same shape as Hemingway's.

⚠ adapter/ is the epoch-3 weights. Whatever ships is a checkpoint, not adapter/.

Provenance fields that look wrong and are not

Checked when the run was first read, three days after it finished. All three reproduce on the yarros and hemingway runs, so none is specific to this one:

field	reading
"run": "r49-babyyarros-pairs-pilot"	a hardcoded label in train_pairs_lora.py, not a stale copy. Cosmetic.
harness_commit: ""	the trainer never recorded its own commit, on any run. No claim rests on it.
pairs_sha256_16 ≠ sha256sum of the pairs file	computed over the loaded records, not the raw bytes. Consistent and per-corpus-unique, so a valid cache key.
pairs recorded as the relative pairs/pairs-full.jsonl	resolves against the launch CWD. Bind it by COUNT, not path: train_pairs 3673 / val_pairs_n 269 match lv-mccarthy/pairs/ exactly and match no other pair set on the box.

D6 — the gate

# ON gx10, ~/lv-mccarthy. 3 arms x 60 beats x 4 seeds = 720 generations, ~2h.
./eval-mccarthy.sh

The design is FROZEN in scripts/mccarthy-corpus/GATE-PREREG.md, written and committed before a single generation existed (9c8a4e9, amended b4ba731). Do not edit the arms, the fixture size or the seeds to chase a result — re-run, do not re-tune.

Three arms: base (negative control + voice baseline), ckpt900 (ship candidate), ckpt450 (prior test — it is not tied, and the pre-registration says so). Axis A voice via voice_distance.py, axis B memorisation, axis C damage via score_beats.py.

deviation from the lv-hemingway gate	why
fixture --max-words 140 (script default 150)	the mccarthy register asks for 90–140 and score_beats.py scores in-band at 90–140. 207 of 269 val pairs are in-band, across all six works.
the run/pairs system-prompt equality is ASSERTED, not claimed in a comment	Hemingway's script said "verified" in prose, which no reader can check. Here a mismatch exits 1.
--punct-report --secondary-normalised	McCarthy-specific; see below.
axis B uses --train-only --heldout-reference	the base-unadapted control is defective; see below.

⚠ The voice axis needed settling before it could run

This corpus measures 0.0 quote marks per 10k against Hemingway's 838, and delta_cb is Burrows's Delta over CHARACTER BIGRAMS — so "emit no quotation marks" moves the metric a long way without having learned a sentence. Settled in three parts, all pre-registered:

1. PRIMARY — the mccarthy register NAMES the punctuation, and --system-from drives the base control with the same prompt. The cheap win is handed to both sides.
2. SECONDARY — --secondary-normalised re-runs everything with punctuation stripped. A conservative LOWER BOUND (it also strips sentence-length signal); reported, never the verdict.
3. TRIGGER — --punct-report checks whether base actually complied. Base quote density above 100 per 10k means it did not, the primary gap is partly the cheap win, and the normalised read is promoted to load-bearing. The line was fixed before the table printed.

⚠ The memorisation control shipped broken and is now fixed

memorization_check.py's negative control was base-unadapted, and the lv-hemingway gate established that this is defective: base writes summary, the adapted arms write pastiche, and text that does not imitate a register cannot collide with its n-grams. The correct innocent sample is the author himself — held-out text, same register by construction. That control was computed by hand for Hemingway and never committed. It is now --heldout-reference (requiring --train-only, or the val text is scored against a gram set containing itself, and the script refuses).

Validated by reproducing Hemingway's hand-computed numbers to the digit — 370 samples, hit-rate 0.01, mean-longest 0.1, max 10, 101-word median chunks.

⚠ An elevated rate is not by itself a no-ship — rate and exposure are different questions. Every matched run gets READ. Hemingway shipped at 7× the author's own rate because every match was stock dialogue, max 9 words, no proper noun. McCarthy is in copyright with a living estate, so a match carrying distinctive imagery or a proper noun is disqualifying in a way a rate number alone is not.

A log tag that reads backwards on this corpus

gen_beats_chat_yarros.py prints RAN-ON when its STOP regex finds no paragraph break in the raw output. That heuristic was written for the Yarros register. McCarthy's register asks for continuous scene prose, so a single unbroken block is the target, not a defect — expect the tag on most generations and do not read it as damage. The axis-C metric is score_beats.py --metric-source raw, where ran-on means words > 140, and it is unaffected.