esh-pfi-infrastructure/scripts/bronte-corpus/RUNBOOK.md

lv-bronte — corpus pipeline, end to end

Charlotte Brontë: Jane Eyre, Villette, Shirley, The Professor. Four unrelated novels, 680,291 words as built, 678,367 after repair. Source is Project Gutenberg, so unlike the Yarros and Hemingway lines there is no licensed master and no distribution question.

$R is this repo's scripts/. Everything below runs on pfi-gx10 (10.100.50.60), which also holds the carrier and does the training.

Every stage refuses to write if its own controls fail, so a silent bad stage is not a failure mode here. Two stages actually did refuse during the first run, and both refusals were correct — see What the controls caught.

# D1b — repair the Gutenberg source. NOT optional; see below.
python3 $R/bronte-corpus/repair_corpus_bronte.py ~/r49-corpus --out corpus-clean

# D2 — entity map.
python3 $R/r49-corpus/entities.py corpus-clean --out entities.json \
  --stoplist $R/bronte-corpus/stoplist_bronte.json \
  --min-count 3 --min-mid-ratio 0.35 --fold-clitics --drop-acronyms \
  --control "Jane,Rochester,Thornfield,Lowood,Helen,Bessie,Adèle,Diana,Bertha,Lucy,Ginevra,Graham,Shirley,Caroline,Robert,Louis,Hortense,Yorke,Crimsworth,Frances,Hunsden,Pelet,Reed,Brocklehurst,Fairfax,Moore,Keeldar,Helstone,Malone,Sympson,Villette,Paulina,Terrasse,Vashti,Labassecour,Ferndean" \
  --negative-control "Indeed,Perhaps,However,Nevertheless,Certainly,Chapter,Yes,Well,Nothing,Something,Truly,Presently,Meantime,Again,Once,Besides,Moreover,Whether,Mdlle,Rue,Shakespeare,India,Dieu,Jesuit"

# D2b — gender by proximity, scored against the CORPUS BASE RATE.
python3 $R/hemingway-corpus/gender_by_proximity.py corpus-clean \
  --entities entities.json --out entities-gender.json --control "Jane=f,Rochester=m,…"

# D2c — pin hand-verified ground truth. Brontë-specific; see below.
python3 $R/bronte-corpus/pin_known_gender.py \
  --entities entities-gender.json --out entities-final.json --pin "Jane=f,Lucy=f,…,Moore=n,Helstone=n"

# D3 — rename. NOTE --scope corpus and --min-cap 3, both deviations.
python3 $R/r49-corpus/rename.py corpus-clean --entities entities-final.json \
  --dictionary ~/r49-prep/name_dictionary.json --out corpus-renamed \
  --preset bronte --scope corpus --min-cap 3 --copies 6 --seed 4919 \
  --phrase-map $R/bronte-corpus/phrase_map_bronte.json

# GATE — must pass before anything is trained.
python3 $R/r49-corpus/leak_gate.py corpus-clean --entities entities-final.json \
  --renamed corpus-renamed --min-cap 3 \
  --phrase-map $R/bronte-corpus/phrase_map_bronte.json \
  --report corpus-renamed/leak_gate_report.json

# D4 — instruction pairs. --source-entities is MANDATORY here; see below.
python3 $R/yarros-corpus/build_sft_pairs.py --corpus corpus-renamed/copies \
  --out pairs/pairs-full.jsonl --register bronte --model gen \
  --source-entities entities-final.json --n 4093 --split train --seed 4919
#   …and the same with --split val --n 200 --out pairs/pairs-val.jsonl

Gate result, 2026-09-16: 0 of 365 renameable surfaces survive, positive control 365/365, negative control clean, phrase audit 0 of 82. Sensitivity floor: a name under 3 capitals per work is never detected, a phrase under 5 recurrences never audited.

Why this corpus needed its own pipeline

Five deviations from the Yarros/Hemingway chain, each forced by a measurement.

deviation	why
D1b repair exists at all	1,922 words of publisher back matter in Shirley's last unit and 1,368 Gutenberg _italic_ spans. See below.
--min-count 3 / --min-cap 3 (was 8)	At 8, Bertha, Ferndean, Rochesters and Creemsvort sat below the floor — detected by nothing, renamed by nothing, reported by nothing.
--scope corpus (was work)	Jane is above threshold in Jane Eyre and The Professor but appears 5 times in Shirley. Work-scope renames it in two novels and prints it verbatim in the third. The gate is a whole-corpus scan and would have caught it; corpus scope means it never happens.
D2c pin_known_gender.py	The resolver puts Jane MALE across 336 occurrences. Structural, not tuning — see below.
--source-entities on the pair build	A leak class the gate cannot see — see below.

The two source defects

Publisher back matter, Shirley only, 1,922 words. A T. Nelson & Sons printer's block, a THE NELSON CLASSICS catalogue advertising Ainsworth, Marryat, Verne, Kingsley and Dickens, and a Gutenberg transcriber's punctuation list — all inside the final chapter record. Not Brontë, and the origin of the entity CHARLES (12 occurrences of another author's given name, in the training text). Structurally identical to the Hemingway back-matter defect and it arises the same way: a splitter cuts on headings and nothing follows the final one. The other three works are clean, which the repair script's per-work report shows as an implicit control.

Gutenberg _italic_ markup, 1,368 spans. Two harms. It teaches the adapter to emit underscores as prose. And _ is a word character, so the gate's \b(Name)\b scan cannot match inside _Antigua_ — an entity that appears only in italics is invisible to the gate, never renamed and never reported, which is the same failure shape as Yarros's possessive-only Afendra. Measured blast radius here is exactly one entity, but that is a fact about this corpus, not a property of the defect. ⚠ 24 of the spans wrap across a single newline (_sotto\nvoce_, _ignis\nfatuus_); a newline-free pattern silently leaves exactly those behind, and the script's read-back guard is what caught it.

Why gender needs a separate pinning stage

The inherited honorific/window resolver scores 19 correct / 5 held / 5 wrong. Hemingway's base-rate resolver is strictly better on the axis that matters — 17 correct / 11 held / 1 wrong — because a wrong gender actively corrupts pronoun agreement while a held one merely yields a neutral name. Brontë's corpus is nearly pronoun-balanced (15,588 male / 16,144 female, base_m 0.491), unlike Hemingway's near-4:1, so the base-rate correction does less work here but still helps.

The one it still gets wrong is Jane, 341 occurrences, the title character and narrator. This is not a tuning miss. Brontë's three narrators are first-person, so their names appear almost only in dialogue, spoken by other characters, surrounded by those characters' pronouns — proximity inference is structurally blind to exactly the characters the adapter is being trained on. Yarros solved its version with the POV chapter header; Brontë's editions have none, so that fix does not transfer any more than it transferred to Hemingway.

⚠ Keep the resolver's evaluation and the shipped map separate. pin_known_gender.py does not improve the resolver's score and must never be quoted as if it had. It applies ground truth a human read out of the novels. It also fails loudly on a pin that matches no entity, which caught a stale Hall=n pin on the first run.

Shared surnames are pinned neutral on purpose: Moore is Robert, Louis and Hortense; Helstone is Matthewson, James and Caroline; Yorke is Hiram, Mrs Yorke, Jessy, Rose and Martin. A gendered rename of a surname shared across sexes is wrong in one direction no matter which you pick.

⭐ The leak the gate cannot see

The corpus gate reads the corpus and the renamed copies. It never reads the generated beats. The beat is written by an LLM that just read the passage, and if it recognises the book it supplies the canonical names from its own training.

Measured on the first 714 Brontë pairs, before the filter existed: 13 beats (1.8%) named source characters — Rochester ×6, Jane ×3, Brocklehurst ×2, Beck, Fairfax, Helen, Burns, Eyre, Reed, Rivers — while 0 of 714 responses did. The rename was perfect and the instruction side was not. One beat read "Saoirse confirms Rochester's flaws, then agrees in English to marry him", mixing a renamed name and a canonical one in a single sentence.

The beat is the INSTRUCTION half of the pair, so training on it re-teaches exactly the inventions the rename exists to remove.

⚠ Exposure scales with how well the generator knows the book, so it is worst for public-domain classics and mildest for recent work. That is why Yarros and Hemingway came up clean and Brontë did not — their clean runs are not evidence this cannot happen to them. Pass --source-entities on every corpus, and rebuild those two with it if they are ever regenerated.

What the controls caught

Worth recording, because in three of four cases the control was wrong and the detector was right — the opposite of the reflex.

• Adele NOT DETECTED. The corpus spells it Adèle (135×). My control string was ASCII. Detector right.
• Hollow NOT DETECTED. 85 capitalised against 20 genuinely lowercase (rushing down the hollow), ratio 0.235. Detector right; Hollow's Mill belongs in the phrase map, which is where it went.
• Grace NOT DETECTED. 0.224 lowercase ratio — grace is a common noun. Detector right. But sampling all 21 bare capitalised Grace found 20 are the character in direct address and one is theological, so it is token-mapped, with that single collateral substitution accepted and recorded.
• Blanche NOT DETECTED — and here the detector was wrong. 19 capitalised against one lowercase gives 0.0526, over the 0.05 bar by a single token. A named character with 19 mentions, dropped by a hair. Token-mapped.
• Antigua missing from the positive control at gate time. This is what exposed the italics defect above.