bronte-corpus: gate lv-bronte for real — 0 of 365 with both controls green

The Brontë corpus's "0 of 203" was a HAND COUNT made before leak_gate.py
existed. On Yarros the automated instrument read 212 surviving where a hand
count said 86, so the hand count was never evidence. This runs the real gate,
and getting it to pass required fixing four defects the hand count could not
have seen.

CORPUS DEFECTS (repair_corpus_bronte.py, both measured):
  - 1,922 words of publisher back matter inside Shirley's last unit — a
    T. Nelson & Sons catalogue advertising Ainsworth, Marryat, Verne, Kingsley
    and Dickens, plus a Gutenberg transcriber's punctuation list. Not Brontë,
    and the source of the entity CHARLES. Same structural cause as the
    Hemingway run: a splitter cuts on headings, nothing follows the final one.
  - 1,368 Gutenberg italic spans. Two harms: they teach the adapter to emit
    underscores, and the underscore is a word character, so the gate's
    word-boundary scan cannot match inside an italicised name. An entity in
    italics is invisible to the gate — the same never-renamed-AND-never-
    reported shape as Yarros's possessive-only Afendra.

DETECTOR GAPS (phrase_map_bronte.json):
  - Blanche is 19 capitalised against ONE lowercase — ratio 0.0526, over the
    0.05 bar by a single token, so a named character with 19 mentions is
    dropped by a hair.
  - Grace (0.224) and Hollow (0.235) are refused correctly — both are common
    nouns — but Grace Poole and Hollow's Mill are Brontë's. Sampling all 21
    bare capitalised Grace found 20 are the character in direct address and
    exactly one is the theological noun.
  - Five compounds whose every component is non-renameable survive verbatim:
    Moor House, Marsh End, Vale Hall, Bigben Close, Royd Lane. The other 77
    audited phrases do not, because each has a renameable component.

GENDER (pin_known_gender.py): the inherited resolver put Jane MALE across 336
occurrences. Hemingway's base-rate resolver is strictly better here (1 wrong vs
4) but still fails on Jane, and the failure is structural, not tuning — Brontë's
three narrators are first-person, so their names appear almost only in dialogue
surrounded by other characters' pronouns. Ground truth is pinned separately from
the resolver's evaluation so the two are never conflated.

Also: min-cap lowered 8 to 3, which pulled Bertha, Ferndean, Rochesters and
Creemsvort in from below the old floor; corpus-scope rename so a name below
threshold in one novel is not printed verbatim there while renamed in another.

Gate: 0 of 365 surviving, positive control 365/365, negative control clean,
phrase audit 0 of 82. Floor stated: 3 capitals per work, 5 recurrences.
This commit is contained in:
vh
2026-09-16 20:51:11 -07:00
parent a5745dcf72
commit fc834a8a23
4 changed files with 300 additions and 0 deletions
+73
View File
@@ -0,0 +1,73 @@
"""D2c for lv-bronte — pin hand-verified gender onto the resolved entity map.
WHY THIS EXISTS, AND WHY IT IS NOT CIRCULAR. `gender_by_proximity.py` scores each
name's local pronoun mix against the corpus base rate, and on this corpus it beats
the inherited resolver where it counts: 1 wrong against 5. But it still resolves
`Jane` MALE (m=182 f=97), and that is not a tuning miss — it is structural. Brontë's
three narrators are first-person, so their names appear almost only in DIALOGUE,
spoken by other characters, surrounded by those characters' pronouns. Proximity
inference is therefore blind to exactly the characters the adapter is being trained
on. Hemingway's notes record the same blindness from the other side, and record that
Yarros's fix (the POV chapter header) does not transfer. Brontë's editions have no
such header either.
So the resolver's EVALUATION (how many did it get right, unaided) and the map we
actually TRAIN on are two different artefacts, and conflating them is the error to
avoid. This script does not improve the resolver's score and must never be quoted as
if it had. It applies ground truth that a human read out of the novels to the map
that goes downstream, and it reports every field it changed so the delta is visible.
Renaming `Jane` to a male name would invert the pronoun agreement of the single most
important text in the corpus, 336 times. That is the cost this step buys off.
"""
from __future__ import annotations
import argparse, json
from pathlib import Path
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--entities", required=True, help="entities json from gender_by_proximity.py")
ap.add_argument("--out", required=True)
ap.add_argument("--pin", required=True,
help="comma-separated Name=f|m|n ground truth, read from the novels. "
"`n` forces NEUTRAL where a surname is genuinely shared across sexes.")
a = ap.parse_args()
pins: dict[str, str | None] = {}
for item in (s.strip() for s in a.pin.split(",") if s.strip()):
name, _, g = item.partition("=")
if g not in ("f", "m", "n"):
print(f"== refusing: {item!r} — gender must be f, m or n")
return 2
pins[name.strip().lower()] = None if g == "n" else g
ents = json.loads(Path(a.entities).read_text())
changed, already, unseen = [], [], sorted(pins)
for slug, w in ents.items():
for key, e in w["entities"].items():
if key in pins:
if key in unseen:
unseen.remove(key)
was = e.get("gender")
if was != pins[key]:
e["gender"] = pins[key]
changed.append((slug, key, was, pins[key], e["cap"]))
else:
already.append((slug, key, was))
Path(a.out).write_text(json.dumps(ents, ensure_ascii=False, indent=2), encoding="utf-8")
print(f" pinned {len(pins)} names · changed {len(changed)} · already correct {len(already)}")
for slug, key, was, now, cap in sorted(changed, key=lambda r: -r[4]):
print(f" {key:<16} {slug:<14} {str(was):>4} -> {str(now):<4} ({cap} occurrences)")
if unseen:
# A pin that matches nothing is a typo or a stale name, and silently
# ignoring it would let the operator believe a character was fixed.
print(f" ⚠ {len(unseen)} pins matched NO entity: {unseen}")
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())