vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

BabyYarros: the leak gate passes, and it found three defects nobody was looking for

Browse Source 
The gate is new. There was no committed instrument for "does any of the author's
own proper nouns survive the rename" -- the Brontë number was produced by hand
-- so leak_gate.py is now that instrument, and it runs both directions every
time: the same scan over the unrenamed source as a positive control, and a nonce
string as a negative one. A detector that only ever sees renamed text cannot
distinguish absent from blind.

Run against BabyYarros as built it reported 212 surviving entities, not the 86
recorded earlier, because it scans the whole corpus rather than each work
separately and it counts the sub-threshold entities rename never looked at.
Three findings came out of closing that.

The corpus had a typography defect of its own. The D1 notes correctly say no
unwrap was needed; a different defect was there instead. The Empyrean books set
their chapter epigraphs in small caps and the extractor rendered the run as
uppercase while leaving the large initial as a separate token, so the corpus
carried "M AJOR A FENDRA'S G UIDE TO THE R IDERS Q UADRANT" -- 106 lines, ~700
splits -- plus 52 drop caps like "T he flight field". That is where the entities
called IDERS, UADRANT, NAUTHORIZED and seventeen bare single letters came from.
A split initial next to an uppercased run is enough to recover the original
mixed case, so the restore is exact rather than approximate: a word with a split
initial was capitalised, an all-caps word without one was lowercase.

Back matter was inside the prose. The builder splits on chapter headings and
nothing follows the last one, so every work carried its acknowledgments,
newsletter pitches and cover-artist credits -- 4,555 words naming the author's
agent, editors and children, in a corpus whose entire purpose is that no
identifiable name survives.

And the gate passed at 0 of 314 while Afendra was still in every copy. The name
never appears unpossessed, so it keyed as an apostrophe form, and rename and the
gate both skip those as contractions -- unrenamed and unreported at once, which
is the worst failure shape available. Baxter escaped a different way: wilder
renders an in-book news article entirely in lowercase, putting the cap/lowercase
ratio at 0.13 against a 0.05 bar.

Then a second class the unigram scan structurally cannot see. Riders Quadrant,
Flame Section, War Games and Fourth Wing -- the book's own title -- are built
from ordinary words the detector correctly refuses to call names. The gate now
audits recurring capitalised 2-3grams against an explicit allow list, and
rename applies a phrase map after the entity pass.

Every new detector flag is opt-in and off by default, and the Brontë entity map
was re-derived after each change and confirmed identical in keys, surfaces and
every field. The stoplist was built by reading each surface in context, which is
why it is short: Violence is Xaden's nickname for Violet, and Continent,
Presentation, Barrens, Originals, Montserrat, Athena, Aura, Curator and Sage are
all in-world. A plausible-looking guess would have excluded most of them.

Final: 0 of 325 entities and 0 of 91 audited phrases survive in any of 30 copy
files, both controls passing. The sensitivity floor is stated in the gate's own
output -- 3 occurrences for a name, 5 for a phrase -- because a negative without
one is unfalsifiable.
This commit is contained in:
vh
2026-09-11 10:06:04 -07:00
parent 16c144fcda
commit 7b0580dcbe
7 changed files with 903 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/r49-corpus/entities.py
+243 -9
View File
@@ -19,6 +19,20 @@ trained on.
Nothing here guesses. Unresolved entities block corpus emission and go to a human
pass: held is cheap, wrong is poison -- a silently mis-gendered entity scrambles
pronoun agreement through every renamed copy and nothing downstream would catch it.
⚠ v4, added for BabyYarros: a MID-SENTENCE test on top of the ratio.
The cap/lowercase ratio calls `Hey`, `Holy`, `Hopefully`, `Yep`, `Whoa`, `Nope`
and `Ugh` names, because a dialogue-heavy contemporary novel opens sentences with
them constantly and never writes them lowercase. The v1 lesson was that POSITION
ALONE misses names that start sentences; position as a SECOND filter has no such
problem, because a real name also appears mid-sentence. Measured on BabyYarros the
two populations do not overlap: 33 verified names sit at 0.567-0.985 mid-sentence,
and 19 verified interjections at 0.000-0.222. The gap is 2.5x wide, so the
threshold is not a tuned parameter.
It is OPT-IN (`--min-mid-ratio`, default 0 = off) so the Brontë run stays
byte-reproducible. A 19th-century novel does not have this failure mode in the
same volume, and an unmeasured change to a settled corpus is not an improvement.
"""
from __future__ import annotations
import argparse, collections, json, re, sys
@@ -38,6 +52,9 @@ STOP_TITLES = {
"Grandmother", "Grandfather", "Nurse", "King", "Queen", "Prince", "Princess",
"Duke", "Duchess", "Earl", "Count", "Countess", "Baron", "Squire", "Parson",
"Monseigneur", "Mlle", "Mme", "M", "Messrs",
# modern ranks and address forms, added for BabyYarros
"Sergeant", "Sgt", "Lieutenant", "Lt", "Corporal", "Admiral", "Commander",
"Cadet", "Officer", "Agent", "Coach", "Senator", "Majesty", "Highness",
}
#: Days, months, and the language/nation adjectives a 19th-century novel is full
#: of. All are always-capitalised and would otherwise pass the ratio test.
@@ -56,7 +73,15 @@ STOP_COMMON = {
"Who","When","Where","Why","How","If","So","As","At","In","On","To","For","Of",
"Nay","Alas","Madam","Sir","Mademoiselle","Monsieur",
}
STOP = STOP_TITLES | STOP_COMMON
#: Structural words from the book's own apparatus. `Chapter` and `Article` pass
#: both the ratio test and the mid-sentence test -- `BONUS CONTENT Chapter Nine`
#: and `Article Three` put them mid-sentence -- and renaming them would rewrite
#: the corpus's own scaffolding.
STOP_STRUCTURAL = {
"Chapter", "Chapters", "Prologue", "Epilogue", "Part", "Appendix", "Volume",
"Article", "Section", "Contents", "Content", "Bonus", "Preface", "Interlude",
}
STOP = STOP_TITLES | STOP_COMMON | STOP_STRUCTURAL
MALE_PRON = {"he", "him", "his", "himself"}
FEM_PRON = {"she", "her", "hers", "herself"}
@@ -71,11 +96,25 @@ def load(corpus: Path) -> dict[str, str]:
return out
def detect(text: str, min_count: int, max_ratio: float) -> dict[str, dict]:
"""Corpus-level capitalised-vs-lowercase ratio. See module docstring."""
#: `’s` is a possessive and the rest are contractions; none of them is part of the
#: name. TOKEN keeps the apostrophe, so without folding `Afendra’s` is its own key.
CLITIC = re.compile(r"[’'](?:s|d|ll|ve|re|m|t)$", re.I)
def detect(text: str, min_count: int, max_ratio: float, fold_clitics: bool = False) -> dict[str, dict]:
"""Corpus-level capitalised-vs-lowercase ratio. See module docstring.
⚠ `fold_clitics` folds `Afendra’s` into `Afendra`. Without it an entity that
NEVER appears unpossessed is keyed with the apostrophe, and both rename.py and
the leak gate skip apostrophe keys as contractions -- so it is never renamed
AND never reported. Measured on BabyYarros: `Afendra` survived every copy
while the gate read 0 of 314, which is the worst failure shape there is.
"""
cap, low = collections.Counter(), collections.Counter()
for m in TOKEN.finditer(text):
t = m.group(0)
if fold_clitics:
t = CLITIC.sub("", t) or t
(cap if t[:1].isupper() else low)[t.lower()] += 1
ents = {}
for key, c in cap.items():
@@ -90,13 +129,128 @@ def detect(text: str, min_count: int, max_ratio: float) -> dict[str, dict]:
return ents
def surface_forms(text: str, keys: set[str]) -> dict[str, str]:
#: Whatever can sit between a sentence terminator and the first word of the next
#: sentence: whitespace, opening quotes, brackets, a dash.
_OPENERS = set(' \t\n\u201c\u201d"\'\u2018\u2019([{\u2014\u2013-*')
_TERM = set('.!?\u2026')
def mid_sentence(text: str, keys: set[str], fold_clitics: bool = False) -> tuple[dict[str, int], dict[str, int]]:
"""(mid, total) capitalised occurrences per key.
`mid` counts the ones whose preceding non-opener character is not a sentence
terminator -- i.e. the capital is the writer's choice and not the position's.
"""
mid, tot = collections.Counter(), collections.Counter()
for m in TOKEN.finditer(text):
t = m.group(0)
if fold_clitics:
t = CLITIC.sub("", t) or t
if not t[:1].isupper():
continue
k = t.lower()
if k not in keys:
continue
tot[k] += 1
i = m.start() - 1
while i >= 0 and text[i] in _OPENERS:
i -= 1
if i >= 0 and text[i] not in _TERM:
mid[k] += 1
return mid, tot
#: A word carrying one of these in front of it is a name, whatever its position
#: statistics say. This is rename.py's title-first idea used as a RESCUE rather
#: than as a gender signal.
_HONORIFIC = (r"(?:Mr|Mrs|Ms|Miss|Dr|Doctor|Professor|Prof|Colonel|Col|Major|General|Gen|"
r"Captain|Capt|Lieutenant|Lt|Sergeant|Sgt|Cadet|Sir|Madam|Lady|Lord|King|Queen|"
r"Officer|Agent|Coach|Senator|Judge|Father|Mother|Aunt|Uncle)")
def rescue_signals(text: str, keys: set[str]) -> dict[str, tuple[int, int]]:
"""key -> (honorific-preceded, possessive) counts.
⚠ The mid-sentence filter drops real SURNAMES that are only ever used as
address -- measured here, `Delgado` 18/64, `Schur` 0/10, `Rhee` 0/8, because
every occurrence is `“Mr. Delgado,”` opening a line of dialogue. Two signals
separate those from the interjections the filter is FOR: a title in front,
and a possessive. Measured on BabyYarros, all 19 verified interjections score
zero on both, and every wrongly-dropped surname scores on at least one.
"""
hon, poss = collections.Counter(), collections.Counter()
for m in re.finditer(_HONORIFIC + r"\.?\s+([A-ZÀ-Þ][A-Za-zà-ÿœæ\-]+)", text):
k = m.group(1).lower()
if k in keys:
hon[k] += 1
for m in re.finditer(r"\b([A-ZÀ-Þ][A-Za-zà-ÿœæ\-]+)[’\']s\b", text):
k = m.group(1).lower()
if k in keys:
poss[k] += 1
return {k: (hon[k], poss[k]) for k in keys}
ACRONYM = re.compile(r"[A-Z]{2,}s?$")
def ratio_rejects(text: str, min_count: int, max_ratio: float, fold_clitics: bool) -> dict[str, dict]:
"""Candidates frequent enough to matter that the cap/lowercase ratio threw out.
⚠ The ratio assumes consistent typography and BabyYarros breaks that: `wilder`
renders an in-book news article entirely in lowercase, so `eleanor baxter` and
`ms. baxter` appear uncapitalised three times against 23 capitalised ones --
ratio 0.13 against a 0.05 bar, and a real character is silently never renamed.
"""
cap, low = collections.Counter(), collections.Counter()
for m in TOKEN.finditer(text):
t = m.group(0)
if fold_clitics:
t = CLITIC.sub("", t) or t
(cap if t[:1].isupper() else low)[t.lower()] += 1
return {k: {"cap": c, "lower": low[k], "ratio": round(low[k] / c, 4)}
for k, c in cap.items() if c >= min_count and low[k] / c > max_ratio}
#: ⚠ DELIBERATELY NARROWER than `_HONORIFIC`. The wide list is safe when both
#: sides must be capitalised; matched case-insensitively it readmitted 143 junk
#: tokens (`the`, `says`, `like`, `up`) because `major`, `general`, `father`,
#: `sir` and `agent` are ordinary words in lowercase prose. These five are never
#: anything but a title, and the lowercase arm additionally REQUIRES the period.
_ABBREV = re.compile(r"\b(?:Mr|Mrs|Ms|Dr|Mister|Miss)\b\.?\s+([A-ZÀ-Þ][A-Za-zà-ÿœæ\-]+)"
r"|\b(?:mr|mrs|ms|dr)\.\s+([a-zà-ÿœæ][a-zà-ÿœæ\-]+)")
def honorific_hits(text: str, keys: set[str]) -> dict[str, int]:
"""`Miss Baxter` and `ms. baxter` both count; `I miss you` does not."""
hits = collections.Counter()
for m in _ABBREV.finditer(text):
k = (m.group(1) or m.group(2)).lower()
if k in keys:
hits[k] += 1
return hits
def surface_forms(text: str, keys: set[str], prefer_mixed: bool = False,
fold_clitics: bool = False) -> dict[str, str]:
"""Dominant spelling per key.
⚠ `prefer_mixed` picks the most common NON-all-caps form when one exists.
Without it a name that happens to sit inside an all-caps passage -- an
in-world dispatch here, an inscription in Shirley -- gets `BRAEVICK` as its
surface, and every rule downstream then reasons about an acronym.
"""
best = collections.defaultdict(collections.Counter)
for m in TOKEN.finditer(text):
t = m.group(0)
if fold_clitics:
t = CLITIC.sub("", t) or t
if t[:1].isupper() and t.lower() in keys:
best[t.lower()][t] += 1
return {k: c.most_common(1)[0][0] for k, c in best.items()}
out = {}
for k, c in best.items():
mixed = [(n, f) for f, n in c.most_common() if not ACRONYM.fullmatch(f)]
out[k] = (max(mixed)[1] if (prefer_mixed and mixed) else c.most_common(1)[0][0])
return out
def link_identities(text: str, names: set[str], min_pairs: int) -> list[tuple[str, str]]:
@@ -149,21 +303,75 @@ def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("corpus")
ap.add_argument("--out", default=None)
ap.add_argument("--stoplist", default=None,
help="JSON file whose every list value holds surfaces to exclude; "
"per-corpus real-world referents, see stoplist_yarros.json")
ap.add_argument("--rescue-honorific", type=int, default=0,
help="readmit a candidate the cap/lowercase ratio rejected when a title "
"precedes it at least this many times (0 = off)")
ap.add_argument("--fold-clitics", action="store_true",
help="count `Afendra’s` as `Afendra` so a possessive-only entity is "
"detected at all (it is otherwise silently unrenamed AND ungated)")
ap.add_argument("--drop-acronyms", action="store_true",
help="treat an ALWAYS-all-caps surface as an acronym, not a name "
"(RSC/ATV/TV/BMX/VIP), and prefer a mixed-case surface when one exists")
ap.add_argument("--min-count", type=int, default=5)
ap.add_argument("--max-ratio", type=float, default=0.05)
ap.add_argument("--min-pairs", type=int, default=2)
ap.add_argument("--min-mid-ratio", type=float, default=0.0,
help="drop a candidate whose capitals are overwhelmingly sentence-initial "
"(0 = off, which reproduces the Bronte run)")
ap.add_argument("--min-mid", type=int, default=2,
help="absolute mid-sentence floor, so a 1-of-2 accident cannot qualify")
ap.add_argument("--control", default="", help="comma-separated known-true names (positive control)")
ap.add_argument("--negative-control", default="",
help="comma-separated known-NON-names that the filter must DROP")
a = ap.parse_args()
corpus = Path(a.corpus)
works = load(corpus)
stop = set(STOP)
if a.stoplist:
blob = json.loads(Path(a.stoplist).read_text())
extra = {n for v in blob.values() if isinstance(v, list) for n in v}
stop |= extra
print(f" stoplist {a.stoplist}: +{len(extra)} real-world / generic surfaces")
controls = [c.strip() for c in a.control.split(",") if c.strip()]
neg_controls = [c.strip() for c in a.negative_control.split(",") if c.strip()]
report, failed_control = {}, []
mid_dropped: dict[str, tuple[int, int]] = {}
rescued: dict[str, tuple[int, int]] = {}
ratio_rescued: dict[str, tuple[int, int, int]] = {}
for slug, text in works.items():
ents = detect(text, a.min_count, a.max_ratio)
keys = {k for k in ents if k.capitalize() not in STOP and k.title() not in STOP}
keys = {k for k in keys if k not in {s.lower() for s in STOP}}
forms = surface_forms(text, keys)
ents = detect(text, a.min_count, a.max_ratio, a.fold_clitics)
if a.rescue_honorific:
rej = ratio_rejects(text, a.min_count, a.max_ratio, a.fold_clitics)
hh = honorific_hits(text, set(rej))
back = {k: rej[k] for k, n in hh.items() if n >= a.rescue_honorific}
for k, v in back.items():
ents.setdefault(k, v)
ratio_rescued[k] = (hh[k], v["cap"], v["lower"])
keys = {k for k in ents if k.capitalize() not in stop and k.title() not in stop}
keys = {k for k in keys if k not in {s.lower() for s in stop}}
# ⚠ An all-caps surface is an acronym, not a name: RSC, ATV, TV, BMX, VIP,
# CTDs. Tested on the DOMINANT surface form, because a name also appears
# inside an all-caps in-world dispatch and must not be lost to that.
if a.drop_acronyms:
forms0 = surface_forms(text, keys, prefer_mixed=True, fold_clitics=a.fold_clitics)
keys = {k for k in keys if not ACRONYM.fullmatch(forms0.get(k, k))}
if a.min_mid_ratio > 0:
mid, tot = mid_sentence(text, keys, a.fold_clitics)
dropped_here = {k for k in keys
if mid[k] < a.min_mid or mid[k] / max(tot[k], 1) < a.min_mid_ratio}
sig = rescue_signals(text, dropped_here)
rescued_here = {k for k in dropped_here if sum(sig.get(k, (0, 0))) > 0}
for k in rescued_here:
rescued[k] = sig[k]
dropped_here -= rescued_here
for k in dropped_here:
mid_dropped[k] = (mid[k], tot[k])
keys -= dropped_here
forms = surface_forms(text, keys, prefer_mixed=a.drop_acronyms, fold_clitics=a.fold_clitics)
links = link_identities(text, keys, a.min_pairs)
gender = resolve_gender(text, keys)
# identity linking propagates gender: a bare surname inherits from its given name
@@ -180,6 +388,32 @@ def main() -> int:
f"{sum(1 for k in keys if gender.get(k)):>3} gendered "
f"{sum(1 for k in keys if not gender.get(k)):>4} ungendered")
if ratio_rescued:
print(f"\n ratio-rejected but title-preceded, readmitted: {len(ratio_rescued)}")
for k, (h, c, l) in sorted(ratio_rescued.items(), key=lambda kv: -kv[1][0]):
print(f" {k:<16} {h:>3} titled · {c:>4} cap / {l:>3} lower")
if a.min_mid_ratio > 0:
print(f"\n mid-sentence filter (>= {a.min_mid} and >= {a.min_mid_ratio:.2f} of capitals): "
f"dropped {len(mid_dropped)} candidates")
for k, (m, t) in sorted(mid_dropped.items(), key=lambda kv: -kv[1][1])[:20]:
print(f" {k:<16} {m:>4} mid / {t:>4} caps")
if len(mid_dropped) > 20:
print(f" ... and {len(mid_dropped) - 20} more")
print(f" rescued by honorific/possessive: {len(rescued)}")
for k, (h, po) in sorted(rescued.items(), key=lambda kv: -sum(kv[1])):
print(f" {k:<16} {h:>3} titled · {po:>3} possessive")
if neg_controls:
print("\n negative control -- these are NOT names and must be DROPPED:")
for name in neg_controls:
hits = [s for s, r in report.items() if name.lower() in r["entities"]]
ok = not hits
print(f" [{'PASS' if ok else 'FAIL'}] {name:<14} "
f"{'dropped' if ok else 'STILL AN ENTITY in ' + ', '.join(hits)}")
if not ok:
failed_control.append(f"{name} (negative)")
if controls:
print("\n positive control -- names known to be real must be FOUND:")
for name in controls:
scripts/r49-corpus/leak_gate.py
+187
View File
@@ -0,0 +1,187 @@
"""R49 Stage D3 gate — do any of the author's own proper nouns survive the rename?
The rename exists so a voice adapter fits *prose style* and not the author's
characters and worldbuilding. That only holds if the renamed copies are actually
clean, and "actually clean" is a measurement, not a property of having run the
script. Brontë's run reached 0 of 203; BabyYarros opened at 86 of 232.
The gate is a whole-corpus scan, not a per-work one, and that distinction is
load-bearing. A name detected in `iron-flame` but below threshold in `fourth-wing`
is renamed in one copy and printed verbatim in the other, and a per-work gate
reports that as clean.
CONTROLS. A detector that only ever sees the renamed text cannot tell "absent"
from "blind", so this instrument runs both directions every time:
* POSITIVE -- the same scan over the UNRENAMED source. Every entity must be
found there. A miss means the matcher is broken and its zeroes are worthless.
* NEGATIVE -- a nonce string that appears in neither tree. A hit means the
matcher is manufacturing signal.
Exit code is the gate: 0 iff the controls pass AND no source entity survives.
"""
from __future__ import annotations
import argparse, json, re, sys
from collections import Counter, defaultdict
from pathlib import Path
NONCE = "Qxzvwolfram" # negative control: appears in no corpus
def load_works(corpus: Path) -> dict[str, str]:
man = json.loads((corpus / "manifest.json").read_text())
out = {}
for w in man["works"]:
rows = [json.loads(l) for l in
(corpus / w["path"]).read_text(encoding="utf-8").splitlines() if l.strip()]
out[w["slug"]] = "\n\n".join(r["text"] for r in rows)
return out
def load_copies(renamed: Path) -> dict[str, str]:
out = {}
for p in sorted((renamed / "copies").glob("*.jsonl")):
rows = [json.loads(l) for l in p.read_text(encoding="utf-8").splitlines() if l.strip()]
out[p.name] = "\n\n".join(r["text"] for r in rows)
return out
def scan(texts: dict[str, str], surfaces: list[str]) -> dict[str, dict[str, int]]:
"""surface -> {text_name: hits}. One alternation pass per text, not one per name.
⚠ Longest-first alternation, so `Xaden Riorson` is consumed before `Xaden`
and a two-part name is not counted twice.
"""
if not surfaces:
return {}
pat = re.compile(r"\b(" + "|".join(re.escape(s) for s in
sorted(surfaces, key=len, reverse=True)) + r")\b")
hits: dict[str, dict[str, int]] = defaultdict(dict)
for name, text in texts.items():
local: dict[str, int] = defaultdict(int)
for m in pat.finditer(text):
local[m.group(1)] += 1
for s, n in local.items():
hits[s][name] = n
return hits
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("corpus", help="source corpus dir (manifest.json + works/)")
ap.add_argument("--entities", required=True)
ap.add_argument("--renamed", required=True, help="rename.py --out dir")
ap.add_argument("--min-cap", type=int, default=8,
help="rename.py's renameable threshold; entities below it are "
"reported separately because rename never touched them")
ap.add_argument("--phrase-map", default=None,
help="the JSON rename.py used; its `allow` list names the phrases judged "
"real-world or generic. Without it the phrase audit does not run.")
ap.add_argument("--phrase-min", type=int, default=5,
help="a capitalised 2-3gram must recur this often in the source to be audited")
ap.add_argument("--report", default=None, help="write the full JSON breakdown here")
a = ap.parse_args()
corpus, renamed = Path(a.corpus), Path(a.renamed)
ents_all = json.loads(Path(a.entities).read_text())
source = load_works(corpus)
copies = load_copies(renamed)
if not copies:
print("== no copy files found -- nothing to gate"); return 1
# Mirror rename.py's own renameable predicate so the two cannot drift apart.
renameable, sub_threshold = {}, {}
for slug, w in ents_all.items():
for key, e in w["entities"].items():
surf = e.get("surface") or key
if "’" in key or "'" in key:
continue
(renameable if e["cap"] >= a.min_cap else sub_threshold).setdefault(surf, set()).add(slug)
surfaces = sorted(set(renameable) | set(sub_threshold))
print(f" {len(renameable)} renameable surfaces (cap >= {a.min_cap}) · "
f"{len(sub_threshold)} sub-threshold · {len(copies)} copy files")
# ---- controls --------------------------------------------------------
src_hits = scan(source, surfaces + [NONCE])
missing = [s for s in surfaces if s not in src_hits]
pos_ok = not missing
neg_ok = NONCE not in src_hits
print(f" [{'PASS' if pos_ok else 'FAIL'}] positive control: every surface found in the "
f"unrenamed source ({len(surfaces) - len(missing)}/{len(surfaces)})"
+ ("" if pos_ok else f" -- MISSING {missing[:10]}"))
print(f" [{'PASS' if neg_ok else 'FAIL'}] negative control: nonce `{NONCE}` absent from source")
# ---- the measurement -------------------------------------------------
copy_hits = scan(copies, surfaces + [NONCE])
neg_ok = neg_ok and NONCE not in copy_hits
surv_renameable = {s: copy_hits[s] for s in renameable if s in copy_hits}
surv_sub = {s: copy_hits[s] for s in sub_threshold if s in copy_hits}
print(f"\n SURVIVING renameable: {len(surv_renameable)} of {len(renameable)}")
for s, where in sorted(surv_renameable.items(), key=lambda kv: -sum(kv[1].values()))[:40]:
tot = sum(where.values())
print(f" {s:<18} {tot:>6} hits across {len(where)} copies "
f"(detected in: {','.join(sorted(renameable[s]))})")
if len(surv_renameable) > 40:
print(f" ... and {len(surv_renameable) - 40} more")
print(f"\n SURVIVING sub-threshold (cap < {a.min_cap}, rename never saw them): "
f"{len(surv_sub)} of {len(sub_threshold)}")
for s, where in sorted(surv_sub.items(), key=lambda kv: -sum(kv[1].values()))[:15]:
print(f" {s:<18} {sum(where.values()):>6} hits")
# ---- phrase audit ----------------------------------------------------
# ⚠ The unigram scan above cannot see `Riders Quadrant` or `Fourth Wing`:
# every component is an ordinary word the detector correctly refuses. This
# pass is what caught them AFTER the unigram gate read 0 of 314.
surviving_phrases = {}
if a.phrase_map:
pm = json.loads(Path(a.phrase_map).read_text())
allow = set(pm.get("allow", []))
PH = re.compile(r"\b([A-Z][a-z]{2,}(?: [A-Z][a-z]{2,}){1,2})\b")
src_ph = Counter()
for t in source.values():
src_ph.update(PH.findall(t))
cop_ph = Counter()
for t in copies.values():
cop_ph.update(PH.findall(t))
# A heading word cannot start a leak: `Chapter Twenty` is the book's own
# scaffolding, not the author's invention.
STRUCT = ("Chapter", "Prologue", "Epilogue", "Part", "Appendix", "Volume", "Book")
audited = {p for p, n in src_ph.items()
if n >= a.phrase_min and not p.startswith(STRUCT)} - allow
surviving_phrases = {p: {"source": src_ph[p], "copies": cop_ph[p]}
for p in audited if cop_ph[p] > 0}
print(f"\n PHRASE AUDIT: {len(audited)} capitalised 2-3grams recur >= {a.phrase_min} "
f"times in the source ({len(allow)} allow-listed as real-world/generic)")
print(f" SURVIVING phrases: {len(surviving_phrases)}")
for ph, w in sorted(surviving_phrases.items(), key=lambda kv: -kv[1]["source"])[:30]:
print(f" {ph:<34} source {w['source']:>4} copies {w['copies']:>5}")
if a.report:
Path(a.report).write_text(json.dumps({
"renameable_total": len(renameable), "sub_threshold_total": len(sub_threshold),
"controls": {"positive_pass": pos_ok, "negative_pass": neg_ok, "missing": missing},
"surviving_renameable": {s: {"hits": sum(w.values()), "copies": len(w),
"detected_in": sorted(renameable[s])}
for s, w in surv_renameable.items()},
"surviving_sub_threshold": {s: {"hits": sum(w.values()), "copies": len(w)}
for s, w in surv_sub.items()},
"surviving_phrases": surviving_phrases,
}, ensure_ascii=False, indent=2), encoding="utf-8")
print(f"\n wrote {a.report}")
if not (pos_ok and neg_ok):
print("\n== CONTROLS FAILED -- this gate's verdict is not trustworthy"); return 2
if surv_renameable or surv_sub or surviving_phrases:
print(f"\n== GATE FAILED: {len(surv_renameable) + len(surv_sub)} source entities and "
f"{len(surviving_phrases)} phrases survive"); return 1
print("\n== GATE PASSED: 0 source entities and 0 audited phrases survive in any copy")
print(f" ⚠ sensitivity floor: a name appearing fewer than {a.min_cap} times per work is "
f"never detected, and a phrase recurring fewer than {a.phrase_min} times is never "
f"audited. Neither is renamed, and neither is reported here.")
return 0
if __name__ == "__main__":
sys.exit(main())
scripts/r49-corpus/rename.py
+65 -3
View File
@@ -98,6 +98,15 @@ def main() -> int:
ap.add_argument("--holdout-chapter", type=int, default=10)
ap.add_argument("--preset", default="bronte", choices=sorted(PRESETS),
help="which corpus's name-pool register to draw from")
ap.add_argument("--scope", default="work", choices=("work", "corpus"),
help="`work` maps each work independently (reproduces the Bronte run); "
"`corpus` uses ONE map across every work in a copy")
ap.add_argument("--phrase-map", default=None,
help="JSON with `phrases` (multiword) and `tokens` (capitalised single "
"words) neutralising in-world compounds the unigram pass cannot reach")
ap.add_argument("--min-cap", type=int, default=8,
help="minimum capitalised count for an entity to be renamed; below it "
"the entity is left in the text verbatim")
a = ap.parse_args()
corpus = Path(a.corpus)
@@ -137,6 +146,21 @@ def main() -> int:
f"{lab} {len(pool[lab]['male'])}m/{len(pool[lab]['female'])}f/{len(pool[lab]['surname'])}s"
for lab in (label_a, label_b)))
# ⚠ Applied AFTER the entity substitution, so it can never eat a replacement
# name. Multiword first and longest first; single tokens are case-SENSITIVE
# and whole-word, so a dragon's lowercase `wing` survives while `Fourth Wing`
# does not.
phrase_sub = None
if a.phrase_map:
pm = json.loads(Path(a.phrase_map).read_text())
table = {**pm.get("phrases", {}), **pm.get("tokens", {})}
if table:
pat_p = re.compile(r"\b(" + "|".join(re.escape(k) for k in
sorted(table, key=len, reverse=True)) + r")\b")
phrase_sub = lambda t: pat_p.sub(lambda m: table[m.group(1)], t)
print(f" phrase map {a.phrase_map}: {len(pm.get('phrases', {}))} phrases + "
f"{len(pm.get('tokens', {}))} capitalised tokens")
out = Path(a.out); (out / "copies").mkdir(parents=True, exist_ok=True)
stats = {"copies": a.copies, "seed": a.seed, "works": {}, "renamed": 0, "held": 0}
@@ -154,7 +178,7 @@ def main() -> int:
titled = set(tg)
renameable, held = {}, []
for key, e in ents.items():
if "’" in key or "'" in key or e["cap"] < 8:
if "’" in key or "'" in key or e["cap"] < a.min_cap:
continue # possessives/contractions are not entities
g = tg.get(key) or e.get("gender")
if g:
@@ -176,16 +200,47 @@ def main() -> int:
stats["renamed"] += len(renameable); stats["held"] += len(held)
print(f" {slug:<14} renamed {len(renameable):>3} ({len(renameable)-len(held)} gendered, {len(held)} neutral)")
# ⚠ CORPUS SCOPE. Per-work maps leak across works and this is measurable, not
# theoretical: `Rebel` is detected in `rebel` and renamed there, then printed
# verbatim in the two Renegades books where it sits below threshold. A
# whole-corpus gate catches it; a per-work one reports clean. It also fixes a
# thing the Bronte corpus never had to care about -- Yarros is TWO SERIES, so
# Violet has to be the same person in Fourth Wing and Iron Flame, and a
# per-work draw gives her two different names inside one copy.
if a.scope == "corpus":
merged: dict[str, dict] = {}
genders: dict[str, set] = collections.defaultdict(set)
for slug, plan in plans.items():
for key, v in plan.items():
merged.setdefault(key, {"surface": v["surface"], "kind": v["kind"], "gender": None})
if v["gender"]:
genders[key].add(v["gender"])
conflicts = 0
for key, v in merged.items():
g = genders.get(key, set())
if len(g) == 1:
v["gender"] = next(iter(g)); v["kind"] = "given"
else:
if len(g) > 1:
conflicts += 1
v["kind"] = "surname" # held -> neutral pool, still renamed
n_gendered = sum(1 for v in merged.values() if v["gender"])
print(f" corpus scope: {len(merged)} distinct surfaces "
f"({n_gendered} gendered, {len(merged) - n_gendered} neutral), "
f"{conflicts} gender conflicts held")
plans = {slug: merged for slug in plans}
# ---- D3: N seeded copies, one consistent map per copy -------------------
emitted = 0
for c in range(a.copies):
rng = random.Random(a.seed + c * 1000)
corpus_map, corpus_used = {}, set()
for slug, rows in works.items():
# Share of pool A for this work. Brontë sets it per novel (Brussels
# vs Yorkshire); Yarros uses one default, because the register does
# not split by book the way hers does.
share_a = cfg["share"].get(slug, cfg["default_share"])
used = set()
used = corpus_used if a.scope == "corpus" else set()
def draw(kind: str, gender: str | None) -> str:
lang = label_a if rng.random() < share_a else label_b
@@ -196,7 +251,12 @@ def main() -> int:
used.add(n); return n
return rng.choice(pool[lang][bucket])
mapping = {k: draw(v["kind"], v["gender"]) for k, v in plans[slug].items()}
if a.scope == "corpus":
for k, v in plans[slug].items():
corpus_map.setdefault(k, draw(v["kind"], v["gender"]))
mapping = corpus_map
else:
mapping = {k: draw(v["kind"], v["gender"]) for k, v in plans[slug].items()}
pat = re.compile(r"\b(" + "|".join(sorted((re.escape(v["surface"]) for v in plans[slug].values()),
key=len, reverse=True)) + r")\b")
surf2key = {v["surface"]: k for k, v in plans[slug].items()}
@@ -204,6 +264,8 @@ def main() -> int:
with path.open("w", encoding="utf-8") as fh:
for r in rows:
txt = pat.sub(lambda m: mapping[surf2key[m.group(1)]], r["text"])
if phrase_sub:
txt = phrase_sub(txt)
split = "val" if r["chapter"] == a.holdout_chapter else "train"
fh.write(json.dumps({"work": slug, "copy": c, "chapter": r["chapter"],
"split": split, "text": txt}, ensure_ascii=False) + "\n")