BabyYarros: the leak gate passes, and it found three defects nobody was looking for
The gate is new. There was no committed instrument for "does any of the author's own proper nouns survive the rename" -- the Brontë number was produced by hand -- so leak_gate.py is now that instrument, and it runs both directions every time: the same scan over the unrenamed source as a positive control, and a nonce string as a negative one. A detector that only ever sees renamed text cannot distinguish absent from blind. Run against BabyYarros as built it reported 212 surviving entities, not the 86 recorded earlier, because it scans the whole corpus rather than each work separately and it counts the sub-threshold entities rename never looked at. Three findings came out of closing that. The corpus had a typography defect of its own. The D1 notes correctly say no unwrap was needed; a different defect was there instead. The Empyrean books set their chapter epigraphs in small caps and the extractor rendered the run as uppercase while leaving the large initial as a separate token, so the corpus carried "M AJOR A FENDRA'S G UIDE TO THE R IDERS Q UADRANT" -- 106 lines, ~700 splits -- plus 52 drop caps like "T he flight field". That is where the entities called IDERS, UADRANT, NAUTHORIZED and seventeen bare single letters came from. A split initial next to an uppercased run is enough to recover the original mixed case, so the restore is exact rather than approximate: a word with a split initial was capitalised, an all-caps word without one was lowercase. Back matter was inside the prose. The builder splits on chapter headings and nothing follows the last one, so every work carried its acknowledgments, newsletter pitches and cover-artist credits -- 4,555 words naming the author's agent, editors and children, in a corpus whose entire purpose is that no identifiable name survives. And the gate passed at 0 of 314 while Afendra was still in every copy. The name never appears unpossessed, so it keyed as an apostrophe form, and rename and the gate both skip those as contractions -- unrenamed and unreported at once, which is the worst failure shape available. Baxter escaped a different way: wilder renders an in-book news article entirely in lowercase, putting the cap/lowercase ratio at 0.13 against a 0.05 bar. Then a second class the unigram scan structurally cannot see. Riders Quadrant, Flame Section, War Games and Fourth Wing -- the book's own title -- are built from ordinary words the detector correctly refuses to call names. The gate now audits recurring capitalised 2-3grams against an explicit allow list, and rename applies a phrase map after the entity pass. Every new detector flag is opt-in and off by default, and the Brontë entity map was re-derived after each change and confirmed identical in keys, surfaces and every field. The stoplist was built by reading each surface in context, which is why it is short: Violence is Xaden's nickname for Violet, and Continent, Presentation, Barrens, Originals, Montserrat, Athena, Aura, Curator and Sage are all in-world. A plausible-looking guess would have excluded most of them. Final: 0 of 325 entities and 0 of 91 audited phrases survive in any of 30 copy files, both controls passing. The sensitivity floor is stated in the gate's own output -- 3 occurrences for a name, 5 for a phrase -- because a negative without one is unfalsifiable.
This commit is contained in:
@@ -98,6 +98,15 @@ def main() -> int:
|
||||
ap.add_argument("--holdout-chapter", type=int, default=10)
|
||||
ap.add_argument("--preset", default="bronte", choices=sorted(PRESETS),
|
||||
help="which corpus's name-pool register to draw from")
|
||||
ap.add_argument("--scope", default="work", choices=("work", "corpus"),
|
||||
help="`work` maps each work independently (reproduces the Bronte run); "
|
||||
"`corpus` uses ONE map across every work in a copy")
|
||||
ap.add_argument("--phrase-map", default=None,
|
||||
help="JSON with `phrases` (multiword) and `tokens` (capitalised single "
|
||||
"words) neutralising in-world compounds the unigram pass cannot reach")
|
||||
ap.add_argument("--min-cap", type=int, default=8,
|
||||
help="minimum capitalised count for an entity to be renamed; below it "
|
||||
"the entity is left in the text verbatim")
|
||||
a = ap.parse_args()
|
||||
|
||||
corpus = Path(a.corpus)
|
||||
@@ -137,6 +146,21 @@ def main() -> int:
|
||||
f"{lab} {len(pool[lab]['male'])}m/{len(pool[lab]['female'])}f/{len(pool[lab]['surname'])}s"
|
||||
for lab in (label_a, label_b)))
|
||||
|
||||
# ⚠ Applied AFTER the entity substitution, so it can never eat a replacement
|
||||
# name. Multiword first and longest first; single tokens are case-SENSITIVE
|
||||
# and whole-word, so a dragon's lowercase `wing` survives while `Fourth Wing`
|
||||
# does not.
|
||||
phrase_sub = None
|
||||
if a.phrase_map:
|
||||
pm = json.loads(Path(a.phrase_map).read_text())
|
||||
table = {**pm.get("phrases", {}), **pm.get("tokens", {})}
|
||||
if table:
|
||||
pat_p = re.compile(r"\b(" + "|".join(re.escape(k) for k in
|
||||
sorted(table, key=len, reverse=True)) + r")\b")
|
||||
phrase_sub = lambda t: pat_p.sub(lambda m: table[m.group(1)], t)
|
||||
print(f" phrase map {a.phrase_map}: {len(pm.get('phrases', {}))} phrases + "
|
||||
f"{len(pm.get('tokens', {}))} capitalised tokens")
|
||||
|
||||
out = Path(a.out); (out / "copies").mkdir(parents=True, exist_ok=True)
|
||||
stats = {"copies": a.copies, "seed": a.seed, "works": {}, "renamed": 0, "held": 0}
|
||||
|
||||
@@ -154,7 +178,7 @@ def main() -> int:
|
||||
titled = set(tg)
|
||||
renameable, held = {}, []
|
||||
for key, e in ents.items():
|
||||
if "’" in key or "'" in key or e["cap"] < 8:
|
||||
if "’" in key or "'" in key or e["cap"] < a.min_cap:
|
||||
continue # possessives/contractions are not entities
|
||||
g = tg.get(key) or e.get("gender")
|
||||
if g:
|
||||
@@ -176,16 +200,47 @@ def main() -> int:
|
||||
stats["renamed"] += len(renameable); stats["held"] += len(held)
|
||||
print(f" {slug:<14} renamed {len(renameable):>3} ({len(renameable)-len(held)} gendered, {len(held)} neutral)")
|
||||
|
||||
# ⚠ CORPUS SCOPE. Per-work maps leak across works and this is measurable, not
|
||||
# theoretical: `Rebel` is detected in `rebel` and renamed there, then printed
|
||||
# verbatim in the two Renegades books where it sits below threshold. A
|
||||
# whole-corpus gate catches it; a per-work one reports clean. It also fixes a
|
||||
# thing the Bronte corpus never had to care about -- Yarros is TWO SERIES, so
|
||||
# Violet has to be the same person in Fourth Wing and Iron Flame, and a
|
||||
# per-work draw gives her two different names inside one copy.
|
||||
if a.scope == "corpus":
|
||||
merged: dict[str, dict] = {}
|
||||
genders: dict[str, set] = collections.defaultdict(set)
|
||||
for slug, plan in plans.items():
|
||||
for key, v in plan.items():
|
||||
merged.setdefault(key, {"surface": v["surface"], "kind": v["kind"], "gender": None})
|
||||
if v["gender"]:
|
||||
genders[key].add(v["gender"])
|
||||
conflicts = 0
|
||||
for key, v in merged.items():
|
||||
g = genders.get(key, set())
|
||||
if len(g) == 1:
|
||||
v["gender"] = next(iter(g)); v["kind"] = "given"
|
||||
else:
|
||||
if len(g) > 1:
|
||||
conflicts += 1
|
||||
v["kind"] = "surname" # held -> neutral pool, still renamed
|
||||
n_gendered = sum(1 for v in merged.values() if v["gender"])
|
||||
print(f" corpus scope: {len(merged)} distinct surfaces "
|
||||
f"({n_gendered} gendered, {len(merged) - n_gendered} neutral), "
|
||||
f"{conflicts} gender conflicts held")
|
||||
plans = {slug: merged for slug in plans}
|
||||
|
||||
# ---- D3: N seeded copies, one consistent map per copy -------------------
|
||||
emitted = 0
|
||||
for c in range(a.copies):
|
||||
rng = random.Random(a.seed + c * 1000)
|
||||
corpus_map, corpus_used = {}, set()
|
||||
for slug, rows in works.items():
|
||||
# Share of pool A for this work. Brontë sets it per novel (Brussels
|
||||
# vs Yorkshire); Yarros uses one default, because the register does
|
||||
# not split by book the way hers does.
|
||||
share_a = cfg["share"].get(slug, cfg["default_share"])
|
||||
used = set()
|
||||
used = corpus_used if a.scope == "corpus" else set()
|
||||
|
||||
def draw(kind: str, gender: str | None) -> str:
|
||||
lang = label_a if rng.random() < share_a else label_b
|
||||
@@ -196,7 +251,12 @@ def main() -> int:
|
||||
used.add(n); return n
|
||||
return rng.choice(pool[lang][bucket])
|
||||
|
||||
mapping = {k: draw(v["kind"], v["gender"]) for k, v in plans[slug].items()}
|
||||
if a.scope == "corpus":
|
||||
for k, v in plans[slug].items():
|
||||
corpus_map.setdefault(k, draw(v["kind"], v["gender"]))
|
||||
mapping = corpus_map
|
||||
else:
|
||||
mapping = {k: draw(v["kind"], v["gender"]) for k, v in plans[slug].items()}
|
||||
pat = re.compile(r"\b(" + "|".join(sorted((re.escape(v["surface"]) for v in plans[slug].values()),
|
||||
key=len, reverse=True)) + r")\b")
|
||||
surf2key = {v["surface"]: k for k, v in plans[slug].items()}
|
||||
@@ -204,6 +264,8 @@ def main() -> int:
|
||||
with path.open("w", encoding="utf-8") as fh:
|
||||
for r in rows:
|
||||
txt = pat.sub(lambda m: mapping[surf2key[m.group(1)]], r["text"])
|
||||
if phrase_sub:
|
||||
txt = phrase_sub(txt)
|
||||
split = "val" if r["chapter"] == a.holdout_chapter else "train"
|
||||
fh.write(json.dumps({"work": slug, "copy": c, "chapter": r["chapter"],
|
||||
"split": split, "text": txt}, ensure_ascii=False) + "\n")
|
||||
|
||||
Reference in New Issue
Block a user