vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

BabyYarros: the leak gate passes, and it found three defects nobody was looking for

Browse Source 
The gate is new. There was no committed instrument for "does any of the author's
own proper nouns survive the rename" -- the Brontë number was produced by hand
-- so leak_gate.py is now that instrument, and it runs both directions every
time: the same scan over the unrenamed source as a positive control, and a nonce
string as a negative one. A detector that only ever sees renamed text cannot
distinguish absent from blind.

Run against BabyYarros as built it reported 212 surviving entities, not the 86
recorded earlier, because it scans the whole corpus rather than each work
separately and it counts the sub-threshold entities rename never looked at.
Three findings came out of closing that.

The corpus had a typography defect of its own. The D1 notes correctly say no
unwrap was needed; a different defect was there instead. The Empyrean books set
their chapter epigraphs in small caps and the extractor rendered the run as
uppercase while leaving the large initial as a separate token, so the corpus
carried "M AJOR A FENDRA'S G UIDE TO THE R IDERS Q UADRANT" -- 106 lines, ~700
splits -- plus 52 drop caps like "T he flight field". That is where the entities
called IDERS, UADRANT, NAUTHORIZED and seventeen bare single letters came from.
A split initial next to an uppercased run is enough to recover the original
mixed case, so the restore is exact rather than approximate: a word with a split
initial was capitalised, an all-caps word without one was lowercase.

Back matter was inside the prose. The builder splits on chapter headings and
nothing follows the last one, so every work carried its acknowledgments,
newsletter pitches and cover-artist credits -- 4,555 words naming the author's
agent, editors and children, in a corpus whose entire purpose is that no
identifiable name survives.

And the gate passed at 0 of 314 while Afendra was still in every copy. The name
never appears unpossessed, so it keyed as an apostrophe form, and rename and the
gate both skip those as contractions -- unrenamed and unreported at once, which
is the worst failure shape available. Baxter escaped a different way: wilder
renders an in-book news article entirely in lowercase, putting the cap/lowercase
ratio at 0.13 against a 0.05 bar.

Then a second class the unigram scan structurally cannot see. Riders Quadrant,
Flame Section, War Games and Fourth Wing -- the book's own title -- are built
from ordinary words the detector correctly refuses to call names. The gate now
audits recurring capitalised 2-3grams against an explicit allow list, and
rename applies a phrase map after the entity pass.

Every new detector flag is opt-in and off by default, and the Brontë entity map
was re-derived after each change and confirmed identical in keys, surfaces and
every field. The stoplist was built by reading each surface in context, which is
why it is short: Violence is Xaden's nickname for Violet, and Continent,
Presentation, Barrens, Originals, Montserrat, Athena, Aura, Curator and Sage are
all in-world. A plausible-looking guess would have excluded most of them.

Final: 0 of 325 entities and 0 of 91 audited phrases survive in any of 30 copy
files, both controls passing. The sensitivity floor is stated in the gate's own
output -- 3 occurrences for a name, 5 for a phrase -- because a negative without
one is unfalsifiable.
This commit is contained in:
vh
2026-09-11 10:06:04 -07:00
parent 16c144fcda
commit 7b0580dcbe
7 changed files with 903 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/yarros-corpus/RUNBOOK.md
+72
View File
@@ -0,0 +1,72 @@
# BabyYarros — corpus pipeline, end to end
Every stage is reproducible from the Kvasir masters. Run them in order; each one
refuses to write if its own controls fail, so a silent bad stage is not a
failure mode here.
`$R` is this repo's `scripts/` directory. Work happens on **nh3-dev** (it has the
Kvasir library); training happens on **pfi-gx10**.
```bash
# D1 — build from the licensed masters
python3 $R/yarros-corpus/build_corpus_yarros.py --out ~/yarros-corpus
# D1b — repair EPUB typography, strip back matter
# small-caps epigraphs, drop caps, acknowledgments/newsletter tails
python3 $R/yarros-corpus/repair_typography.py ~/yarros-corpus --out ~/yarros-corpus-r
# D2 — entity map. Every flag below is OPT-IN and off by default so the Brontë
# run stays byte-reproducible; all five are needed for this corpus.
python3 $R/r49-corpus/entities.py ~/yarros-corpus-r \
--out ~/yarros-corpus-r/entities.json \
--min-count 3 --min-mid-ratio 0.35 --drop-acronyms --fold-clitics \
--rescue-honorific 2 --stoplist $R/yarros-corpus/stoplist_yarros.json \
--control "Violet,Xaden,Basgiath,Tairn,Leah,Landon,Brennan,Mira,Rhiannon,Dain,Jesinia,Bodhi,Garrick,Imogen,Sorrengail,Riorson,Delgado,Schur,Rhee,Messina,Masen,Violence,Montserrat,Barrens,Originals,Lilith,Nyra,Naolin,Afendra,Baxter" \
--negative-control "Hey,Holy,Hopefully,Yep,Hi,Whoa,Nope,Ugh,Ouch,Okay,Yeah,Thankfully,Honestly,Seriously,Hmm,Jesus,Logically,Chapter,Article,TV,VIP,ATV,BMX,Google,Nepal,American,Pacific,Harvard,Sergeant,Majesty,YouTube,Colorado"
# D2b — gender, resolved from the chapters a name does NOT narrate
python3 $R/yarros-corpus/pov_gender.py ~/yarros-corpus-r \
--entities ~/yarros-corpus-r/entities.json --out ~/yarros-corpus-r/entities-pov.json \
--control "Violet=f,Xaden=m,Leah=f,Landon=m,Rhiannon=f,Dain=m,Mira=f,Brennan=m,Imogen=f,Paxton=m,Rachel=f,Penna=f,Nick=m,Liam=m,Sloane=f,Bodhi=m,Garrick=m,Jesinia=f"
# D3 — rename, ONE map per copy across the whole corpus
python3 $R/r49-corpus/rename.py ~/yarros-corpus-r \
--entities ~/yarros-corpus-r/entities-pov.json \
--dictionary ~/r49-prep/name_dictionary.json \
--out ~/yarros-corpus-renamed-v2 --preset yarros --scope corpus --min-cap 3 \
--copies 6 --seed 4919 --phrase-map $R/yarros-corpus/phrase_map_yarros.json
# GATE — must pass before anything is trained
python3 $R/r49-corpus/leak_gate.py ~/yarros-corpus-r \
--entities ~/yarros-corpus-r/entities-pov.json --renamed ~/yarros-corpus-renamed-v2 \
--min-cap 3 --phrase-map $R/yarros-corpus/phrase_map_yarros.json \
--report ~/yarros-corpus-renamed-v2/leak_gate_report.json
```
## Why each opt-in flag exists
Each one was added because the gate caught something, and each is measured, not
assumed. All five default to OFF, and the Brontë entity map was re-derived after
every change and confirmed identical in keys, surfaces and every field.
| flag | the defect it fixes | evidence |
|---|---|---|
| `--min-mid-ratio 0.35` | `Hey`, `Holy`, `Hopefully`, `Yep`, `Whoa`, `Nope`, `Ugh` were entities | 33 verified names sit at 0.567–0.985 mid-sentence, 19 verified interjections at 0.000–0.222 |
| `--drop-acronyms` | `TV`, `VIP`, `ATV`, `BMX`, `RSC` renamed to surnames | tested on the DOMINANT surface, so `Braevick` inside an all-caps dispatch is not lost |
| `--fold-clitics` | `Afendra` never appears unpossessed, so it keyed as `Afendra’s` — which rename AND the gate both skip | it survived every copy while the gate read 0 of 314 |
| `--rescue-honorific 2` | `Baxter` rejected at ratio 0.13 because an in-book news article is set all-lowercase | `ms. baxter` ×14; the wide honorific list matched case-insensitively readmitted 143 junk tokens, so the rescue list is 5 abbreviations and the lowercase arm requires the period |
| `--stoplist` | real-world referents renamed (`Google`, `Nepal`, `American`) | every surface read in context first — `Violence` is Xaden's nickname for Violet and would have been wrongly excluded by a guess |
`--scope corpus` and `--phrase-map` are rename-side, same shape:
- **`--scope corpus`** — per-work maps leak across works (`Rebel` renamed in `rebel`,
printed verbatim in the other two Renegades books) and give one character two
names inside a single copy. Yarros is two *series*; Brontë was four unrelated novels.
- **`--phrase-map`** — the unigram pass cannot reach `Riders Quadrant`, `Flame Section`
or `Fourth Wing`, the book's own title, because every component is an ordinary word.
## Sensitivity floor
⚠ The gate resolves leak down to **3 capitalised occurrences per work** for names
and **5 recurrences** for phrases. Below those it does not detect, does not
rename, and does not report. "0 survive" means zero above that floor.
scripts/yarros-corpus/phrase_map_yarros.json
+55
View File
@@ -0,0 +1,55 @@
{
"corpus": "BabyYarros",
"why": "The unigram rename cannot reach an in-world compound built out of ordinary words. `Riders Quadrant`, `Flame Section`, `War Games` and `Fourth Wing` -- the book's own title -- all survived a gate that read 0 of 324, because every component is a common noun the cap/lowercase ratio correctly refuses to call a name. This is the `Thornfield x 100` case one level up, and it needs a map rather than a detector: substituting a head noun is a choice about register, not a measurement.",
"how_applied": "rename.py runs this AFTER the entity substitution. Multiword keys first, longest first; then capitalised single tokens, whole-word and case-sensitive, so the lowercase noun (a dragon's `wing`, a `squad` of cadets) is untouched.",
"phrases": {
"Silver One": "Argent One",
"First Six": "First Founders",
"Great War": "Long War",
"Unedited History": "Unabridged Record",
"Recovered Correspondence": "Retrieved Letters",
"The Fables": "The Legends",
"The Journal": "The Ledger",
"Field Guide": "Field Primer",
"Dreamless Sleep": "Endless Sleep",
"Conscription Day": "Levy Day"
},
"tokens": {
"Quadrant": "Division",
"Quadrants": "Divisions",
"Wing": "Flight",
"Wings": "Flights",
"Section": "Cohort",
"Sections": "Cohorts",
"Squad": "Unit",
"Squads": "Units",
"Games": "Trials",
"Daggertail": "Spinecrest",
"Daggertails": "Spinecrests",
"Swordtail": "Bladecrest",
"Swordtails": "Bladecrests"
},
"allow": [
"Thank God",
"Abu Dhabi",
"Sri Lanka",
"Los Angeles",
"Machu Picchu",
"Taj Mahal",
"Buenos Aires",
"World Religion",
"High Roller",
"Hong Kong",
"Oak Moss",
"Moss Grove",
"Oak Moss Grove",
"Fox Motocross",
"Nitro Circus",
"Red Bull",
"Unauthorized Edition",
"Battle Brief",
"Pacific Ocean",
"Las Vegas"
],
"allow_why": "Real-world referents and generic English that any novelist could write. `Battle Brief` and `Unauthorized Edition` stay because their distinctive halves -- the class and the in-world author -- are already renamed by the unigram pass, leaving ordinary words behind."
}
scripts/yarros-corpus/repair_typography.py
+192
View File
@@ -0,0 +1,192 @@
"""BabyYarros D1b — repair two EPUB typography defects the D1 build carried through.
The Brontë corpus needed an unwrap because it was hard-wrapped; this one does not,
and the D1 notes say so correctly. It has a DIFFERENT defect, and it was found by
the leak gate rather than by reading: the detector kept returning entities called
`IDERS`, `UADRANT`, `NAUTHORIZED`, `DITION`, and 17 bare single letters.
1. SMALL-CAPS EPIGRAPHS (fourth-wing + iron-flame, 106 lines, ~700 splits).
The Empyrean books open each chapter with an in-world citation set in small
caps. The extractor rendered the small-caps run as uppercase and left the
large initial as its own token:
— M AJOR A FENDRA’S G UIDE TO THE R IDERS Q UADRANT (U NAUTHORIZED E DITION )
A split initial plus an uppercased run is exactly enough to recover the
original mixed case: a word WITH a split initial was capitalised in the
source (`M`+`AJOR` -> `Major`), and an all-caps word WITHOUT one was
lowercase (`TO THE` -> `to the`). So the line restores to
—Major Afendra’s Guide to the Riders Quadrant (Unauthorized Edition)
⚠ The restoration is applied ONLY to lines carrying at least two splits.
One split is an ordinary sentence next to an acronym; two is a run.
2. DROP CAPS (52 occurrences, 51 of them iron-flame): `T he flight field`,
`X aden.`, `R evolution tastes`. Same cause, one letter instead of a run.
⚠ `I`, `A` and `O` are EXCLUDED from the join because they are real
single-letter words -- `A slow smile spreads` is not a drop cap, and
joining it would invent `Aslow`.
Both defects cost three ways: they manufacture entities the rename then scatters
through the corpus, they spend tokens on fragments, and they teach the adapter a
typography the author never wrote.
The original corpus is left untouched so the D1 build stays reproducible; this
writes a repaired tree beside it, the same way `unwrap_corpus.py` did for Brontë.
"""
from __future__ import annotations
import argparse, json, re, shutil, sys
from pathlib import Path
#: Back matter rides inside the LAST chapter, because the builder splits on
#: chapter headings and nothing follows the final one. Measured: 620-1,279 words
#: per work of acknowledgments, newsletter pitches and cover-artist credits --
#: not the author's prose, and carrying the names of real people (her agent, her
#: editors, her children) straight into a corpus whose whole point is that no
#: identifiable name survives.
BACKMATTER = re.compile(
r"(?im)^[ \t]*(?:ACKNOWLEDGE?MENTS?|About the Author|Also by\b|Discover more\b|"
r"Don[’']t miss more books\b|Join the Entangled\b|Sign up for our newsletter\b|"
r"Keep reading for\b|Turn the page for\b)")
SPLIT = re.compile(r"\b([A-Z]) ([A-Z]{2,})\b")
DROPCAP = re.compile(r"^([B-HJ-NP-Z]) ([a-z]{2,})")
ALLCAPS = re.compile(r"\b([A-Z]{2,})\b")
#: All-caps tokens that are genuinely acronyms rather than small-caps lowercase.
#: Kept uppercase when a small-caps line is restored.
ACRONYMS = {"RSC", "PTSD", "OK", "IV", "II", "III", "IV", "VI", "VII", "VIII", "IX", "XI"}
def restore_smallcaps(line: str) -> str:
"""Two or more split initials means the whole line was a small-caps run."""
if len(SPLIT.findall(line)) < 2:
return line
prev = None
while prev != line: # `A FENDRA’S` can chain with its neighbour
prev = line
line = SPLIT.sub(lambda m: m.group(1) + m.group(2).lower(), line)
# Whatever is still all-caps had no large initial, so it was lowercase.
line = ALLCAPS.sub(lambda m: m.group(1) if m.group(1) in ACRONYMS else m.group(1).lower(), line)
# ⚠ A possessive survives both passes: `A FENDRA’S` splits as `A`+`FENDRA`,
# so the run’s trailing `’S` is a lone capital that neither rule sees.
line = re.sub(r"([’'])S\b", r"\1s", line)
return re.sub(r"\(\s+", "(", re.sub(r"\s+\)", ")", line))
def repair_text(text: str, counts: dict) -> str:
out = []
for line in text.split("\n"):
before = line
line = restore_smallcaps(line)
if line != before:
counts["smallcap_lines"] += 1
counts["smallcap_joins"] += len(SPLIT.findall(before))
before2 = line
line = DROPCAP.sub(lambda m: m.group(1) + m.group(2), line.lstrip()) \
if DROPCAP.match(line.lstrip()) else line
if line != before2:
counts["dropcap_joins"] += 1
out.append(line)
return "\n".join(out)
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("corpus")
ap.add_argument("--out", required=True)
a = ap.parse_args()
src, dst = Path(a.corpus), Path(a.out)
man = json.loads((src / "manifest.json").read_text())
(dst / "works").mkdir(parents=True, exist_ok=True)
counts = {"smallcap_lines": 0, "smallcap_joins": 0, "dropcap_joins": 0, "backmatter_words": 0}
samples, words_before, words_after, unchanged = [], 0, 0, 0
for w in man["works"]:
rows = [json.loads(l) for l in (src / w["path"]).read_text(encoding="utf-8").splitlines() if l.strip()]
per = dict(counts)
# ⚠ Last chapter only. An earlier chapter that happens to contain the word
# `Acknowledgments` in dialogue must not be truncated.
m = BACKMATTER.search(rows[-1]["text"])
if m:
cut = rows[-1]["text"][m.start():]
counts["backmatter_words"] += len(cut.split())
rows[-1]["text"] = rows[-1]["text"][:m.start()].rstrip()
print(f" {w['slug']:14} back matter stripped at {m.group(0).strip()!r}: "
f"{len(cut.split()):,} words")
with (dst / w["path"]).open("w", encoding="utf-8") as fh:
for r in rows:
t0 = r["text"]
t1 = repair_text(t0, counts)
words_before += len(t0.split()); words_after += len(t1.split())
if t0 == t1:
unchanged += 1
elif len(samples) < 6:
for l0, l1 in zip(t0.split("\n"), t1.split("\n")):
if l0 != l1 and len(samples) < 6:
samples.append((w["slug"], l0[:110], l1[:110]))
r["text"] = t1; r["words"] = len(t1.split())
fh.write(json.dumps(r, ensure_ascii=False) + "\n")
d = {k: counts[k] - per[k] for k in counts}
print(f" {w['slug']:14} smallcap lines {d['smallcap_lines']:>4} "
f"(joins {d['smallcap_joins']:>4}) dropcap joins {d['dropcap_joins']:>4}")
print(f"\n chapters unchanged: {unchanged} of {sum(w['chapters'] for w in man['works'])}")
print(f" words {words_before:,} -> {words_after:,} "
f"({words_before - words_after:,} fragments rejoined)")
print("\n sample repairs:")
for slug, a0, a1 in samples:
print(f" {slug}\n - {a0}\n + {a1}")
# ---- acceptance: the defect must be GONE and the join must not have run wild
joined = "\n".join((dst / w["path"]).read_text(encoding="utf-8") for w in man["works"])
fails = []
for must_not in ("R IDERS Q UADRANT", "T he flight field", "U NAUTHORIZED"):
if must_not in joined:
fails.append(f"still present: {must_not!r}")
if "Louise Fury" in joined:
fails.append("back matter survived: the author's agent is still named in the corpus")
if counts["backmatter_words"] > 0.02 * words_before:
fails.append(f"back-matter strip removed {counts['backmatter_words']:,} words, over 2% "
f"of the corpus -- a marker probably matched inside the prose")
for must in ("Riders Quadrant", "The flight field"):
if must not in joined:
fails.append(f"repair did not produce: {must!r}")
# negative control: a line with a single split is NOT a small-caps run
probe = "He got an A GRADE for it."
if restore_smallcaps(probe) != probe:
fails.append("single-split line was rewritten -- the >=2 guard is not holding")
# ⚠ Scoped to RESTORED lines only. The corpus also contains a genuinely
# all-caps in-world dispatch (`...BRAEVICK’S GRYPHON FLEET...`) that carries
# no split initials, so the restore never touches it and it is not a defect.
probe = "— M AJOR A FENDRA’S G UIDE TO THE R IDERS Q UADRANT"
if re.search(r"[’']S\b", restore_smallcaps(probe)):
fails.append("a restored small-caps line still carries an uppercase possessive `’S`")
probe2 = "A slow smile spreads across her face."
if DROPCAP.match(probe2):
fails.append("dropcap join would fire on the article `A`")
for w in man["works"]:
pass
shutil.copy(src / "manifest.json", dst / "manifest.json")
shutil.copy(src / "corpus_alphabet.json", dst / "corpus_alphabet.json")
m2 = json.loads((dst / "manifest.json").read_text())
for w in m2["works"]:
w["words"] = sum(json.loads(l)["words"] for l in (dst / w["path"]).read_text(encoding="utf-8").splitlines() if l.strip())
m2["totals"]["words"] = m2["total_words"] = sum(w["words"] for w in m2["works"])
m2["repaired_from"] = str(src)
m2["repair"] = counts
(dst / "manifest.json").write_text(json.dumps(m2, indent=2, ensure_ascii=False), encoding="utf-8")
print()
for f in fails:
print(f" [FAIL] {f}")
if fails:
print("\n== REPAIR REJECTED"); return 1
print(" [PASS] known-broken strings gone, repaired forms present, guards hold")
print(f"\n wrote {dst}")
return 0
if __name__ == "__main__":
sys.exit(main())
scripts/yarros-corpus/stoplist_yarros.json
+89
View File
@@ -0,0 +1,89 @@
{
"corpus": "BabyYarros",
"why": "The rename exists to keep the author's INVENTIONS out of the adapter. A real-world referent any novelist could use is not hers, and renaming it damages prose for no leak benefit. This is the same call Brontë's STOP_COMMON made for London/Paris/Brussels/Yorkshire, written per corpus because the register is per corpus.",
"how_derived": "Every capitalised surface the detector returned was read IN CONTEXT before landing here. That pass is why the list is short: `Violence` is Xaden's nickname for Violet, `Continent` and `Presentation` and `Battle Brief` and `Curator` and `Sage` and `Barrens` and `Originals` and `Montserrat` and `Athena` and `Aura` are all in-world, and every one of them would have been excluded by a plausible-looking guess. Ambiguous cases are deliberately NOT here: renaming is the safe direction, leaving is the leaking one.",
"real_world_geography": [
"Nepal",
"Vegas",
"Las",
"Cuba",
"Miami",
"Dubai",
"Barcelona",
"Istanbul",
"Pacific",
"Everest",
"Korea",
"Angeles",
"Los",
"Fiji",
"Havana",
"Madagascar",
"Machu",
"Picchu",
"Venezuela",
"Himalayas",
"Taj",
"Mahal",
"Jakarta",
"Seoul",
"Chile",
"California",
"Peru",
"Morocco",
"America",
"Sri",
"Lanka",
"Abu",
"Dhabi",
"Buenos",
"Aires",
"Mykonos",
"Dartmouth",
"Colorado",
"India",
"Kathmandu",
"Hong",
"Kong",
"Lukla",
"Tahoe",
"Lima",
"Trenton"
],
"real_world_institutions_and_brands": [
"Harvard",
"UCLA",
"Google",
"Jeep",
"Marines",
"YouTube",
"GoPro",
"Bellagio",
"Fox"
],
"real_world_nationalities": [
"American",
"Cuban",
"Korean",
"Indian"
],
"real_world_culture": [
"Thanksgiving",
"Sherpas",
"Casanova"
],
"generic_words_capitalised_only_in_in-world_titles": [
"Guide",
"Edition",
"Unauthorized",
"Religion",
"Saturdays",
"Majesty",
"Property",
"Unedited",
"Secondhand",
"Excelling",
"T-shirt"
],
"min_count_note": "Detection runs at --min-count 3 for this corpus, not the default 5. Measured: dropping to 3 recovers Lilith, Nyra, Naolin, Tirvainne, Dajalair, Kiralair, Morraine, Beinhaven, Codagh and Tairneanach -- all author inventions that the 5 floor left in the text verbatim. The cost is more real-world nouns reaching the pool, which is what this file absorbs. ⚠ The gate's SENSITIVITY FLOOR is this number: a name appearing fewer than 3 times is never detected, so it is neither renamed nor reported."
}