vh
8bb7686a16
Stoplisting a surface removes it from the entity map, so rename never touches it
and the gate never scans for it. That is exactly what a stoplist is FOR when the
surface is a real-world referent — and exactly how a wrongly stoplisted CHARACTER
becomes an undetectable leak. The gate reports 0 of N surviving and is telling the
truth about the set it was given.
Found by luck on lv-bronte: a generated beat said "Mrs. Leaven", and Leaven had
been filed under scripture as the bread noun. Reading it back: "Robert Leaven,
the coachman" — Bessie's married surname in Jane Eyre.
Running the audit instead of trusting that luck caught two more:
Pierrot "Madame Pierrot: she comes from Lisle, in France" — a teacher in
The Professor, filed as the commedia dell'arte figure
Samuel "Mr. Samuel Wynne" — filed as scripture
and correctly CLEARED two:
Wellington "that Baal of a Lord Wellington" — the real Duke
Moses "the Rev. Moses Barraclough" — the documented dual-use
Signal is an honorific in front of the surface: real-world referents are not
addressed as Mr/Mrs/Miss/Madame/Lord. It is a heuristic and not a proof, which is
why every hit is REPORTED FOR READING and never auto-removed — Wellington and
Moses both trip it and both are correct. Exit 1 on anything not on --allow, so it
can gate a pipeline.
Blast radius of the three errors was 16 of 3781 train pairs and 3 of 80 val —
small, but they are the author's characters in training data, which is the one
thing this pipeline exists to prevent. Corpus rebuilt rather than dropping the
affected pairs: a corpus on disk that disagrees with its committed config is how
superseded claims get made. Gate re-passes at 0 of 368 (three more surfaces than
before, exactly the restored characters), both controls green.
90 lines
4.0 KiB
Python
90 lines
4.0 KiB
Python
"""Audit a stoplist for entries that are actually CHARACTERS.
|
|
|
|
⭐ WHY THIS IS A SEPARATE INSTRUMENT. A stoplist entry is an assertion the leak gate
|
|
can no longer check. Stoplisting a surface removes it from the entity map, so rename
|
|
never touches it and the gate never scans for it — which is exactly what a stoplist is
|
|
FOR when the surface is a real-world referent, and exactly how a wrongly stoplisted
|
|
CHARACTER becomes an undetectable leak. The gate will report 0 of N surviving and be
|
|
telling the truth about the set it was given.
|
|
|
|
Found on lv-bronte 2026-09-16, and found by luck: a generated beat said "Mrs. Leaven",
|
|
and `Leaven` had been filed under scripture (the bread noun). Reading it back:
|
|
"Robert Leaven, the coachman" — Bessie's married surname. Running this audit instead of
|
|
trusting the luck then caught two more, `Pierrot` ("Madame Pierrot: she comes from
|
|
Lisle") and `Samuel` ("Mr. Samuel Wynne"), and correctly cleared `Wellington`
|
|
("that Baal of a Lord Wellington" — the real Duke).
|
|
|
|
THE SIGNAL: an honorific in front of it. Real-world referents are not addressed as
|
|
Mr/Mrs/Miss/Madame/Lord. It is a heuristic, not a proof — `Lord Wellington` is a real
|
|
person and `Rev. Moses Barraclough` is a genuine dual-use — so every hit is REPORTED
|
|
FOR READING, never auto-removed. Deciding what a surface is requires reading it in
|
|
context, which is the lesson this whole line keeps relearning.
|
|
|
|
Exit 1 when anything is flagged, so it can gate a pipeline: a clean run is silence.
|
|
"""
|
|
from __future__ import annotations
|
|
import argparse, json, re
|
|
from pathlib import Path
|
|
|
|
HONORIFIC = (r"(?:Mr|Mrs|Miss|Misses|Madame|Mdlle|Mademoiselle|Monsieur|Lord|Lady|Sir|Dr"
|
|
r"|Doctor|Captain|Colonel|Major|General|Aunt|Uncle|Rev|Reverend|Professor"
|
|
r"|Master|Saint|St)\.?\s+")
|
|
|
|
|
|
def load_text(corpus: Path) -> str:
|
|
man = json.loads((corpus / "manifest.json").read_text())
|
|
parts = []
|
|
for w in man["works"]:
|
|
for line in (corpus / w["path"]).read_text(encoding="utf-8").splitlines():
|
|
if line.strip():
|
|
parts.append(json.loads(line)["text"])
|
|
return "\n\n".join(parts)
|
|
|
|
|
|
def main() -> int:
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("corpus")
|
|
ap.add_argument("--stoplist", required=True)
|
|
ap.add_argument("--allow", default="",
|
|
help="comma-separated surfaces already READ and confirmed real-world "
|
|
"or accepted dual-use; they are still shown, but do not fail the run")
|
|
ap.add_argument("--context", type=int, default=46)
|
|
a = ap.parse_args()
|
|
|
|
text = load_text(Path(a.corpus))
|
|
blob = json.loads(Path(a.stoplist).read_text())
|
|
surfaces = sorted({s for v in blob.values() if isinstance(v, list) for s in v})
|
|
cleared = {s.strip() for s in a.allow.split(",") if s.strip()}
|
|
|
|
flagged = []
|
|
for s in surfaces:
|
|
pat = re.compile(HONORIFIC + re.escape(s) + r"\b")
|
|
hits = list(pat.finditer(text))
|
|
if hits:
|
|
flagged.append((len(hits), s, hits[0]))
|
|
|
|
print(f" {len(surfaces)} stoplisted surfaces scanned for a preceding honorific")
|
|
if not flagged:
|
|
print(" [PASS] none of them is ever addressed as a person")
|
|
return 0
|
|
|
|
unresolved = 0
|
|
for n, s, m in sorted(flagged, reverse=True):
|
|
mark = "cleared" if s in cleared else "⚠ READ THIS"
|
|
if s not in cleared:
|
|
unresolved += 1
|
|
ctx = text[max(0, m.start() - a.context): m.end() + a.context].replace("\n", " ")
|
|
print(f" {mark:<12} {s:<16} {n:>4} honorific hits …{ctx}…")
|
|
|
|
if unresolved:
|
|
print(f"\n== {unresolved} stoplisted surface(s) look like PEOPLE and are not on --allow.")
|
|
print(" Read each in context. If it is a character, take it OUT of the stoplist and")
|
|
print(" re-run the rename — the leak gate cannot see it while it is stoplisted.")
|
|
return 1
|
|
print("\n [PASS] every honorific hit is on the read-and-cleared list")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|