fix(lv-mccarthy): the leak gate passed with five protagonist names still in every copy
`leak_gate.py` scans `\b(Surface)\b`. Any character inserted inside a name defeats
that pattern outright, so a mangled occurrence is unrenameable by rename.py AND
unreportable by the gate. lv-mccarthy's 2026-09-17 tree passed at "0 of 75
renameable and 0 of 37 sub-threshold" while carrying 13 occurrences of Bell,
Chigurh, Moss, Toadvine and Glanton in all six copies:
B ell C higurh M oss T oadvine a small-caps drop cap kept as its own token
Toad-vine Glan-ton a print line-break hyphen kept by the extractor
Every visible occurrence HAD been renamed, which is what made the residue invisible
to a spot-read. Fixed at three levels, all three of which must stay:
build_corpus_mccarthy.py rules 4 and 5 repair the source text — 32 split initials
with a lowercase remainder, 5 hyphen-split names, each with an expected count so a
master change fails the build. Rule 4's letter class is consonants only: `I` opens
1,966 paragraphs, `A` 143 and `Y` 32 (Spanish `y`); folding any would corrupt 2,141
lines to fix 32.
leak_gate.py gains a separator-tolerant pass with its own positive and negative
controls, and it FAILS the gate. Validated against the pre-fix tree: reports all
five surfaces, exits 1. Its fragment filter is what makes it usable — a naive scan
returns 18 false positives on Hemingway (`God damn`, `I run`) against 3 real ones;
requiring one fragment to be a non-word of the corpus cleared all 18 and kept all 3.
The whole D1→D3 chain is reproduced byte-identically before and after, so the fix
is the only delta: 6 works, the entity map, the final map and all 36 copy files.
Cross-checked on the shipped corpora: lv-bronte is clean of this class, lv-hemingway
carries 3 (`Primi tivo`, `Pasionar ia`, `Chi cote`) and is live on fv-ml1.
Also in build_sft_pairs.py, both needed before lv-mccarthy's pairs:
DEFECT 4, hard-wrap reflow. Measured on the SHIPPED lv-bronte adapter, which emits
mid-sentence line breaks at 12.46 per 1k chars against 0.00 for its own base control
and 0.00 for every Hemingway arm. McCarthy is the mixed case — The Road is wrapped,
the other five works are not — so the corpus teaches the break as a coin flip. The
obvious fix (join every interior newline) corrupts 46 two-speaker exchanges whose
blank line was lost, and unmarked dialogue is the one thing this adapter exists to
learn; the rule splits on sentence-final punctuation instead and takes the cheaper
error. Self-targeting and off by default, so every shipped pair set is unchanged.
A `mccarthy` register, which names the punctuation deliberately: the eval drives the
base control arm with this same prompt, so tics left out of it are a surface trick
only the adapter can perform, and delta_cb is a character-bigram measure.
drop_leading_heading now also consumes Blood Meridian's dash-separated chapter
arguments — 131 paragraphs, 0 in every other work of all three corpora.
And a RUNBOOK, because the D1→D3 session recorded nothing and the chain had to be
recovered by rebuilding candidates and matching sha256 against the artifacts on disk.
This commit is contained in:
@@ -0,0 +1,108 @@
|
||||
# lv-mccarthy — corpus → gate → pairs
|
||||
|
||||
Six works, 167 units, 584,684 words. Built on **pfi-gx10** under `~/lv-mccarthy/`
|
||||
as `infra-ops`, except **D1, which must run on nh3-dev**: the builder reads the
|
||||
kvasir catalogue at `/home/lkraven/development/kvasir/data/library/catalog.sqlite`
|
||||
and that path exists only there.
|
||||
|
||||
⚠ **This file exists because the 2026-09-17 D1→D3 session recorded nothing.** There
|
||||
was no runbook, the commands were issued over non-interactive ssh so no shell history
|
||||
survived, and a later session had to recover the whole chain by rebuilding candidates
|
||||
and matching sha256 against the artifacts on disk. Every deviation below is now pinned
|
||||
by a reproducibility control; keep it that way.
|
||||
|
||||
## The chain
|
||||
|
||||
```bash
|
||||
R=~/development/eshpfi-management/scripts # nh3-dev for D1, ~/lv-mccarthy/scripts on gx10
|
||||
|
||||
# D1 — build. ON nh3-dev (needs the kvasir catalogue), then rsync corpus-clean/ to gx10.
|
||||
python3 $R/mccarthy-corpus/build_corpus_mccarthy.py --out corpus-clean
|
||||
|
||||
# D2 — entity map. --min-mid-ratio is what keeps `Yeah`/`Buenas`/`Shh` out; see below.
|
||||
python3 $R/r49-corpus/entities.py corpus-clean --out corpus-clean/entities.json \
|
||||
--stoplist $R/mccarthy-corpus/stoplist_mccarthy.json \
|
||||
--min-count 5 --fold-clitics --drop-acronyms --min-mid-ratio 0.2 --min-mid 2
|
||||
|
||||
# D2c — the four hand-verified genders.
|
||||
python3 $R/r49-corpus/apply_gender_overrides.py --entities corpus-clean/entities.json \
|
||||
--overrides $R/mccarthy-corpus/gender_overrides_mccarthy.json \
|
||||
--out corpus-clean/entities-final.json
|
||||
|
||||
# D3 — rename. --scope corpus, --preset mccarthy and --min-cap 5 are all deviations.
|
||||
python3 $R/r49-corpus/rename.py corpus-clean --entities corpus-clean/entities-final.json \
|
||||
--dictionary ~/r49-prep/name_dictionary.json --out corpus-renamed \
|
||||
--preset mccarthy --scope corpus --min-cap 5 --copies 6 --seed 4919 \
|
||||
--holdout-chapter 7 17
|
||||
|
||||
# GATE — must pass before anything is trained. --min-cap 8 here is deliberate; see below.
|
||||
python3 $R/r49-corpus/leak_gate.py corpus-clean --entities corpus-clean/entities-final.json \
|
||||
--renamed corpus-renamed --min-cap 8 --report corpus-renamed/leak_gate_report.json
|
||||
|
||||
# D4 — instruction pairs. ON gx10; ~24 min against the `gen` seat.
|
||||
./run-pairs.sh
|
||||
```
|
||||
|
||||
Gate result, 2026-09-17 (after the separator fix): **0 of 75 renameable, 0 of 37
|
||||
sub-threshold, 0 separator-split surfaces**, all four controls green. Sensitivity
|
||||
floor: a name under 8 capitals per work is never detected, a phrase under 5
|
||||
recurrences never audited, and **no phrase map exists for this corpus, so the phrase
|
||||
audit does not run at all** — the Yarros and Brontë runs both had one.
|
||||
|
||||
## The deviations, and what forced each
|
||||
|
||||
| deviation | why |
|
||||
|---|---|
|
||||
| **D1 runs on nh3-dev** | The builder reads the kvasir catalogue by absolute path. gx10 has no copy. |
|
||||
| **`--min-count 5`** (was 8) | The entity map's floor has to match rename's, or entities between the two floors sit in the map, are never renamed, and count as leaks. The first D3 gate run failed with 45 survivors for exactly this. |
|
||||
| **`--min-mid-ratio 0.2 --min-mid 2`** | Without it the map admits `Yeah`, `Buenas`, `Shh`, `Mande`, `Sí`, `Git` — words that are only ever capitalised at a sentence start. ⚠ The map is INSENSITIVE to the exact value: any ratio in **[0.05, 0.3]** with `--min-mid` 1 or 2 reproduces it byte-for-byte. The original run's values are unrecoverable and it does not matter. `--min-mid 3` does NOT reproduce it. |
|
||||
| **`--fold-clitics --drop-acronyms`** | Both are needed to reproduce the map; `--rescue-honorific` is inert here (0 rescued). |
|
||||
| **`--scope corpus`** (was `work`) | Nine surfaces appear in more than one work — Parham, Grady, Cole, Socorro, Héctor. A per-work map gives John Grady a different invented name in each Border Trilogy novel. |
|
||||
| **`--preset mccarthy`** | Hemingway's romance pool carries `it_IT`/`fr_FR` for his Italian and French casts; McCarthy writes neither. `en_US` + `es_MX`/`es_ES` at an even share. |
|
||||
| **`--holdout-chapter 7 17`** (a LIST) | The val split is one chapter per work, so its size scales with WORK COUNT, not corpus size. Six works would have given a Brontë-class ~18,000-word val reference; two indices give 11 units and 40,653 words per copy, larger than Hemingway's, for 7% of the corpus. |
|
||||
| **gate at `--min-cap 8`, rename at `--min-cap 5`** | Not a mistake. The gate reports the 37 entities rename never touched *separately*, which is strictly more informative than running both at 5 (where the sub-threshold bucket is empty). |
|
||||
| **D4 `--reflow-hard-wraps`** | The Road is hard-wrapped at ~76 characters and the other five works are not. See DEFECT 4 in `build_sft_pairs.py`. |
|
||||
| **D4 `--source-entities`** | Mandatory. The beat generator reads the passage and will supply canonical names from its own memory of the book; the corpus gate never reads the generated beats. |
|
||||
|
||||
## The defect the gate could not see, and now can
|
||||
|
||||
⚠⚠ **The 2026-09-17 tree passed this gate at "0 of 75 renameable and 0 of 37
|
||||
sub-threshold" while carrying 13 occurrences of five protagonist names in all six
|
||||
copies.** `leak_gate.py` scanned `\b(Surface)\b`, and a character inserted inside a
|
||||
name defeats that pattern outright — so the occurrence was unrenameable by `rename.py`
|
||||
*and* unreportable by the gate. Two mechanisms, both in the source extraction:
|
||||
|
||||
```
|
||||
B ell C higurh M oss T oadvine a small-caps drop cap survived as its own token
|
||||
Toad-vine Glan-ton a print line-break hyphen survived the extraction
|
||||
```
|
||||
|
||||
Every *visible* occurrence had been renamed correctly, which is what made the residue
|
||||
invisible to any spot-read: the names are gone everywhere you look.
|
||||
|
||||
Fixed in three places, and all three must stay:
|
||||
|
||||
1. **`build_corpus_mccarthy.py` rules 4 and 5** repair the text at its source — 32
|
||||
split initials with a lowercase remainder, 5 hyphen-split names, both with expected
|
||||
counts so a master change fails the build. ⚠ Rule 4's letter class is **consonants
|
||||
only**: `I` opens 1,966 paragraphs, `A` opens 143 and `Y` opens 32 (Spanish *y*), and
|
||||
folding any of them would corrupt 2,141 lines to fix 32.
|
||||
2. **`leak_gate.py --split-frag-max`** runs a separator-tolerant scan every time, with
|
||||
its own positive and negative controls, and **fails the gate**. Verified against the
|
||||
pre-fix tree: it reports all five surfaces and exits 1.
|
||||
3. The exact-match passes are unchanged, so the old verdict is reproduced alongside.
|
||||
|
||||
Cross-checked on the two shipped corpora: **lv-bronte is clean** of this class;
|
||||
**lv-hemingway carries 3** (`Primi tivo`, `Pasionar ia`, `Chi cote`) and is live.
|
||||
|
||||
## Corpus properties worth knowing before you change anything
|
||||
|
||||
- **`repair_typography.py` MUST NOT be run on this corpus.** It normalises "toward what
|
||||
the text does" and would put quotation marks back into a corpus that measures 0.0 per
|
||||
10k against Hemingway's 838. The builder runs no normalisation and asserts the density.
|
||||
- **Back matter is stripped BEFORE the split**, inverting the Hemingway order: Blood
|
||||
Meridian and The Crossing end with a dumped table of contents made of bare roman
|
||||
numerals, which is the exact shape of a chapter marker.
|
||||
- **Blood Meridian sets a dash-separated chapter argument under each roman numeral.**
|
||||
131 paragraphs; `--drop-leading-heading` now consumes them.
|
||||
- Two truncated catalogue rows are excluded in favour of complete siblings.
|
||||
@@ -135,6 +135,60 @@ LOST_INITIALS = {
|
||||
"all-the-pretty-horses": [(re.compile(r"(?m)^HE CANDLEFLAME"), "THE CANDLEFLAME", 1)],
|
||||
}
|
||||
|
||||
# ⚠⚠⚠ RULES 4 AND 5 ARE A LEAK FIX, NOT A TYPOGRAPHY FIX, AND THE GATE PASSED WITHOUT THEM.
|
||||
#
|
||||
# `leak_gate.py` scans `\b(Surface)\b`. ANY character inserted inside a name defeats that
|
||||
# pattern outright, so a mangled occurrence is not merely unrepaired -- it is UNRENAMEABLE and
|
||||
# UNREPORTABLE. Measured on the gated 2026-09-17 tree, which had reported "0 of 75 renameable
|
||||
# and 0 of 37 sub-threshold survive in any copy":
|
||||
#
|
||||
# B ell x2 Chigurh cap 119 in the map no-country-for-old-men
|
||||
# C higurh x3 Bell cap 147 no-country-for-old-men
|
||||
# M oss x2 Moss cap 126 no-country-for-old-men
|
||||
# T oadvine x1 Toadvine cap 111 blood-meridian
|
||||
# Toad-vine x3 Glanton cap 365 blood-meridian
|
||||
# Glan-ton x2
|
||||
#
|
||||
# 13 occurrences of FIVE protagonist names, present in ALL SIX renamed copies, with 0 visible
|
||||
# survivors -- so the gate's verdict was true for the forms it can see and false overall. Every
|
||||
# unmangled occurrence WAS renamed (exact-match survivors: 0), which is what makes the residue
|
||||
# so easy to miss: the names are gone everywhere you look.
|
||||
#
|
||||
# Positive control on the probe that found it: the same separator-tolerant scan over the
|
||||
# UNRENAMED source returns 115-365 hits per name, so it detects these names when present.
|
||||
# Cross-check on the two SHIPPED corpora: lv-bronte is clean of this class; lv-hemingway carries
|
||||
# 3 (`Primi tivo`, `Pasionar ia`, `Chi cote`). `leak_gate.py` now runs the scan itself.
|
||||
#
|
||||
# 4. SPLIT INITIAL, LOWERCASE REMAINDER `S ee the child` -> `See the child` 32 cases
|
||||
# The drop cap survived as its own token and the small-caps remainder came through
|
||||
# LOWERCASE, so rule 1 cannot see it -- rule 1 requires a following ALL-CAPS word, and
|
||||
# DROPCAP requires two. This is the class both of them leave behind. Confined to
|
||||
# blood-meridian (16) and no-country-for-old-men (16); the other four works have none.
|
||||
#
|
||||
# ⚠ THE VOWELS AND `Y` ARE EXCLUDED BECAUSE THEY ARE REAL WORDS AT A SENTENCE START, and
|
||||
# this is the `I`/`A` trap that already bit audit_stoplist.py once. Measured in the built
|
||||
# corpus: `I` opens 1,966 paragraphs (the pronoun), `A` opens 143 (the article, e.g.
|
||||
# `A blivet is ten pounds of shit in a five pound sack.`), and `Y` opens 32 (Spanish `y`,
|
||||
# e.g. `Y de los hombres?`). Folding any of those would corrupt 2,141 lines to fix 32.
|
||||
# The letter class is consonants only, and all 32 survivors were read: the second
|
||||
# fragments are `ell heir he hen hey higurh ith ive oadvine or oss ow e ee hen`.
|
||||
#
|
||||
# ⚠ Runs LAST, after DROPCAP. DROPCAP rewrites `T HE HOUSE was built` to `The house was
|
||||
# built` -- the space between its groups is consumed, not captured -- so its output is
|
||||
# never a rule-4 target. Verified: line-anchored and paragraph-anchored counts are both 32,
|
||||
# i.e. no hit sits mid-paragraph on one of The Road's hard-wrapped lines.
|
||||
#
|
||||
# 5. HYPHEN-SPLIT NAME `Toad-vine` -> `Toadvine` 5 cases
|
||||
# A print edition hyphenated the name across a line break and the extraction kept the
|
||||
# hyphen while dropping the break. Patched BY NAME with an expected count, never by
|
||||
# heuristic: McCarthy writes real hyphenated compounds and a rule broad enough to catch
|
||||
# these would eat them. A master change makes the count wrong and says so.
|
||||
SPLIT_INITIAL_LOWER = re.compile(r"(?m)^([B-DF-HJ-NP-TV-XZ]) ([a-z][a-z']*)")
|
||||
HYPHEN_SPLIT_NAMES = {
|
||||
"blood-meridian": [(re.compile(r"\bToad-vine\b"), "Toadvine", 3),
|
||||
(re.compile(r"\bGlan-ton\b"), "Glanton", 2)],
|
||||
}
|
||||
|
||||
|
||||
def restore_smallcaps(line: str) -> str:
|
||||
if len(SPLIT_INITIAL.findall(line)) < 2:
|
||||
@@ -143,8 +197,9 @@ def restore_smallcaps(line: str) -> str:
|
||||
|
||||
|
||||
def repair_smallcaps(text: str, slug: str) -> tuple[str, dict]:
|
||||
"""Run the three repairs in order; the lost initials MUST go first."""
|
||||
stats = {"lost_initial": 0, "split_initial": 0, "unmarked_run": 0}
|
||||
"""Run the five repairs in order; the lost initials MUST go first, rule 4 MUST go last."""
|
||||
stats = {"lost_initial": 0, "split_initial": 0, "unmarked_run": 0,
|
||||
"split_initial_lower": 0, "hyphen_split_name": 0}
|
||||
for pat, good, expect in LOST_INITIALS.get(slug, []):
|
||||
text, n = pat.subn(good, text)
|
||||
if n != expect:
|
||||
@@ -157,6 +212,15 @@ def repair_smallcaps(text: str, slug: str) -> tuple[str, dict]:
|
||||
lambda m: m.group(1) + m.group(2).lower(), text)
|
||||
text, stats["unmarked_run"] = SMALLCAPS_OPENING.subn(
|
||||
lambda m: m.group(1)[0] + m.group(1)[1:].lower(), text)
|
||||
for pat, good, expect in HYPHEN_SPLIT_NAMES.get(slug, []):
|
||||
text, n = pat.subn(good, text)
|
||||
if n != expect:
|
||||
print(f" ⚠ {slug}: expected {expect} occurrence(s) of {pat.pattern!r} and "
|
||||
f"replaced {n} — the master changed; re-check before trusting this build",
|
||||
file=sys.stderr)
|
||||
stats["hyphen_split_name"] += n
|
||||
text, stats["split_initial_lower"] = SPLIT_INITIAL_LOWER.subn(
|
||||
lambda m: m.group(1) + m.group(2), text)
|
||||
return text, stats
|
||||
|
||||
|
||||
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/bin/bash
|
||||
# lv-mccarthy D4 — instruction pairs. Run on pfi-gx10 as infra-ops.
|
||||
#
|
||||
# Deviations from the Hemingway D4 recipe, each forced by a measurement:
|
||||
# --register mccarthy new; it names McCarthy's punctuation deliberately, see REGISTERS
|
||||
# --source-entities MANDATORY; Hemingway's pairs predate it
|
||||
# --reflow-hard-wraps DEFECT 4; The Road is hard-wrapped and the other 5 works are not
|
||||
# --drop-leading-heading now also drops Blood Meridian's dash-separated chapter arguments
|
||||
# --n 3722 / --n 276 the full passage yield of each split, not a round number
|
||||
set -e
|
||||
cd ~/lv-mccarthy
|
||||
mkdir -p pairs
|
||||
S=scripts/yarros-corpus/build_sft_pairs.py
|
||||
COMMON="--corpus corpus-renamed/copies --register mccarthy --model gen
|
||||
--key-file $HOME/.config/litellm/all-agents-key
|
||||
--source-entities corpus-clean/entities-final.json
|
||||
--context-frac 1.0 --drop-leading-heading --reflow-hard-wraps --seed 4919"
|
||||
python3 $S $COMMON --split train --n 3722 --out pairs/pairs-full.jsonl > pairs/train.log 2>&1
|
||||
python3 $S $COMMON --split val --n 276 --out pairs/pairs-val.jsonl > pairs/val.log 2>&1
|
||||
echo DONE > pairs/.complete
|
||||
@@ -27,6 +27,65 @@ from pathlib import Path
|
||||
|
||||
NONCE = "Qxzvwolfram" # negative control: appears in no corpus
|
||||
|
||||
# ⚠⭐ THE SEPARATOR CLASS — a leak the `\b(Surface)\b` scan above structurally CANNOT see.
|
||||
# Any character inserted inside a name defeats a word-boundary pattern outright, so a mangled
|
||||
# occurrence is not merely unrepaired: it is unrenameable AND unreportable, and the gate prints
|
||||
# a clean zero over it. Two mechanisms are already measured in this line:
|
||||
#
|
||||
# lv-bronte 2026-09-16 `_Antigua_` `_` is a word character, so \bAntigua\b cannot match
|
||||
# inside it. Found by hand, not by this gate.
|
||||
# lv-mccarthy 2026-09-17 `B ell` a small-caps drop cap survived as its own token
|
||||
# `Toad-vine` a print line-break hyphen survived the extraction
|
||||
#
|
||||
# McCarthy's tree had already PASSED this gate at "0 of 75 renameable and 0 of 37
|
||||
# sub-threshold" while carrying 13 occurrences of five protagonist names — Bell, Chigurh,
|
||||
# Moss, Toadvine, Glanton — in all six copies. Every VISIBLE occurrence had been renamed
|
||||
# correctly, which is exactly what makes the residue invisible to a spot-read.
|
||||
#
|
||||
# ⚠ A NAIVE separator-tolerant scan is dominated by FALSE POSITIVES, because it matches across
|
||||
# genuine word boundaries. Measured on lv-hemingway: 21 raw hits, of which 18 are ordinary text
|
||||
# (`God damn` for the surface `Goddamn` ×14, plus `I run`/`On an`/`Do me`/`Si le`) and 3 are
|
||||
# real (`Primi tivo`, `Pasionar ia`, `Chi cote`). The discriminator that separates them without
|
||||
# a dictionary: in a genuine split, at least one FRAGMENT is not a word — it occurs as a
|
||||
# standalone token almost nowhere in the source. `God` and `damn` occur constantly; `Primi`,
|
||||
# `tivo`, `ell`, `higurh` and `Toad` do not. That single test cleared all 18 and kept all 3.
|
||||
SPLIT_SEP = r"[ _\-\u00ad\u2010\u2011]"
|
||||
TOKEN = re.compile(r"[A-Za-z\u00c0-\u017f']+")
|
||||
|
||||
|
||||
def split_scan(source: dict[str, str], copies: dict[str, str], surfaces: list[str],
|
||||
frag_max: int, allow: set[str]) -> dict[str, dict]:
|
||||
"""Surfaces surviving in the copies with ONE separator hiding them from the unigram scan.
|
||||
|
||||
Returns surface -> {"forms": {matched_string: hits}, "copies": n}. Exact matches are the
|
||||
unigram pass's business and are excluded here so the two cannot double-report.
|
||||
"""
|
||||
src_tokens = Counter()
|
||||
for t in source.values():
|
||||
src_tokens.update(TOKEN.findall(t))
|
||||
out: dict[str, dict] = {}
|
||||
for s in surfaces:
|
||||
if len(s) < 4 or not TOKEN.fullmatch(s):
|
||||
continue
|
||||
body = (SPLIT_SEP + "?").join(re.escape(c) for c in s)
|
||||
pat = re.compile(r"(?<![A-Za-z])(" + body + r")(?![A-Za-z])")
|
||||
forms: Counter = Counter()
|
||||
seen_copies = set()
|
||||
for name, text in copies.items():
|
||||
for m in pat.finditer(text):
|
||||
form = m.group(1)
|
||||
if form == s or form in allow:
|
||||
continue
|
||||
frags = TOKEN.findall(form)
|
||||
# A genuine split leaves a fragment that is not a word of this corpus.
|
||||
if len(frags) < 2 or all(src_tokens[f] > frag_max for f in frags):
|
||||
continue
|
||||
forms[form] += 1
|
||||
seen_copies.add(name)
|
||||
if forms:
|
||||
out[s] = {"forms": dict(forms), "copies": len(seen_copies)}
|
||||
return out
|
||||
|
||||
|
||||
def load_works(corpus: Path) -> dict[str, str]:
|
||||
man = json.loads((corpus / "manifest.json").read_text())
|
||||
@@ -79,6 +138,14 @@ def main() -> int:
|
||||
"real-world or generic. Without it the phrase audit does not run.")
|
||||
ap.add_argument("--phrase-min", type=int, default=5,
|
||||
help="a capitalised 2-3gram must recur this often in the source to be audited")
|
||||
ap.add_argument("--split-frag-max", type=int, default=3,
|
||||
help="a separator-split candidate is reported only when one of its "
|
||||
"fragments occurs as a standalone token no more than this often in "
|
||||
"the source. Raising it reports more and flags more ordinary text; "
|
||||
"0 turns the fragment filter off entirely.")
|
||||
ap.add_argument("--no-split-scan", action="store_true",
|
||||
help="skip the separator-split pass. It is ON by default because the pass "
|
||||
"exists to catch a class that made this gate print a false zero.")
|
||||
ap.add_argument("--report", default=None, help="write the full JSON breakdown here")
|
||||
a = ap.parse_args()
|
||||
|
||||
@@ -158,6 +225,37 @@ def main() -> int:
|
||||
for ph, w in sorted(surviving_phrases.items(), key=lambda kv: -kv[1]["source"])[:30]:
|
||||
print(f" {ph:<34} source {w['source']:>4} copies {w['copies']:>5}")
|
||||
|
||||
# ---- separator-split audit --------------------------------------------
|
||||
# ⚠ Its own controls, because a scan that only ever sees clean text cannot tell `absent`
|
||||
# from `blind` -- the same argument that put the positive control on the unigram pass.
|
||||
split_surv: dict[str, dict] = {}
|
||||
split_ok = True
|
||||
if not a.no_split_scan:
|
||||
allow = set()
|
||||
if a.phrase_map:
|
||||
allow = set(json.loads(Path(a.phrase_map).read_text()).get("split_allow", []))
|
||||
probe = next((s for s in sorted(surfaces, key=len, reverse=True)
|
||||
if len(s) >= 4 and TOKEN.fullmatch(s)), None)
|
||||
if probe:
|
||||
# POSITIVE: the same surface with one separator inserted MUST be detected.
|
||||
planted = {"__control__": f"the {probe[0]} {probe[1:]} rode on"}
|
||||
pos = split_scan(source, planted, [probe], a.split_frag_max, set())
|
||||
# NEGATIVE: the same split nonce, hunted in the REAL copies, must NOT be found.
|
||||
# ⚠ The first version of this control planted the nonce in its own probe text and
|
||||
# then asserted it was absent, so it failed by construction on every run. A control
|
||||
# that cannot pass is not a control; it is an alarm wired to itself.
|
||||
neg = split_scan(source, copies, [NONCE], a.split_frag_max, set())
|
||||
split_ok = bool(pos) and not neg
|
||||
print(f"\n [{'PASS' if split_ok else 'FAIL'}] split-scan controls: planted "
|
||||
f"`{probe[0]} {probe[1:]}` {'detected' if pos else 'MISSED'}, split nonce "
|
||||
f"{'absent' if not neg else 'FALSELY DETECTED'}")
|
||||
split_surv = split_scan(source, copies, surfaces, a.split_frag_max, allow)
|
||||
print(f" SEPARATOR-SPLIT survivors: {len(split_surv)} surfaces hidden from the "
|
||||
f"unigram scan by a space, hyphen or underscore inside the name")
|
||||
for s_, w in sorted(split_surv.items(), key=lambda kv: -sum(kv[1]["forms"].values())):
|
||||
print(f" {s_:<18} {sum(w['forms'].values()):>5} hits in {w['copies']} copies "
|
||||
f"as {', '.join(repr(k) for k in sorted(w['forms']))}")
|
||||
|
||||
if a.report:
|
||||
Path(a.report).write_text(json.dumps({
|
||||
"renameable_total": len(renameable), "sub_threshold_total": len(sub_threshold),
|
||||
@@ -168,15 +266,21 @@ def main() -> int:
|
||||
"surviving_sub_threshold": {s: {"hits": sum(w.values()), "copies": len(w)}
|
||||
for s, w in surv_sub.items()},
|
||||
"surviving_phrases": surviving_phrases,
|
||||
"split_scan_ran": not a.no_split_scan,
|
||||
"split_scan_controls_pass": split_ok,
|
||||
"split_frag_max": a.split_frag_max,
|
||||
"surviving_separator_split": split_surv,
|
||||
}, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print(f"\n wrote {a.report}")
|
||||
|
||||
if not (pos_ok and neg_ok):
|
||||
if not (pos_ok and neg_ok and split_ok):
|
||||
print("\n== CONTROLS FAILED -- this gate's verdict is not trustworthy"); return 2
|
||||
if surv_renameable or surv_sub or surviving_phrases:
|
||||
print(f"\n== GATE FAILED: {len(surv_renameable) + len(surv_sub)} source entities and "
|
||||
f"{len(surviving_phrases)} phrases survive"); return 1
|
||||
print("\n== GATE PASSED: 0 source entities and 0 audited phrases survive in any copy")
|
||||
if surv_renameable or surv_sub or surviving_phrases or split_surv:
|
||||
print(f"\n== GATE FAILED: {len(surv_renameable) + len(surv_sub)} source entities, "
|
||||
f"{len(surviving_phrases)} phrases and {len(split_surv)} separator-split "
|
||||
f"surfaces survive"); return 1
|
||||
print("\n== GATE PASSED: 0 source entities, 0 audited phrases and 0 separator-split "
|
||||
"surfaces survive in any copy")
|
||||
print(f" ⚠ sensitivity floor: a name appearing fewer than {a.min_cap} times per work is "
|
||||
f"never detected, and a phrase recurring fewer than {a.phrase_min} times is never "
|
||||
f"audited. Neither is renamed, and neither is reported here.")
|
||||
|
||||
@@ -70,6 +70,31 @@ REGISTERS = {
|
||||
"analysis and moral reflection, occasional direct address to the reader, and "
|
||||
"physical setting — Yorkshire weather, schoolrooms, Belgian pensionnats — "
|
||||
"rendered with the feeling it carries"),
|
||||
# ⭐ McCarthy is the first register in this map that names PUNCTUATION, and that is a
|
||||
# GATE-DESIGN choice made before any McCarthy number existed, not a description choice.
|
||||
# The eval harness drives the base (unadapted) control arm with this same prompt via
|
||||
# `gen_beats_chat_yarros.py --system-from <pairs provenance>`, and `voice_distance.py` is
|
||||
# Burrows's Delta over CHARACTER BIGRAMS. An adapter that learns only "emit no quotation
|
||||
# marks" moves delta_cb a long way without having learned a sentence -- and on a corpus
|
||||
# measuring 0.0 quote marks per 10k words against Hemingway's 838 that is the single
|
||||
# cheapest available trick. Stating the tics here hands them to the control arm too, so
|
||||
# the adapter earns no delta for them and the remaining gap is attributable to sentence
|
||||
# structure, which is what the axis claims to measure. lv-mccarthy D1 pre-registered a
|
||||
# punctuation-normalised SECONDARY read for exactly this risk; this closes it in the
|
||||
# PRIMARY read as well. Cost, stated up front: the voice axis gets harder, and on an
|
||||
# underpowered fixture that risks a Brontë-style marginal result. McCarthy's val split
|
||||
# yields 269 in-band passages against Brontë's 44, which is the reason that trade is
|
||||
# affordable here and was not there.
|
||||
# ⚠ Deliberately NOT mentioned: the untranslated Spanish dialogue of the Border Trilogy.
|
||||
# It is a property of 3 of the 6 works, not of the voice, and inviting an LLM to produce
|
||||
# Spanish on a beat that has none is damage rather than register.
|
||||
"mccarthy": ("Cormac McCarthy", "third-person past-tense narration held at the surface of "
|
||||
"things, with no access to what anyone thinks; long sentences strung together "
|
||||
"on `and`, set against clipped fragment paragraphs; dialogue carried WITHOUT "
|
||||
"quotation marks, each speech its own paragraph, attributed plainly or not at "
|
||||
"all; contractions written with no apostrophe — `dont`, `aint`, `wont`, "
|
||||
"`didnt`; terrain, animals, weather and tools named concretely and "
|
||||
"technically; violence and landscape rendered flatly and without comment"),
|
||||
}
|
||||
|
||||
CONTEXT_BLOCK = """The passage is preceded by this, for reference only. Do NOT write a beat for it — it is
|
||||
@@ -135,6 +160,8 @@ def post(path: str, payload: dict, key: str, timeout: int = 120) -> dict:
|
||||
|
||||
|
||||
HEADING_MAX_WORDS = 6
|
||||
# A dash-separated fragment list: the shape of a chapter argument, not of prose.
|
||||
ARGUMENT_LIST = re.compile(r"\s[-\u2013\u2014]\s")
|
||||
|
||||
|
||||
def drop_leading_heading(paras: list[str], enabled: bool) -> list[str]:
|
||||
@@ -152,10 +179,112 @@ def drop_leading_heading(paras: list[str], enabled: bool) -> list[str]:
|
||||
"""
|
||||
if not enabled or not paras:
|
||||
return paras
|
||||
return paras[1:] if len(paras[0].split()) <= HEADING_MAX_WORDS else paras
|
||||
if len(paras[0].split()) <= HEADING_MAX_WORDS:
|
||||
paras = paras[1:]
|
||||
# ⭐ AND THEN THE CHAPTER ARGUMENT, which is a heading that is 40 words long.
|
||||
# Blood Meridian sets each chapter's argument as a dash-separated list of title-case
|
||||
# fragments under the roman numeral -- `Desert castaways - The backtrack - A hideout -
|
||||
# The wind takes a side - The judge returns` -- hard-wrapped across several short
|
||||
# paragraphs. The <=6-word rule drops the `XXI` above it and leaves the argument, so a
|
||||
# chapter-opening passage trains the carrier to emit a dash-separated summary before the
|
||||
# prose. Same harm as the bare heading, a different shape.
|
||||
# Measured: 131 paragraphs, ALL in blood-meridian; 0 in the other five McCarthy works,
|
||||
# 0 in all ten Hemingway works and 0 in all four Brontë works, so this extension is a
|
||||
# byte no-op on every pair set already shipped under this flag.
|
||||
while paras and ARGUMENT_LIST.search(paras[0]) and len(paras[0].split()) <= 20:
|
||||
paras = paras[1:]
|
||||
return paras
|
||||
|
||||
|
||||
def chunk(corpus: Path, lo: int, hi: int, split: str, drop_heading: bool = False) -> list[dict]:
|
||||
# A line that ends a sentence, after detached terminal punctuation is closed up: this corpus
|
||||
# carries 151 occurrences of `boxcutter .` in The Road, and a naive test reads those as
|
||||
# mid-sentence and joins across a real paragraph break.
|
||||
SENT_FINAL = re.compile(r'[.!?"\u201d]$')
|
||||
DETACHED_PUNCT = re.compile(r"\s+([.,;:!?])")
|
||||
|
||||
|
||||
def reflow_hard_wraps(paras: list[str], enabled: bool) -> tuple[list[str], int, int]:
|
||||
"""Undo an extraction's HARD LINE WRAPPING inside a paragraph. Returns (paras, joined, split).
|
||||
|
||||
⚠ DEFECT 4 of this pair build, and unlike the first three it was MEASURED ON A SHIPPED
|
||||
ADAPTER before it was fixed here. `lv-bronte` trained on a corpus whose four works are
|
||||
100% hard-wrapped at ~68 characters (Gutenberg plain text), and the wrap transfers
|
||||
straight through to the product:
|
||||
|
||||
arm intra-para breaks MID-SENTENCE per 1k chars
|
||||
bronte base (control) 0 0 0.00
|
||||
bronte ckpt475 (SHIPPED) 1,251 1,219 12.46
|
||||
bronte ckpt925 1,134 1,087 11.79
|
||||
hemingway base/ckpt*/all 58/0/0 0 0.00 <- 0% wrapped corpus
|
||||
|
||||
Both controls fire: the base arms emit none, so the carrier is not the source, and the
|
||||
Hemingway arms emit none, so the instrument is not manufacturing signal. `score_beats.py`
|
||||
does not look for this and passed Brontë's damage axis anyway (ran-on +0.15 / 0.400 floor),
|
||||
so nothing downstream would have reported it.
|
||||
|
||||
McCarthy is the MIXED case, which is worse to learn than either pure one: The Road is
|
||||
hard-wrapped (3,587 intra-paragraph newlines, 60.9 per 1k words) and the other five works
|
||||
have exactly ZERO, so the corpus teaches the break as a coin flip. The transform is
|
||||
therefore SELF-TARGETING -- a paragraph with no interior newline is returned untouched --
|
||||
and needs no per-work special-casing.
|
||||
|
||||
⚠⚠ THE OBVIOUS FIX -- join every interior newline with a space -- CORRUPTS THE THING THIS
|
||||
ADAPTER EXISTS TO LEARN. 46 paragraphs in The Road are two-speaker exchanges whose blank
|
||||
line was lost, and McCarthy's dialogue is unmarked, so the speaker boundary IS the
|
||||
paragraph boundary:
|
||||
|
||||
"And we're carrying the fire." || 'Yes.'
|
||||
"I'm going to blow out the lamp." || 'Is that okay?'
|
||||
'Take me with you, the boy said.' || 'He looked as if he was going to cry.'
|
||||
|
||||
Joining those puts two speakers in one paragraph and unmarked dialogue stops parsing.
|
||||
|
||||
⚠ A WRAP-WIDTH test cannot separate them either, and this was measured rather than
|
||||
assumed: the wrap is POSITION-DEPENDENT -- first lines of a paragraph break at ~30
|
||||
characters and later lines at ~78 -- so `We're not the first ones here.` (29) is
|
||||
geometrically indistinguishable from a full line, and a max-line-length rule splits 347
|
||||
paragraphs of which the majority are genuine wraps (`If you died I would want to die` ||
|
||||
`too.`).
|
||||
|
||||
So the discriminator is PUNCTUATION, and the error budget is deliberately asymmetric:
|
||||
|
||||
- Li does NOT end sentence-final -> a wrap. JOIN with a space. 3,315 of 3,587 (92.4%),
|
||||
and this is the ENTIRE defect class: a mid-sentence newline is the thing the Brontë
|
||||
adapter learned to emit.
|
||||
- Li DOES end sentence-final -> promote the newline to a paragraph BREAK. 272 cases.
|
||||
Correct for all 46 dialogue exchanges; for some fragmentary narration it inserts a
|
||||
paragraph break the book does not have.
|
||||
|
||||
That residual is the cheap error on purpose. A spurious break inside McCarthy narration is
|
||||
invisible -- the page is already full of one-line fragment paragraphs -- while a merged
|
||||
pair of speakers is a form error in the corpus's most distinctive feature. Splitting
|
||||
where the book does not costs paragraphing; joining where the book does not costs the
|
||||
voice.
|
||||
"""
|
||||
if not enabled:
|
||||
return paras, 0, 0
|
||||
out: list[str] = []
|
||||
joined = promoted = 0
|
||||
for para in paras:
|
||||
lines = [x.strip() for x in para.split("\n") if x.strip()]
|
||||
if len(lines) < 2:
|
||||
out.append(para)
|
||||
continue
|
||||
buf = [lines[0]]
|
||||
for nxt in lines[1:]:
|
||||
if SENT_FINAL.search(DETACHED_PUNCT.sub(r"\1", buf[-1])):
|
||||
out.append(" ".join(buf))
|
||||
buf = [nxt]
|
||||
promoted += 1
|
||||
else:
|
||||
buf.append(nxt)
|
||||
joined += 1
|
||||
out.append(" ".join(buf))
|
||||
return out, joined, promoted
|
||||
|
||||
|
||||
def chunk(corpus: Path, lo: int, hi: int, split: str, drop_heading: bool = False,
|
||||
reflow: bool = False) -> list[dict]:
|
||||
"""Accumulate consecutive paragraphs into product-band passages, per chapter.
|
||||
|
||||
Chapter-bounded so a passage never straddles a chapter break. The trailing buffer of
|
||||
@@ -163,12 +292,16 @@ def chunk(corpus: Path, lo: int, hi: int, split: str, drop_heading: bool = False
|
||||
teach the length the product is trying to hold.
|
||||
"""
|
||||
out = []
|
||||
reflow_joined = reflow_split = 0
|
||||
for f in sorted(corpus.glob("*.copy0.jsonl")):
|
||||
for line in f.read_text(encoding="utf-8").splitlines():
|
||||
d = json.loads(line)
|
||||
if d.get("split") != split:
|
||||
continue
|
||||
paras = [p.strip() for p in re.split(r"\n\s*\n", d["text"]) if p.strip()]
|
||||
paras, _j, _s = reflow_hard_wraps(paras, reflow)
|
||||
reflow_joined += _j
|
||||
reflow_split += _s
|
||||
paras = drop_leading_heading(paras, drop_heading)
|
||||
buf, n, prev = [], 0, None
|
||||
for p in paras:
|
||||
@@ -181,6 +314,9 @@ def chunk(corpus: Path, lo: int, hi: int, split: str, drop_heading: bool = False
|
||||
else:
|
||||
prev = None # oversized run dropped; context would be a lie
|
||||
buf, n = [], 0
|
||||
if reflow:
|
||||
print(f"[reflow] {reflow_joined} wrapped lines rejoined, "
|
||||
f"{reflow_split} interior newlines promoted to paragraph breaks", flush=True)
|
||||
return out
|
||||
|
||||
|
||||
@@ -316,6 +452,14 @@ def main() -> int:
|
||||
ap.add_argument("--drop-leading-heading", action="store_true",
|
||||
help="DEFECT 3 fix: drop a unit's opening block when it is a bare chapter "
|
||||
"heading. OFF by default so the BabyYarros pair set stays byte-reproducible.")
|
||||
ap.add_argument("--reflow-hard-wraps", action="store_true",
|
||||
help="DEFECT 4 fix: rejoin lines an extraction hard-wrapped mid-sentence "
|
||||
"inside a paragraph. MEASURED on the shipped lv-bronte adapter, which "
|
||||
"emits mid-sentence line breaks at 12.5 per 1k chars against 0.00 for "
|
||||
"its own base control. Self-targeting -- a no-op on any work whose "
|
||||
"paragraphs hold no interior newline, which is 5 of 6 McCarthy works "
|
||||
"and all 10 Hemingway works. OFF by default so the Yarros, Hemingway "
|
||||
"and Brontë pair sets stay byte-reproducible.")
|
||||
ap.add_argument("--control-in", default=None,
|
||||
help="JSON of hand-written beats [{idx,beat}]; generate beats for the SAME "
|
||||
"passages and print side by side. The positive control -- a beat "
|
||||
@@ -328,7 +472,8 @@ def main() -> int:
|
||||
author, register = REGISTERS[a.register]
|
||||
sys_prompt = SYS_PASSAGE.format(author=author, register=register)
|
||||
key = a.key or (Path(a.key_file).read_text().strip() if a.key_file else None)
|
||||
passages = chunk(Path(a.corpus), a.lo, a.hi, a.split, a.drop_leading_heading)
|
||||
passages = chunk(Path(a.corpus), a.lo, a.hi, a.split, a.drop_leading_heading,
|
||||
a.reflow_hard_wraps)
|
||||
print(f"[chunk] {len(passages)} passages in split={a.split}, band {a.lo}-{a.hi}", flush=True)
|
||||
if not passages:
|
||||
print("REFUSING: no passages -- wrong corpus dir or split", file=sys.stderr)
|
||||
@@ -474,6 +619,7 @@ def main() -> int:
|
||||
"system_prompt": sys_prompt,
|
||||
"register": a.register or "yarros-frozen-one-paragraph",
|
||||
"drop_leading_heading": a.drop_leading_heading,
|
||||
"reflow_hard_wraps": a.reflow_hard_wraps,
|
||||
}
|
||||
Path(str(out_path) + ".provenance.json").write_text(json.dumps(prov, indent=2), encoding="utf-8")
|
||||
print(f"[done] {kept} pairs -> {out_path}")
|
||||
|
||||
Reference in New Issue
Block a user