From 328e9b1e5642db5743621430775a0a481c4ab314 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Thu, 17 Sep 2026 11:39:07 -0700 Subject: [PATCH] =?UTF-8?q?memory:=20snapshot=20=E2=80=94=20the=20mccarthy?= =?UTF-8?q?=20leak=20gate=20passed=20with=20five=20protagonist=20names=20i?= =?UTF-8?q?n=20every=20copy,=20and=20the=20chain=20it=20happened=20on=20wa?= =?UTF-8?q?s=20unrecorded?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-09-17-mccarthy-split-name-leak.md | 95 +++++++++++++++++++ persistent-memory.md | 44 ++++----- 2 files changed, 116 insertions(+), 23 deletions(-) create mode 100644 persistent-memory.d/2026-09-17-mccarthy-split-name-leak.md diff --git a/persistent-memory.d/2026-09-17-mccarthy-split-name-leak.md b/persistent-memory.d/2026-09-17-mccarthy-split-name-leak.md new file mode 100644 index 0000000..a3c8d21 --- /dev/null +++ b/persistent-memory.d/2026-09-17-mccarthy-split-name-leak.md @@ -0,0 +1,95 @@ +# `[2026-09-17]` The leak gate passed with five protagonist names still in every copy + +Found during D4 pre-flight, three stages downstream of where it happened. Commit `c559664`. + +``` +leak gate, 2026-09-17 morning 0 of 75 renameable, 0 of 37 sub-threshold, both controls green +actually present, all 6 copies Bell x2 Chigurh x3 Moss x2 Toadvine x4 Glanton x2 +``` + +## The mechanism + +`leak_gate.py` scans `\b(Surface)\b`. **A character inserted inside a name defeats that +pattern outright**, so a mangled occurrence is not merely unrepaired — it is *unrenameable* +by `rename.py` and *unreportable* by the gate, and the gate prints a clean zero over it. +Two extraction artifacts produce exactly that: + +``` +B ell C higurh M oss T oadvine a small-caps drop cap kept as its own token +Toad-vine Glan-ton a print line-break hyphen kept by the extractor +``` + +⭐ **Every VISIBLE occurrence had been renamed correctly** — exact-match survivors were 0, +as the gate said. That is what makes this residue invisible to a spot-read: the names are +gone everywhere you look. `Bell` sits in the entity map at 147 capitals, `Glanton` at 365. + +This is the third member of a family. lv-bronte's was `_Antigua_` (`_` is a word character, +so `\bAntigua\b` cannot match inside it), found by hand in 2026-09-16 and never generalised. +**The generalisation is the point: any separator inside a name blinds a word-boundary scan.** + +## Fixed at three levels, and all three must stay + +1. **`build_corpus_mccarthy.py` rules 4 and 5** repair the source text — 32 split initials + with a *lowercase* remainder (rule 1 requires a following ALL-CAPS word and DROPCAP + requires two, so this is the class both leave behind), 5 hyphen-split names by name. + Both carry expected counts so a master change fails the build. + ⚠ **Rule 4's letter class is consonants only.** `I` opens **1,966** paragraphs (the + pronoun), `A` opens 143 (the article), `Y` opens 32 (Spanish *y*). Folding any of them + would corrupt 2,141 lines to fix 32 — the same `I`/`A` trap that bit `audit_stoplist.py`. +2. **`leak_gate.py` runs a separator-tolerant pass every time**, with its own positive and + negative controls, and **it fails the gate**. Validated against the pre-fix tree: reports + all five surfaces, exits 1. +3. The exact-match passes are untouched, so the old verdict is reproduced alongside the new. + +⚠ **The fragment filter is what makes the new pass usable.** A naive separator-tolerant +scan is dominated by false positives — on Hemingway it returns 21 hits of which **18 are +ordinary text** (`God damn` for the surface `Goddamn` ×14, plus `I run`, `On an`, `Do me`, +`Si le`). The discriminator, with no dictionary: in a genuine split at least one FRAGMENT +is not a word of this corpus. `God` and `damn` occur constantly; `Primi`, `tivo`, `ell`, +`higurh`, `Toad` do not. That one test cleared all 18 and kept all 3 real ones. + +⚠ **My first negative control could not pass.** It planted the split nonce in its own probe +text and then asserted the nonce was absent — an alarm wired to itself, failing on every +run. It now hunts the split nonce in the *real* copies. A control that cannot pass is not a +control. + +## The shipped corpora, checked with the committed instrument + +``` +lv-bronte clean of this class (`English women` vs `Englishwomen` is a false positive) +lv-hemingway 3 real: `Primi tivo`, `Pasionar ia`, `Chi cote` — SHIPPED and LIVE +``` + +**Operator call outstanding** on whether 3 occurrences in 958k words warrant re-gating a +live adapter. Not acted on. + +## The chain was recovered, not remembered — and is now written down + +There was **no McCarthy runbook**, and the D1→D3 session issued its commands over +non-interactive ssh so no shell history survived. The chain was recovered by rebuilding +candidates and matching sha256 against the artifacts on disk, then pinned: + +``` +D1 build_corpus_mccarthy.py 6 works byte-identical +D2 entities.py --min-count 5 --fold-clitics --drop-acronyms --min-mid-ratio 0.2 --min-mid 2 +D2c apply_gender_overrides.py entities-final.json byte-identical +D3 rename.py --preset mccarthy --scope corpus --min-cap 5 --copies 6 --seed 4919 + --holdout-chapter 7 17 all 36 copy files byte-identical +``` + +⚠ `--min-mid-ratio` is what keeps `Yeah`/`Buenas`/`Shh`/`Sí` out of the map. The map is +**insensitive** to it: any value in [0.05, 0.3] with `--min-mid` 1 or 2 reproduces byte-for- +byte; `--min-mid 3` does not. The original values are unrecoverable and it does not matter — +which is worth saying, because an exact-looking recipe that was never pinned invites a +false claim of reproduction. Full recipe and every deviation: `scripts/mccarthy-corpus/RUNBOOK.md`. + +⚠ **D1 must run on nh3-dev** — the builder reads the kvasir catalogue by absolute path and +gx10 has no copy. The previous session's "on gx10" note is true of D2 onward only. + +⚠ **No phrase map exists for this corpus**, so the gate's phrase audit does not run at all. +Yarros and Brontë both had one. Not closed. + +Rollback: `~/lv-mccarthy/corpus-{clean,renamed}.pre-splitfix` on gx10. + +Related: [[2026-09-17-mccarthy-d1-d3]], [[2026-09-17-beat-contamination-leak]], +[[2026-09-17-lv-hemingway-gate]], [[2026-09-17-lv-bronte-gate]]. diff --git a/persistent-memory.md b/persistent-memory.md index 80b395d..694356f 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -1,6 +1,6 @@ # Persistent memory — eshpfi-management -_Last updated: 2026-09-17 ~09:00 PT (lv-mccarthy through D3 on gx10 — leak gate PASSED, val split larger than Hemingway's; SFT pairs are next and need a `mccarthy` register written first. lv-krakauer parked.)_ +_Last updated: 2026-09-17 ~12:15 PT (lv-mccarthy: the leak gate had PASSED with five protagonist names still in every copy — hidden from `\bName\b` by a drop-cap space or a line-break hyphen. Corpus rebuilt, gate taught to see the class, D4 pairs generating on gx10. lv-hemingway carries 3 of the same and is LIVE — operator call. lv-krakauer parked.)_ > **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its > `Written:` stamp is under **8 hours** old, read it (it carries the in-flight @@ -117,32 +117,24 @@ no longer deployed sidecars here. See Recent decisions.) _As of 2026-09-17 ~09:00 PT._ -### NEXT: build the SFT pairs for `lv-mccarthy` +### IN FLIGHT: lv-mccarthy D4 pairs, generating on pfi-gx10 -Everything upstream is done and gated. `~/lv-mccarthy/` on **pfi-gx10** holds `corpus-clean/` -(167 units, 584,716 words, entities-final.json), `corpus-renamed/` (6 copies, 1,002 records, -leak gate PASSED) and `scripts/`. +`~/lv-mccarthy/run-pairs.sh` launched 2026-09-17 ~11:50 PT: **3,722 train** then **276 val** +pairs against the `gen` seat, ~25 min. Seat fingerprint `vllm-0.1.1.dev50+geed1f3d0c-601ad866` +— **identical to Hemingway's run**, so the two pair sets are generator-comparable. +Watch `pairs/train.log`; `pairs/.complete` is written at the end. -The pairs step, mirroring the Hemingway run: +Then `train_pairs_lora.py`. Before the eval, honour D1's pre-registration: a +**punctuation-normalised secondary read** of the voice axis. -``` -gx10:~/lv-mccarthy -scripts/yarros-corpus/build_sft_pairs.py \ - --corpus corpus-renamed/copies --split train --copy copy0 \ - --register \ - --source-entities corpus-clean/entities-final.json \ - --context-frac 1.0 --drop-leading-heading --out pairs/pairs-full.jsonl -``` +⚠ The corpus under it was **rebuilt at ~11:45 PT** — see the split-name leak below. Old tree +kept at `corpus-{clean,renamed}.pre-splitfix` on gx10. Recipe: `scripts/mccarthy-corpus/RUNBOOK.md`. -⚠ **`--register mccarthy` DOES NOT EXIST YET** — `build_sft_pairs.py`'s REGISTERS map has -yarros/bronte/hemingway only. It must be written before the run: it is the system prompt the -beat-generator uses AND the one the trainer and the eval are bound to. -⚠ **Pass `--source-entities`** — it exists now and Hemingway's pairs predate it. This is the -beat-contamination filter; running without it is how 70 of 7,094 Hemingway beats ended up -naming Santiago, Catherine and Rinaldi. -⚠ Pairs generation runs against the `gen` seat through LiteLLM (`10.250.50.70:4000`), took -**2,759 s (~46 min)** for Hemingway's 7,094 pairs, and the seat was confirmed up. -Then `--split val` for the val pairs, then `train_pairs_lora.py`. +### OPERATOR CALL — lv-hemingway carries 3 of the same hidden survivors and is LIVE + +`Primi tivo`, `Pasionar ia`, `Chi cote` in a 958k-word corpus, on the adapter serving at +fv-ml1 GPU 0 `:8027`. lv-bronte is clean of the class. Re-gating means a corpus rebuild and +a retrain; three occurrences may not be worth it. **Not acted on.** ### SHIPPED — `lv-hemingway` on ckpt850, live on `vllm-voices` (fv-ml1 GPU 0 :8027) @@ -187,6 +179,12 @@ fused MoE kernel path (**parked, id 47**), TTS-stack move to fv-ml1 (**parked, i ## Recent decisions +- `[2026-09-17]` ⭐⭐⭐ **The leak gate PASSED lv-mccarthy while five protagonist names sat in all six copies, and the blind spot generalises to every corpus in the line.** `\b(Surface)\b` cannot match a name with a character inserted in it, so a mangled occurrence is unrenameable AND unreportable: `B ell`, `C higurh`, `M oss`, `T oadvine` (a small-caps drop cap kept as its own token) and `Toad-vine`, `Glan-ton` (a print line-break hyphen). Every VISIBLE occurrence had been renamed, which is what made it invisible. Same family as lv-bronte's `_Antigua_`, now generalised: **any separator inside a name blinds a word-boundary scan.** Fixed in the corpus builder (rules 4+5, counted), and `leak_gate.py` now runs a separator-tolerant pass with its own controls that FAILS the gate — validated against the pre-fix tree. ⚠ Its fragment filter is load-bearing: a naive scan returns 18 false positives on Hemingway (`God damn`, `I run`) against 3 real. Whole D1→D3 chain reproduced byte-identically before and after. Commit `c559664`. → `persistent-memory.d/2026-09-17-mccarthy-split-name-leak.md` +- `[2026-09-17]` ⭐⭐ **The SHIPPED lv-bronte adapter emits mid-sentence line breaks at 12.46 per 1k chars, and nothing downstream looks for it.** Its corpus is 100% hard-wrapped at ~68 chars (Gutenberg plain text) and the wrap transfers: base control 0.00, ckpt475 (shipped) 12.46, ckpt925 11.79, every Hemingway arm 0.00 on a 0%-wrapped corpus. Both controls fire. `score_beats.py` passed Brontë's damage axis anyway. McCarthy is the MIXED case — The Road wrapped, the other five works not — which is worse to learn than either pure one, so `build_sft_pairs.py --reflow-hard-wraps` (DEFECT 4) fixes it at pair time, off by default. ⚠ The obvious fix, joining every interior newline, CORRUPTS 46 two-speaker exchanges whose blank line was lost — and unmarked dialogue is the one thing this adapter exists to learn. The rule splits on sentence-final punctuation and takes the cheaper error deliberately. +- `[2026-09-17]` ⭐ **The `mccarthy` register names the punctuation ON PURPOSE, and that is a gate-design call made before any McCarthy number existed.** `eval-*.sh` drives the base control arm with the SAME system prompt via `--system-from`, and `voice_distance.py` is Burrows's Delta over character bigrams — so a tic left OUT of the register is a cheap win only the adapter can take, on a corpus measuring 0.0 quote marks per 10k against Hemingway's 838. Stating them hands them to the control too. Cost stated up front: the voice axis gets harder, and McCarthy's 276-passage val split (against Brontë's 44) is why that trade is affordable here and was not there. +- `[2026-09-17]` **lv-mccarthy's D1→D3 chain was RECOVERED, not remembered — there was no runbook and the commands went over non-interactive ssh, so no history survived.** Rebuilt candidates and matched sha256 against the artifacts on disk: 6 works, the entity map, the final map and all 36 copy files byte-identical. Now pinned in `scripts/mccarthy-corpus/RUNBOOK.md` with every deviation. ⚠ **D1 must run on nh3-dev** (the builder reads the kvasir catalogue by absolute path); the prior "on gx10" note is true of D2 onward only. ⚠ No phrase map exists for this corpus, so the gate's phrase audit never ran — Yarros and Brontë both had one. +- `[2026-09-17]` **Measured and DELIBERATELY not changed, three of them.** The oversize-passage drop is 13.9% of McCarthy's train words, between Hemingway's 10.0% and the shipped Brontë's 18.3% — in range, no change. `BEAT_PROMPT` asserts the passage is first-person and McCarthy is third; measured inert (**0** narrator-retries against Hemingway's 615 of 7,094), so the prompt was left alone. Blood Meridian's 131 dash-separated chapter-argument paragraphs DID warrant a change and `--drop-leading-heading` now eats them (0 in every other work of all three corpora). + - `[2026-09-17]` ⭐⭐ **A unit splitter must choose by SIZE, not count — and the val split scales with WORK COUNT, not corpus size.** `scripts/r49-corpus/split_units.py` + a multi-index `--holdout-chapter`. The inherited most-units rule gave Cities of the Plain 4 units of 22,312w (the book's PARTS); the single-index holdout would have given McCarthy a Brontë-class 18k-word val reference on a 588k corpus. Both fixed, both caught by controls. → `persistent-memory.d/2026-09-17-mccarthy-d1-d3.md` - `[2026-09-17]` ⭐ **lv-mccarthy D1–D3 complete on gx10, leak gate PASSED (0 of 75 renameable, 0 of 37 sub-threshold, both controls green).** Three McCarthy-specific calls, each forced by a measurement: corpus-scoped rename (the Border Trilogy shares 9 surfaces across books), a new `mccarthy` name preset (Hemingway's carries it_IT/fr_FR and McCarthy writes neither), and `--min-cap 5` to match the entity map's floor — the first gate run failed with 45 survivors purely because rename's floor was 8 and the map's was 5. → `persistent-memory.d/2026-09-17-mccarthy-d1-d3.md` - `[2026-09-17]` ⭐⭐ **PARKED lv-krakauer, and the reason is a selection criterion the line was missing: ask whether the author HAS a voice before investigating whether the corpus is clean.** Operator: *"he's a great writer because of his research, not because he has a strong identifiable voice."* I surveyed, built the corpus, measured containment and fixed three stripping defects before anyone asked the question that decided it. henge id 82. → auto-memory `feedback_voice_worth_adapting_before_corpus`.