vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(lv-mccarthy): the leak gate passed with five protagonist names still in every copy

Browse Source 
`leak_gate.py` scans `\b(Surface)\b`. Any character inserted inside a name defeats
that pattern outright, so a mangled occurrence is unrenameable by rename.py AND
unreportable by the gate. lv-mccarthy's 2026-09-17 tree passed at "0 of 75
renameable and 0 of 37 sub-threshold" while carrying 13 occurrences of Bell,
Chigurh, Moss, Toadvine and Glanton in all six copies:

    B ell  C higurh  M oss  T oadvine    a small-caps drop cap kept as its own token
    Toad-vine  Glan-ton                  a print line-break hyphen kept by the extractor

Every visible occurrence HAD been renamed, which is what made the residue invisible
to a spot-read. Fixed at three levels, all three of which must stay:

  build_corpus_mccarthy.py rules 4 and 5 repair the source text — 32 split initials
  with a lowercase remainder, 5 hyphen-split names, each with an expected count so a
  master change fails the build. Rule 4's letter class is consonants only: `I` opens
  1,966 paragraphs, `A` 143 and `Y` 32 (Spanish `y`); folding any would corrupt 2,141
  lines to fix 32.

  leak_gate.py gains a separator-tolerant pass with its own positive and negative
  controls, and it FAILS the gate. Validated against the pre-fix tree: reports all
  five surfaces, exits 1. Its fragment filter is what makes it usable — a naive scan
  returns 18 false positives on Hemingway (`God damn`, `I run`) against 3 real ones;
  requiring one fragment to be a non-word of the corpus cleared all 18 and kept all 3.

  The whole D1→D3 chain is reproduced byte-identically before and after, so the fix
  is the only delta: 6 works, the entity map, the final map and all 36 copy files.

Cross-checked on the shipped corpora: lv-bronte is clean of this class, lv-hemingway
carries 3 (`Primi tivo`, `Pasionar ia`, `Chi cote`) and is live on fv-ml1.

Also in build_sft_pairs.py, both needed before lv-mccarthy's pairs:

  DEFECT 4, hard-wrap reflow. Measured on the SHIPPED lv-bronte adapter, which emits
  mid-sentence line breaks at 12.46 per 1k chars against 0.00 for its own base control
  and 0.00 for every Hemingway arm. McCarthy is the mixed case — The Road is wrapped,
  the other five works are not — so the corpus teaches the break as a coin flip. The
  obvious fix (join every interior newline) corrupts 46 two-speaker exchanges whose
  blank line was lost, and unmarked dialogue is the one thing this adapter exists to
  learn; the rule splits on sentence-final punctuation instead and takes the cheaper
  error. Self-targeting and off by default, so every shipped pair set is unchanged.

  A `mccarthy` register, which names the punctuation deliberately: the eval drives the
  base control arm with this same prompt, so tics left out of it are a surface trick
  only the adapter can perform, and delta_cb is a character-bigram measure.

  drop_leading_heading now also consumes Blood Meridian's dash-separated chapter
  arguments — 131 paragraphs, 0 in every other work of all three corpora.

And a RUNBOOK, because the D1→D3 session recorded nothing and the chain had to be
recovered by rebuilding candidates and matching sha256 against the artifacts on disk.
This commit is contained in:
vh
2026-09-17 11:37:33 -07:00
parent 4dce0d0a43
commit c55966433f
5 changed files with 452 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/mccarthy-corpus/run-pairs.sh
Executable
+20
View File
@@ -0,0 +1,20 @@
#!/bin/bash
# lv-mccarthy D4 — instruction pairs. Run on pfi-gx10 as infra-ops.
#
# Deviations from the Hemingway D4 recipe, each forced by a measurement:
# --register mccarthy new; it names McCarthy's punctuation deliberately, see REGISTERS
# --source-entities MANDATORY; Hemingway's pairs predate it
# --reflow-hard-wraps DEFECT 4; The Road is hard-wrapped and the other 5 works are not
# --drop-leading-heading now also drops Blood Meridian's dash-separated chapter arguments
# --n 3722 / --n 276 the full passage yield of each split, not a round number
set -e
cd ~/lv-mccarthy
mkdir -p pairs
S=scripts/yarros-corpus/build_sft_pairs.py
COMMON="--corpus corpus-renamed/copies --register mccarthy --model gen
--key-file $HOME/.config/litellm/all-agents-key
--source-entities corpus-clean/entities-final.json
--context-frac 1.0 --drop-leading-heading --reflow-hard-wraps --seed 4919"
python3 $S $COMMON --split train --n 3722 --out pairs/pairs-full.jsonl > pairs/train.log 2>&1
python3 $S $COMMON --split val --n 276 --out pairs/pairs-val.jsonl > pairs/val.log 2>&1
echo DONE > pairs/.complete