vh
c55966433f
`leak_gate.py` scans `\b(Surface)\b`. Any character inserted inside a name defeats
that pattern outright, so a mangled occurrence is unrenameable by rename.py AND
unreportable by the gate. lv-mccarthy's 2026-09-17 tree passed at "0 of 75
renameable and 0 of 37 sub-threshold" while carrying 13 occurrences of Bell,
Chigurh, Moss, Toadvine and Glanton in all six copies:
B ell C higurh M oss T oadvine a small-caps drop cap kept as its own token
Toad-vine Glan-ton a print line-break hyphen kept by the extractor
Every visible occurrence HAD been renamed, which is what made the residue invisible
to a spot-read. Fixed at three levels, all three of which must stay:
build_corpus_mccarthy.py rules 4 and 5 repair the source text — 32 split initials
with a lowercase remainder, 5 hyphen-split names, each with an expected count so a
master change fails the build. Rule 4's letter class is consonants only: `I` opens
1,966 paragraphs, `A` 143 and `Y` 32 (Spanish `y`); folding any would corrupt 2,141
lines to fix 32.
leak_gate.py gains a separator-tolerant pass with its own positive and negative
controls, and it FAILS the gate. Validated against the pre-fix tree: reports all
five surfaces, exits 1. Its fragment filter is what makes it usable — a naive scan
returns 18 false positives on Hemingway (`God damn`, `I run`) against 3 real ones;
requiring one fragment to be a non-word of the corpus cleared all 18 and kept all 3.
The whole D1→D3 chain is reproduced byte-identically before and after, so the fix
is the only delta: 6 works, the entity map, the final map and all 36 copy files.
Cross-checked on the shipped corpora: lv-bronte is clean of this class, lv-hemingway
carries 3 (`Primi tivo`, `Pasionar ia`, `Chi cote`) and is live on fv-ml1.
Also in build_sft_pairs.py, both needed before lv-mccarthy's pairs:
DEFECT 4, hard-wrap reflow. Measured on the SHIPPED lv-bronte adapter, which emits
mid-sentence line breaks at 12.46 per 1k chars against 0.00 for its own base control
and 0.00 for every Hemingway arm. McCarthy is the mixed case — The Road is wrapped,
the other five works are not — so the corpus teaches the break as a coin flip. The
obvious fix (join every interior newline) corrupts 46 two-speaker exchanges whose
blank line was lost, and unmarked dialogue is the one thing this adapter exists to
learn; the rule splits on sentence-final punctuation instead and takes the cheaper
error. Self-targeting and off by default, so every shipped pair set is unchanged.
A `mccarthy` register, which names the punctuation deliberately: the eval drives the
base control arm with this same prompt, so tics left out of it are a surface trick
only the adapter can perform, and delta_cb is a character-bigram measure.
drop_leading_heading now also consumes Blood Meridian's dash-separated chapter
arguments — 131 paragraphs, 0 in every other work of all three corpora.
And a RUNBOOK, because the D1→D3 session recorded nothing and the chain had to be
recovered by rebuilding candidates and matching sha256 against the artifacts on disk.
21 lines
1.1 KiB
Bash
Executable File
21 lines
1.1 KiB
Bash
Executable File
#!/bin/bash
|
|
# lv-mccarthy D4 — instruction pairs. Run on pfi-gx10 as infra-ops.
|
|
#
|
|
# Deviations from the Hemingway D4 recipe, each forced by a measurement:
|
|
# --register mccarthy new; it names McCarthy's punctuation deliberately, see REGISTERS
|
|
# --source-entities MANDATORY; Hemingway's pairs predate it
|
|
# --reflow-hard-wraps DEFECT 4; The Road is hard-wrapped and the other 5 works are not
|
|
# --drop-leading-heading now also drops Blood Meridian's dash-separated chapter arguments
|
|
# --n 3722 / --n 276 the full passage yield of each split, not a round number
|
|
set -e
|
|
cd ~/lv-mccarthy
|
|
mkdir -p pairs
|
|
S=scripts/yarros-corpus/build_sft_pairs.py
|
|
COMMON="--corpus corpus-renamed/copies --register mccarthy --model gen
|
|
--key-file $HOME/.config/litellm/all-agents-key
|
|
--source-entities corpus-clean/entities-final.json
|
|
--context-frac 1.0 --drop-leading-heading --reflow-hard-wraps --seed 4919"
|
|
python3 $S $COMMON --split train --n 3722 --out pairs/pairs-full.jsonl > pairs/train.log 2>&1
|
|
python3 $S $COMMON --split val --n 276 --out pairs/pairs-val.jsonl > pairs/val.log 2>&1
|
|
echo DONE > pairs/.complete
|