2 Commits

Include renames

Author	SHA1	Message	Date
vh	5ddb0472e4	lv-mccarthy D3 on gx10: leak gate PASSED, and the val split is now bigger than Hemingway's ... ~/lv-mccarthy on pfi-gx10: corpus-clean, corpus-renamed (6 copies, 1,002 records), scripts. leak gate 0 of 75 renameable and 0 of 37 sub-threshold survive in any copy positive control 108/108 surfaces found in the unrenamed source negative control nonce absent from both trees THREE McCARTHY-SPECIFIC DECISIONS, each forced by a measurement. 1. --scope corpus, NOT the default per-work map. The Border Trilogy shares characters across books -- 9 surfaces appear in more than one work, including Parham (The Crossing + Cities of the Plain), Grady and Cole (All the Pretty Horses + Cities of the Plain), Socorro and Héctor. A per-work map would give John Grady a different invented name in each novel, turning one character into two. 2. A NEW `mccarthy` rename preset rather than reusing `hemingway`. Both are Spanish-inflected, but Hemingway's romance pool carries it_IT and fr_FR for his Italian and French casts, and McCarthy writes neither language -- drawing from it would drop Italian and French surnames into a Texas-Mexico border novel. en_GB goes for the same reason. en_US + es_MX/es_ES at an even share. 3. --min-cap 5 to MATCH the entity map's floor. The first gate run FAILED with 45 survivors, and the diagnosis is the Brontë lesson exactly: entities.py admits cap >= 5 while rename.py only renamed cap >= 8, so every entity between 5 and 7 sat in the map, was never renamed, and was counted as a leak. Hemingway never hit it because its map had sub_threshold_total 0. ⭐ --holdout-chapter NOW TAKES A LIST, and this is the change with the most downstream effect. The val split is one chapter index per work, so its SIZE is set by how many WORKS a corpus has, not how many words: Hemingway 10 works -> 9 val units -> 36,563 words/copy -> gate DECISIVE Brontë 4 works -> 4 val units -> 17,043 words/copy -> gate MARGINAL McCarthy 6 works -> 6 val units -> ~18,000 would have been Brontë's end of that Holding out chapters 7 AND 17 gives 11 units and 40,653 words per copy -- larger than Hemingway's, at a cost of 7% of the corpus -- on a corpus 40% smaller than his. No amount of corpus size fixes a val split that scales with work count. THE HUMAN GENDER PASS IS NOW AN AUDITABLE FILE, not a hand edit. The honorific/window resolver scored 21 correct / 3 held / 1 WRONG against a 26-name control; the base-rate proximity resolver built for Hemingway scored 18/6/1 and its own guard correctly REFUSED to write. So the incumbent stands and four entries are fixed by hand in gender_overrides_mccarthy.json, each carrying its evidence. ⚠ All four are female and all four look male-dominated in raw pronoun counts, because this corpus runs 29,144 male pronouns to 5,036 female -- a base rate of 85.3% male. Carla Jean Moss at 31m/21f would be 44m/8f at that base rate, so 21 female against an expected 8 is decisive. Same arithmetic that recovered Pilar and Brett on Hemingway. Alfonsa was in my control set and is correctly absent from the map at 4 occurrences, below the min-count floor -- an error in the control, not the pipeline. apply_gender_overrides.py refuses two ways: a name absent from the map is an error rather than a silent no-op, and overruling a gender the detector already holds needs an explicit "correcting": true so it cannot look like filling a held entity in a diff.	2026-09-17 08:59:00 -07:00
vh	5aa10bf138	lv-mccarthy D2: entity map + stoplist, both audits green — and audit_stoplist was scanning its own rationale ... Entity map at ~/mccarthy-corpus/entities.json. 123 surfaces after a 107-surface stoplist. entities.py 27/27 controls -- 19 positive (Glanton, Toadvine, Rawlins, Blevins, Alejandra, Chigurh, Moss, Bell, Boyd, Holden, Tobin, Magdalena, Eduardo, Parham, Socorro, Webster, Redbo, Niño, Franklin) and 8 negative audit_stoplist PASS -- no stoplisted surface is ever addressed as a person audit_entity_map PASS -- positive `boy` 0.89, negative band tops out at Riddle 0.17, all 5 remaining flags on the read-and-cleared list ⚠⚠ A DEFECT IN audit_stoplist.py ITSELF, latent for every corpus before this one. It built its surface set from every list value in the stoplist JSON -- including `_why`, which by convention is a LIST OF PROSE LINES. Every sentence of the rationale went into the matcher, and the empty separator line matched the honorific pattern 139 times, printing a flag with no surface name at the top of the report, above the one real catch. It now skips `_`-prefixed metadata keys and empty strings. THE ONE REAL CATCH WAS A CONTRADICTION INSIDE MY OWN FILE. `Franklin` sat in the geography list because it is the old name for El Paso, while the same file's context note recorded 'I'm here to see Mr Franklin' -- a lawyer in All the Pretty Horses. The honorific audit found the contradiction between the two halves of the file. Franklin is now renameable. A SECOND SELF-INFLICTED ONE: the fragments list was a speculative A-Z, which stoplisted `I` and `A` -- ordinary English words -- and `Sir I dont think I can do that` duly tripped the honorific audit. It is now the four letters actually MEASURED as entities (E, H, T, K). Stoplist what the entity map produced, not the alphabet. Everything ambiguous was read in context before placement, and the reasoning is in the file: Socorro is the ranch COOK in Cities of the Plain, not the New Mexico town -- renameable Webster, Jackson, Harlan, Lamar are Glanton's men and lawmen, not places -- renameable Niño, Keno, Redbo are HORSES, the author's inventions -- renameable, the `Inglés` precedent Mangas, Travis, Venada, Moderno are genuinely dual-use -- renamed, the safe direction Santa, Varas, Griffin, Eagle, Avenue, Calle, Terrell are real geography -- stoplisted Yaqui and Gilenos are real peoples; Ford and Hashknives are a brand and a real outfit Ed (Ed Tom Bell) and JC are short but are names, read and kept renameable Sensitivity floor, stated because it is part of the result: the top 170 of 199 surfaces were classified. The bottom 29 were not individually read, so a rare real-world referent may be renamed -- the safe direction, an accepted cost, not an oversight.	2026-09-17 08:43:46 -07:00