2 Commits

Include renames

Author	SHA1	Message	Date
vh	0fa68cb465	lv-mccarthy D1 fix: three small-caps defects the entity map caught, and one I nearly added ... D2's entity map returned `E`, `H`, `T` and `K` as renameable entities with 17-33 capitalised occurrences each. A bare initial is never a name -- that is the `G` class from the Hemingway build, where `G` was about to be renamed to a surname 248 times. Reading them in context showed the McCarthy editions set section openings in small caps and the extractor mangled them three different ways, none of which the D1 build repaired: 1. SPLIT INITIAL `T HE HOUSE was built` -> `The house was built` 32 cases Hemingway's restore_smallcaps only fires on TWO or more split initials in a line, so it is structurally blind to these single ones. 2. UNMARKED RUN `THEY STOOD in the doorway` -> `They stood in the ...` 88 cases Concentrated in Cities of the Plain (49) and The Crossing (37). 3. LOST INITIAL `HE CANDLEFLAME` -> `THE CANDLEFLAME` 1 case Rule 1 requires a FOLLOWING all-caps word, because `A TV was playing` and `A Mexican was changing` are an article plus a capitalised word, not a drop cap. All four such probes verified untouched. Rule 2's `[a-z]` lookahead is what makes it safe: lowercasing every all-caps run at a block start would eat a genuine shout or a sign, and requiring the run to be followed immediately by a lowercase word means it is a sentence continuing. All 23 distinct first words of the 88 were checked and are real words -- HE, WHEN, THE, THEY, QUINQUAGESIMA -- except one, which was case 3. ⚠⚠ AND A SECOND LOST-INITIAL ENTRY WAS NEARLY SHIPPED THAT WOULD HAVE CORRUPTED THE TEXT. `HEY RODE` -> `THEY RODE` looked right from a survey of the BUILT corpus. The raw master has `THEY RODE` intact, twice: `HEY RODE` was matching as a SUBSTRING, and the unanchored replace produced `TTHEY RODE`, which rule 2 then lowercased to `Tthey rode`. Two things caught it -- the count assertion (expected 1, replaced 2) and then reading the master. Rule 3 is now a block-anchored regex rather than a string replace, so a substring cannot fire it. ⚠ My first corruption check also missed it, searching for `TTHEY` when the pipeline had already lowercased it to `Tthey`. Check the shape the pipeline actually emits, not the shape you imagined it would. Totals move 584,756 -> 584,716 words, 167 units unchanged. Both guards still pass: quote marks 0.0/10k, author's own name 26 -> 0. Entity map positive control is 14/14 on real McCarthy characters (Glanton, Toadvine, Rawlins, Blevins, Alejandra, Chigurh, Moss, Bell, Boyd, Holden, Tobin, Magdalena, Eduardo, Parham); `T` and `E` no longer appear as entities.	2026-09-17 08:40:31 -07:00
vh	f3bf3ca89c	lv-mccarthy D1: 167 units, 584,756 words, and a style that looks exactly like damage ... Six complete novels from the licensed Kvasir masters. Same record schema as the Brontë, Yarros and Hemingway builders, so entities.py, rename.py, leak_gate.py and the trainers run unchanged. Splits via the new shared split_units module. all-the-pretty-horses 33u 99,242w paragraph-blocks [back -1,768w] [drop cap restored] blood-meridian 23u 116,651w roman-numeral [back -354w] cities-of-the-plain 30u 90,166w paragraph-blocks no-country-for-old-men 13u 69,841w roman-numeral [back -463w] the-crossing 49u 149,985w paragraph-blocks [back -30w] the-road 19u 58,871w paragraph-blocks THE THING THIS BUILDER PROTECTS IS A VOICE THAT READS AS A DEFECT. McCarthy uses no quotation marks and drops the apostrophe from most contractions -- dont, aint, wont, didnt. Measured over the built corpus: 0.0 quote marks per 10k words against Hemingway's 838, and 123 apostrophes against his 241. repair_typography.py normalises "toward what the text does" and would put the quotes back, deleting the single most identifiable thing about the author before training starts. This builder runs NO typography normalisation and then ASSERTS the quote density, so a future well-meaning change fails the build instead of quietly undoing it. ⚠ That same property will make the voice gate easy to pass for the wrong reason. voice_distance.py is Burrows's Delta over character bigrams; an adapter that learns only "emit no quotation marks" moves delta_cb a long way without having learned a sentence. A punctuation-normalised secondary read needs pre-registering before this one is gated. Exclusions, measured rather than assumed: - two truncated catalogue rows dropped for their complete mobi siblings (Blood Meridian epub 1,167w, The Crossing epub 222w -- both real prose, both `accepted`) - nothing else. All 15 cross-work 8-gram containment pairs measured on the Hemingway precedent; worst is 0.10%. Six independent works, no subsumption. Back matter rides inside the last unit in four of six works and the marker differs every time -- THE END, a dumped Table of Contents, a Reader's Guide, an About-the-Author, press blurbs, a CIP page. It carried the author's own name 26 times across the raw masters. Both guards report and gate: name 26 -> 0, quotes 0.0/10k. ⚠⚠ The back-matter strip runs BEFORE the split here, inverting the Hemingway order. Blood Meridian and The Crossing end with a dumped table of contents made of bare roman numerals on their own lines -- the exact shape of a chapter marker. Splitting first feeds the TOC to the splitter as two dozen extra chapters; only the 150-word floor accidentally saves it today. One lost drop cap is patched by name, not by heuristic: the All the Pretty Horses epub opens `HE CANDLEFLAME` because the decorative T was an image the extractor dropped. A general restore-the-missing-initial rule would have to guess the letter, so this is asserted against the known string and fails loudly if the master ever changes. The alphabet is re-derived, not inherited: 1,411 non-ASCII letters across 14 forms (á é í ñ ó ú ü). The Border Trilogy is half set in Mexico, so the Yarros ASCII-only conclusion does not transfer -- same finding as Hemingway, same reason.	2026-09-17 08:17:57 -07:00