870f42960e7088322fedf220a874033c806eeeeb
16 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 vh
|
5558d9c7d3 |
R49/BabyYarros: voice + beat evaluation tooling and the seat-free delta_cb
Adds the Yarros-side evaluation the training exists to justify: does the adapter move arbitrary prose toward Yarros, and can the instruct arm still expand a beat to a paragraph on direction. Yarros-flavoured voice prompts (modern/neutral/ romantasy tiers so any Yarros voice in the modern tier is adapter-attributable, not prompt-supplied) and a Yarros-register beat SYS on the chat generator. voice_distance.py is the honest slice of adjudication that needs no seat: Burrows's Delta over character bigrams against held-out Yarros. Its first cut mis-framed the noise floor — it used the same-author distance (held-out vs itself) as the between-arm significance threshold, which is the target, not the threshold. Fixed to the measured floor: the within-arm seed spread, which is this metric's sampling variance at this sample size, computed from the two seeds already generated rather than assumed. Result on the built corpus, ordering base-125-tuned < instruct-tuned < base-unadapted, both adapters clearing the 0.046 measured floor (base +0.157, instruct +0.076), and the ordering corroborating the independent held-out loss ordering (Base below Instruct). One seed-pair per arm, so it corroborates rather than settles; the full frozen adjudication still needs a romantasy control panel, a second seed, and the gen seat for the beat-incumbent leg. |
||
|
vh
|
7b0580dcbe |
BabyYarros: the leak gate passes, and it found three defects nobody was looking for
The gate is new. There was no committed instrument for "does any of the author's own proper nouns survive the rename" -- the Brontë number was produced by hand -- so leak_gate.py is now that instrument, and it runs both directions every time: the same scan over the unrenamed source as a positive control, and a nonce string as a negative one. A detector that only ever sees renamed text cannot distinguish absent from blind. Run against BabyYarros as built it reported 212 surviving entities, not the 86 recorded earlier, because it scans the whole corpus rather than each work separately and it counts the sub-threshold entities rename never looked at. Three findings came out of closing that. The corpus had a typography defect of its own. The D1 notes correctly say no unwrap was needed; a different defect was there instead. The Empyrean books set their chapter epigraphs in small caps and the extractor rendered the run as uppercase while leaving the large initial as a separate token, so the corpus carried "M AJOR A FENDRA'S G UIDE TO THE R IDERS Q UADRANT" -- 106 lines, ~700 splits -- plus 52 drop caps like "T he flight field". That is where the entities called IDERS, UADRANT, NAUTHORIZED and seventeen bare single letters came from. A split initial next to an uppercased run is enough to recover the original mixed case, so the restore is exact rather than approximate: a word with a split initial was capitalised, an all-caps word without one was lowercase. Back matter was inside the prose. The builder splits on chapter headings and nothing follows the last one, so every work carried its acknowledgments, newsletter pitches and cover-artist credits -- 4,555 words naming the author's agent, editors and children, in a corpus whose entire purpose is that no identifiable name survives. And the gate passed at 0 of 314 while Afendra was still in every copy. The name never appears unpossessed, so it keyed as an apostrophe form, and rename and the gate both skip those as contractions -- unrenamed and unreported at once, which is the worst failure shape available. Baxter escaped a different way: wilder renders an in-book news article entirely in lowercase, putting the cap/lowercase ratio at 0.13 against a 0.05 bar. Then a second class the unigram scan structurally cannot see. Riders Quadrant, Flame Section, War Games and Fourth Wing -- the book's own title -- are built from ordinary words the detector correctly refuses to call names. The gate now audits recurring capitalised 2-3grams against an explicit allow list, and rename applies a phrase map after the entity pass. Every new detector flag is opt-in and off by default, and the Brontë entity map was re-derived after each change and confirmed identical in keys, surfaces and every field. The stoplist was built by reading each surface in context, which is why it is short: Violence is Xaden's nickname for Violet, and Continent, Presentation, Barrens, Originals, Montserrat, Athena, Aura, Curator and Sage are all in-world. A plausible-looking guess would have excluded most of them. Final: 0 of 325 entities and 0 of 91 audited phrases survive in any of 30 copy files, both controls passing. The sensitivity floor is stated in the gate's own output -- 3 occurrences for a name, 5 for a phrase -- because a negative without one is unfalsifiable. |
||
|
vh
|
6dba912324 |
BabyYarros: corpus built, gender resolution fixed, rename blocked on leak gate
Located the source: five Rebecca Yarros works in the Kvasir licensed library, with rights recorded as gated. Built D1 at 208 chapters and 780,744 words, which is 15% larger than the Brontë corpus. No unwrap step was needed because Kvasir's cleaner already emits flowing paragraphs, so the hard-wrap defect that cost a re-cut on Brontë does not exist here. The alphabet was re-derived rather than inherited: 23 non-ASCII letters across three forms, against F02's 4 on a smaller sample. Same ASCII-fold conclusion from a different measurement, which is the reason to re-derive per corpus. The interesting finding is a new pathology. In a rotating first-person POV corpus, every book's narrator gets the wrong gender. Measured against six names verified in the text, the pronoun resolver called Violet male, Leah male and Landon female -- three of eighteen wrong, and all three are the narrator of the book where they were misgendered. A narrator is "I" in her own book, so her name appears mostly inside the other lead's dialogue surrounded by his pronouns. This is Brontë's "Jane called male" amplified by rotating POV. Title-first resolution, which fixed it for Brontë, is nearly blind here because contemporary romance uses given names rather than honorifics. What works is the POV header: resolve each name from the chapters it does not narrate. Validated at 9 correct, 9 held, 0 wrong against the previous 7, 8 and 3 wrong, and the instrument refuses to write unless it beats what it replaces. Re-pointing rename.py surfaced three bugs, two of which would have silently corrupted the corpus. Gender came only from honorifics and the entities file's gender field was ignored, so the POV fix had no effect until wired through; that took wilder from 1 gendered entity to 13. The pool labels were hardcoded in a print statement, so any non-Brontë preset crashed. And the collision-filter log claimed it dropped names colliding with Brontë entities regardless of which corpus it filtered against -- the logic was right but the message named the wrong corpus, which is how a reader later concludes the filter ran on the wrong thing. D3 is blocked and nothing has been trained. The leak gate shows 86 of 232 renameable source entities surviving where the Brontë run reached 0 of 203. It decomposes into detector false positives that need a stopword filter rather than renaming, genuine misses among worldbuilding proper nouns, and a third class whose cause is not yet established. Training before the gate passes means fitting in-copyright text with 86 identifiable source entities intact, in a corpus F02 already flagged as small enough for leak to be a real concern. |
||
|
vh
|
e15c5ee5ea |
Instruct probe: voice and instruction-following coexist
Trained the same corpus onto Qwen3-4B instruct rather than -Base, with seed, steps and token count held so the carrier is the only variable. The chain picked checkpoint-150 by loss automatically, applying the lesson that cost a re-cut on 4B-Base. The central risk did not materialise. The assistant prior did not block the voice: curly quotes land at 16 of 18, identical to the 4B-Base tuned arm, against 1 of 18 on the unadapted control, and task-leak is 0 of 18 where the base carrier leaked 4. Instruction-following also survived raw-text training -- 10 of 10 on-beat through the chat template, the same as the untuned control. The cost is length discipline rather than comprehension. In-band dropped from 10 of 10 to 6 of 10 and the median went from 124 to 140 words. Training on Victorian prose made it wordier, which is a soft degradation and not a break. Held-out sits at 2.908 against 4B-Base's 2.814, and it plateaus without turning where the base carrier overfit at step 75. The assistant prior competes for capacity, so the instruct carrier absorbs less rather than overfitting more. What raw-continuation training does not fix is the plot furniture. The tuned instruct arm renders the beat and then drags the referent -- "He licked her clean... my master thus, my husband thus", turning the dog into a man, because the corpus is about masters and husbands. Another beat ran to 247 words and gave the narrator a list of duties. That is precisely what instruction-pair training addresses, since pairs teach render-this-and-stop where continuation teaches keep-writing. The probe de-risks the instruction-pair path without substituting for it. One metric note against future misreading: ran_on reports 10 of 10 on both arms and is uninformative on this job, because a single paragraph contains no blank line for it to find. |
||
|
vh
|
d9cf486004 |
Settle the Skaldsong architecture question with a ten-format negative result
Skaldsong's requirement is beat-sentence in, paragraph out, stitchable. That is narrower than free-form continuation and it fails differently, so it was tested directly rather than assumed from the earlier prompt work. The adapted 4B carrier cannot do it. Ten prompt formats across three seeds each -- bare, paragraph-break, labelled, epigraph, one-shot, one-shot unlabelled, three-shot, "it happened in this way", "I remember the whole of it", and the letter prompt's label-and-begin move applied to a beat -- produced thirty samples and not one that reliably renders the beat. They drift, they frame, or they truncate. The cause is structural rather than a shortfall of the adapter: "write a paragraph about this sentence" is an instruction, and a completion model has no mechanism for "about". It continues the text it is handed. Two formats failed worse than drifting and leaked pretraining task data. The paragraph-break format emitted an NLI multiple-choice item, and label-begin emitted a grammar-correction exercise. A standalone sentence followed by a blank line looks exactly like a dataset entry, and style adaptation does not remove base-model task artifacts. The same five beats through the gen seat with a style instruction and no adapter landed all ten samples inside the requested 90 to 140 word band, every one on its beat, with no drift into a following scene. The voice, however, is generic literary pastiche -- abstract-noun-heavy, fond of aphoristic openers, and over-written -- rather than Brontë. Voice without direction against direction without voice. The product needs both, which makes rebuilding the corpus as instruction-response pairs on an instruct carrier the evidence-backed option rather than more prompt cleverness. This applies to Yarros identically, because the carrier question is orthogonal to the author, so the next corpus should not re-run this experiment. Stitching also has its own failure mode, shown in the booth's third panel: the paragraphs were generated independently and the point of view slides between them, so by the fourth beat the narrator is both watching the girl carry the animals and carrying them herself. A real stitcher has to feed prior paragraphs back as context, which means the instruction-pair corpus wants multi-paragraph continuity examples and not only isolated beat-to-paragraph pairs. |
||
|
vh
|
8787daf04f |
Find the prompt recipe that works on a completion carrier
Two operator prompts settled the question the handoff prompt opened. The winner labels the artifact and then begins it: "This is the letter I wrote verbatim, my two short paragraphs, detailing the time I saw the mangy gray dog meet and then lovingly and tenderly lick a calico kitten: Auntie, You'll never believe what I saw--". Two of three seeds delivered the actual event in first person, and one is the strongest output of the sweep -- the dog following her to the mill, the kitten caught in a hedge, the dog carrying it out and laying it in her lap before licking it. Coherent arc, correct subject, no gloom-override, no meta-frame. It also reproduced the Gutenberg underscore-italics convention. It works where the handoff failed for a specific reason. The handoff could be satisfied by narrating compliance, because the letter did not yet exist. Naming the artifact and starting it leaves nothing to narrate around, so the only continuation is the artifact. The chapter-heading prompt refuted a hypothesis of mine. I predicted that rendering the title in the corpus's own conventions would make it land harder than the operator's inline form. It did the opposite: both corpus-form seeds ignored the title and opened unrelated scenes, while the inline form at least finished the heading and wrote a chapter about the story. Corpus chapter titles are short and decorative, so a long descriptive one in that slot reads as decoration to skip. A label only instructs if the model treats that slot as load-bearing. That prompt also surfaced an unnoticed consequence of the D2/D3 rename pipeline: given "Alice", the adapter produced "Alexander the Alexander, as he was known in Little London". The corpus was entity-renamed from a French/English pool, so the adapter learned that character names come from that pool and rewrites outside names into it. Callers cannot reliably name their own characters at prompt time. That is not a defect of the rename, which exists to prevent memorising Brontë's cast, but it is a usability constraint worth stating. |
||
|
vh
|
e47295d2ce |
Re-cut the 4B arms from checkpoint-75 and test an in-fiction instruction
The 4B tuned arms now come from checkpoint-75, the actual loss minimum at 2.813826 confirmed from loss-series.json rather than from reading the training log. Only the tuned arms needed re-cutting; the base arm never touches the adapter. One surprise in the comparison: step-75 and end-of-run differ on typography rather than voice. Curly quotes read 16 of 18 against 17, collapse is 0 either way, but the hard-wrap ratio is 0.33 at step-75 against 0.12 at the end. Further training washes the residual line-break habit out while held-out loss gets worse, so best loss and best typography are different checkpoints. Neither is near the original 0.85 defect and the corpus's own residual from preserved verse is 0.25. The operator's second prompt embeds the instruction inside the fiction: Abernathy tells the tale badly and then asks the narrator to retell it in a few short paragraphs. Across six seeds the model acknowledged the handoff every time and never once performed it. It wrote "I told it, briefly, to his satisfaction", and "So I wrote it out, and kept it in my pocket-book", and one seed negotiated the brief in character, objecting that it could not condense the story into a few short paragraphs because there were too many points to touch. The reason is structural rather than a failure of the adapter. In a novel, "she retold the story" is an ordinary sentence, so the likeliest continuation of a request is narration of compliance rather than compliance. A trailing paragraph break did shift the behaviour -- one seed opened in the narrator's own quoted speech -- so typography is a real lever, just not a sufficient one. That is direct evidence on the instruct question raised earlier: if the product is asking for a scene and getting the scene, no amount of in-fiction framing substitutes for a post-trained instruction-follower, which favours rebuilding the corpus as instruction pairs over more prompt cleverness. |
||
|
vh
|
25eb65e23b |
Complete the R49 carrier sweep: 4B closes the continuity gap and overfits
The three rungs now sit on the same unwrapped corpus, seed, step count and token count, so carrier size is the only variable. Held-out loss reads 3.329 at 0.6B, 3.018 at 1.7B and 2.814 at 4B -- deltas of 0.311 and then 0.204, diminishing but still real. 4B answers the question the rung existed for. Scene-level continuity holds: on the office-kitchen prompt it produces a named character with motivated dialogue, a spatial layout the narrator navigates, and a physical description, all in one passage, where 1.7B wrote pretty but eventless prose about opening doors and looking at stars. On the letter prompt it opens the letter, promises to quote it, and then quotes it across a paragraph break. Voice saturation is also the best of any rung: curly quotes 17 of 18 against its own base arm's 1 of 18, and collapse 0 of 18 against 4 of 18. Two findings that change earlier conclusions. 4B is the first rung to overfit inside one epoch. Its series runs 2.832, 2.816, 2.814, 2.820, 2.824, 2.825, 2.825 -- a minimum around step 75 and then a turn. Both smaller rungs plateaued without turning, so the optimal epoch count shrinks as the carrier grows and my earlier "one epoch is right for this corpus" holds only for the small end. The consequence is operational: the adapter directory holds the end-of-run weights at 2.825 rather than the step-75 best at 2.814, and it exists as a recoverable checkpoint only because save_steps was set. The voice arms were cut from the end-of-run adapter, so the booth understates 4B slightly. The tone-override also appears to close. On the operator's frame prompt asking for a wonderful story, 1.7B held the frame on every seed but killed the animals on two of four; 4B kept them alive on both seeds, and one of them had the narrator doubt the story he was told and supply a parallel childhood memory to explain the doubt. That is a narrator with an interior position on the tale. Two samples per arm, so directionally right rather than established. |
||
|
vh
|
4cee4b179d |
Launch the 4B rung and record that voice transfer carries subject matter
Rung 3 of the sweep is training: Qwen3-4B-Base, one epoch, seed 4919, the same unwrapped corpus, 159 steps at roughly 37.8 s/it for about 100 minutes. Carrier size remains the only variable across the three rungs. A two-arm voice test is chained behind it, gated on the adapter existing, so the rung is readable when it lands rather than needing a second round trip to become informative. The finding worth keeping came from an operator prompt rather than from my test set. Asked for a wonderful story about a stray dog licking a calico kitten, framed as reported speech, the 1.7B adapter held the frame across all four seeds and was unmistakably Victorian throughout -- and two of the four turned the premise into animal death, with the cat licking the puppy to death and the narrator's interlocutor answering that he wished they were all dead. That is not a coherence failure. It is Charlotte Brontë's preoccupations arriving alongside her sentences: Jane Eyre opens on a beaten child, Helen Burns dies, and Villette is saturated with grief. Train 680k words of that into a small carrier and a request for something wholesome reaches for mortality. My nine test prompts were all emotionally neutral and structurally could not have surfaced this. The operator's prompt found it on the first try, which is worth recording as a property of the test set rather than of the adapter. The implication for the regime is that "voice transfer" includes tone and subject, so wanting the voice without the gloom is a corpus-selection or prompt-framing problem rather than a training-length one. Also observed: one seed closed its anecdote and began "CHAPTER XIX. THE CHILD'S CLOSET", having learned book structure unprompted. The base control on the same prompt went modern and essayistic, so the shift is the adapter rather than the carrier. |
||
|
vh
|
9e18fd58ec |
Complete R49 rung 2 and booth the three-way voice comparison
Both rungs now sit on the same unwrapped corpus with seed, steps and token count held, so carrier size is the only difference and the effect is attributable: held-out 3.329 at 0.6B against 3.018 at 1.7B, a gap of 0.311 nats. The chained 0.6B rerun closed the confound the unwrap opened. Two things in those numbers need stating or they will be misread. First, the original wrapped-corpus 0.6B reached 3.172, which looks better than the unwrapped 0.6B's 3.329 and is not. Different corpus means a different held-out set, and the wrapped version's 5.7% newline tokens are near-deterministic after a 70-character line, so they deflate the loss with cheap wins. Removing them removed the easy tokens. It is a measurement artifact, not a regression. Second, a correction to my own earlier claim: I twice described the 0.6B run as still descending and undertrained at 3.172. Its series reads 3.176, 3.173, 3.172, 3.172 -- it flattened. All three runs plateau, so one epoch is about right for this corpus rather than short. The three-way booth puts 1.7B base, 1.7B tuned and 0.6B tuned side by side on the same nine prompts and seeds. The base arm is the control that matters: curly quotes go 0 of 18 on 1.7B base to 15 of 18 on 1.7B tuned, and worksheet-or- explainer collapse goes 3 of 18 to 0 of 18, so the shift is the adapter rather than the larger carrier. Hard-wrapping fell from 0.85 to 0.18, confirming the corpus unwrap carried through into the adapter. Sense partially returned. The 1.7B arm produces locally coherent sequential Victorian prose where the 0.6B produced word salad, but scene-level continuity still breaks mid-passage. One observation held loosely: curly quotes are slightly lower at 1.7B than 0.6B, which would fit a bigger model's stronger priors resisting the adapter at the same rank. That is untested and is not offered as established. |
||
|
vh
|
935b35ac2e |
Unwrap the Brontë corpus and launch the 1.7B rung
Operator: "start the 1.7b training." The 0.6B adapter learned the Gutenberg transcription's ~70-character line breaks along with the prose -- its output wrapped at a mid-length-line ratio of 0.85 against the base model's 0.00. That is typography rather than style, and every further rung would have inherited it, so the corpus is reflowed before rung 2 rather than after the sweep. The reflow joins 57,430 of 85,380 paragraph blocks and keeps 27,950. Verse is the hazard a blind join would destroy, so the decision is per block by median line length: blocks whose lines cluster near the wrap width are flowed prose, blocks of consistently short lines keep their breaks. Every kept multi-line block in the sample was genuinely verse with its lineation intact. No line ended in a lone hyphen, so the space-join could not split a word across lines. The acceptance check is content identity -- " ".join(text.split()) byte-identical before and after -- and it passed on all 852 records, proving only whitespace changed. Concrete cost of the old defect: 5.7% of the training budget was newline tokens. The same words pack to 5,210,112 tokens unwrapped against 5,525,504 wrapped. The 1.7B run is live at 159 steps and roughly 18.7 s/it. Everything but the carrier and the corpus is held from the 0.6B run: seed 4919, rank 32, lr 1e-4, seq 4096, batch 1 by accum 8, one epoch, eval and save every 25 steps so the minimum is located rather than assumed. That corpus change is a second variable and it is named as one. A 0.6B-vs-1.7B comparison is descriptive, not attributable, until the chained 0.6B rerun on the same unwrapped corpus lands behind it -- gated on the 1.7B actually producing an adapter, because a chain that fires on failure turns one lost run into two. "Did sense come back at 1.7B" is a within-arm reading and survives the confound; any between-rung delta does not. The original wrapped corpus is untouched, so the 0.6B run's pinned corpus sha 3959036cf851bf62 stays reproducible. |
||
|
vh
|
b8dbe71a1c |
memory: snapshot — R49 1-epoch pilot complete and awaiting adjudication; MeroMero A4B quantized, v2 blocked; althing 3.6.2 rolled
Ships the two pending code changes alongside the memory that describes them: train_voice_lora.py gains --eval-steps/--save-steps (the 3-epoch pilot overfit with per-epoch eval and save_strategy="no", so the minimum was neither visible nor recoverable), and the althing post-office compose is pinned to 3.6.2. Index rewritten: in-flight compressed from 176 lines to the live state, three detail files added for R49 D1-D3, the MeroMero four-failure chain, and the althing rollout. Seven closed pre-08-27 entries archived; the guards held back the rest because they carry open deferred pointers. |
||
|
vh
|
20bbb95113 |
fix(r49): loss-series collector silently dropped every eval record
The LossLog callback filtered on `"loss" in logs`. Trainer emits eval under `eval_loss` with no `loss` key, so every eval record was discarded and loss-series.json showed zero eval points while the log a metre away carried `eval_loss: 3.198`. An artefact that omits data which demonstrably exists reads as "no eval was run" rather than "the collector dropped it", which is the failure mode that costs someone a re-run. Collector now accepts either key. Seed 2 gets it from launch; seed 1 is already running with the old code, so recover_eval_series.py parses the eval records back out of its log -- the data was printed, not lost -- and the chain runs that recovery before cutting the generation arms, so the artefact is complete before anything reads it. The two seeds therefore differ in logging code but not in training math: the callback only affects what is recorded, never what is computed, so the weight trajectories remain comparable. Noting it because a difference between the two arms whose spread sets the decision threshold is worth stating even when it is provably inert. |
||
|
vh
|
8fff722f2c |
feat(r49): incumbent arm generated against the concrete gen seat, not the stale H02 name
brokkr-smithy corrected H02's incumbent naming: qwen3.6-35-a3b-heretic was retired from the gateway roster on 2026-08-15 and is not what Skaldsong would call today. Verified against the gateway and the seat itself -- alias `gen` resolves to hosted_vllm/qwen3.8-27b-uncensored on ana-ml2:8015, container vllm-gen, 262,144 ctx. The arm targets that. 24 records, style-prompted on the same prompts and sampler as the other arms. Alias resolved at run start AND end and confirmed stable across the run, per the fleet rule that an artefact records the backing model rather than the alias. Two things recorded rather than glossed: The harness is NOT matched to the other arms and the artefact says so. Base and adapted arms are local transformers on gx10; the incumbent is a served NVFP4 27B reached over the gateway, and it is an instruct model receiving a style instruction where the others are base models receiving none. That asymmetry is the comparison H02 asks for -- prompted imitation against trained voice -- but it must not be reported as if the harnesses were identical. The gateway echoes the ALIAS in each response's `model` field, so a row read on its own would have recorded "gen" as provenance -- the same class of mistake that inflated an exposure count 4.7x on this fleet. Rows now carry alias_echoed_by_gateway beside backing_model_resolved and its date, and the generator was fixed at source rather than only in the emitted file. Sanity: median 392 completion tokens, zero records opening with markdown or meta-commentary, output reads as continuation prose. The style prompt was written to be a fair incumbent rather than a strawman, since this arm is what the adapter must beat. |
||
|
vh
|
375244ad05 |
feat(r49): D2/D3 complete and the H02 pilot is training on gx10
Entity resolution, deterministic rename augmentation, packing and the pilot trainer. Qwen3-0.6B-Base is training now: 507 steps, 11.2 s/it, ~1h35m. D2 -- gender resolution is TITLE-FIRST, and that is a change from F02's method rather than a port of it. F02 used pronoun proximity and recorded that it is structurally blind to the first-person narrator, whose name appears mainly in dialogue surrounded by other people's pronouns. Measured here, proximity called JANE MALE -- the narrator of Jane Eyre and the single worst entity to get wrong. Titles have no such blind spot: Miss Eyre, Mrs. Fairfax, Mr. Rochester, Madame Beck, M. Paul, and a 19th-century novel is saturated with them. Measured: 16 entities resolved, zero wrong, every ambiguous case landing on HELD -- shared family surnames like Helstone and Pelet genuinely belong to both a man and a woman and hold as they should. Held means ungendered, not unrenamed. A HELD entity is still renamed, from the gender-neutral surname pool, because the operator's Yarros directive was "rename all proper nouns" and holding a place leaks it -- Thornfield appears 100 times in Jane Eyre and is as author-specific as Riders Quadrant was. Substituting a neutral token makes no gender claim, so no gender claim can be wrong. D3 -- pool is French + English per the operator, weighted per work by setting: Brussels novels 60% French, Yorkshire novels 25%. Locales restricted to fr_FR/fr_BE/en_GB/en_IE; en_US and en_AU carry modern surnames that are wrong register for the 1840s. The pool is filtered against Brontë's own 75-letter alphabet, so French accents stay and Czech/Latvian marks do not. Two collision defects found by running the leak gate rather than trusting it: `Burns` and `Marie` were drawn as replacements while being Brontë characters -- F02's collision filter was built against Yarros and does not carry -- and then `Pierre-Yves` passed a whole-string filter while `Pierre` (Mademoiselle St. Pierre) is a Villette character. The filter now compares by COMPONENT. Final gate: 0 of 203 source entities survive in any of 24 copy-files. Trainer records what the run RESOLVED to rather than what it requested -- attention implementation, dtype, device, corpus sha and harness cleanliness are read back off the live objects. transformers 5.x has dropped warmup_ratio, caught by reading the signature after the first launch failed on it; the 3% warmup is computed into warmup_steps instead. |
||
|
vh
|
ba8dac2c80 |
feat(r49): D1 corpus built and green — Charlotte Brontë, 680k words, 951k tokens
scripts/r49-corpus/{build_corpus,verify_corpus}.py; corpus staged at
gx10:~/r49-corpus/. Catalogue ids verified against gutenberg.org's own search
rather than recalled. Charlotte only -- the Bell poems are co-authored and the
Gaskell biography is a different hand, so neither belongs in a single-voice corpus.
Jane Eyre 1260 · Villette 9182 · Shirley 30486 · The Professor 1028
680,291 words · 142 chapters · 950,974 Qwen3 tokens (1.40 tok/word)
alphabet 75 letters, 23 non-ASCII · round-trip lossless · 0 byte-fallback
All 11 acceptance checks pass, including both tokenizer legs run against the pilot
carrier itself. With a real denominator the projections tighten: at 6 rename copies
x 3 epochs = 17.1M tokens, the 0.6B pilot is 1.98 h.
THE ALPHABET INVERTS THE YARROS RESULT. Brontë writes French constantly -- Villette
is set in a French-speaking city, Jane Eyre has Adèle, The Professor is set in
Brussels -- so the corpus carries é 432, è 237, à 93, ê 79, ô 48 plus œ and æ. F02
measured Yarros at 0.0002% non-ASCII and derived an ASCII-fold for the name pool.
Under F02's own subset rule the Brontë pool may keep FRENCH accents and must still
exclude the Czech/Latvian/Slovak/Hungarian marks that never appear here. The fold is
per-work, and this is the first corpus where deriving it changes the answer.
Typography was inconsistent across works and it was the transcriber, not the author:
Shirley uses straight quotes and `--` with zero em-dashes while Jane Eyre and
Villette use curly and em-dash. Normalised toward what the text means.
Three defects, each found by running something rather than reasoning about it:
`Produced by` matched Brontë's own prose four times, which is the adjective-"minor"
shape again and is fixed by anchoring boilerplate patterns to line start; asserting
open/close quote counts must be equal is wrong, because 19th-century multi-paragraph
speech legitimately runs a surplus of opens, so the real error signature is that no
paragraph may begin with a closing quote; and The Professor's table of contents puts
two chapter names per line, so a bare regex returns 38 headings for a 25-chapter
novel and a minimum-gap filter still leaks its tail -- the rule that works is that
the body's "CHAPTER I" is the last one in the file.
Records the operator's pilot ruling: trial on Qwen3-0.6B-Base first, move up only if
it produces something useful.
|