feat(r49): D1 corpus built and green — Charlotte Brontë, 680k words, 951k tokens

scripts/r49-corpus/{build_corpus,verify_corpus}.py; corpus staged at
gx10:~/r49-corpus/. Catalogue ids verified against gutenberg.org's own search
rather than recalled. Charlotte only -- the Bell poems are co-authored and the
Gaskell biography is a different hand, so neither belongs in a single-voice corpus.

  Jane Eyre 1260 · Villette 9182 · Shirley 30486 · The Professor 1028
  680,291 words · 142 chapters · 950,974 Qwen3 tokens (1.40 tok/word)
  alphabet 75 letters, 23 non-ASCII · round-trip lossless · 0 byte-fallback

All 11 acceptance checks pass, including both tokenizer legs run against the pilot
carrier itself. With a real denominator the projections tighten: at 6 rename copies
x 3 epochs = 17.1M tokens, the 0.6B pilot is 1.98 h.

THE ALPHABET INVERTS THE YARROS RESULT. Brontë writes French constantly -- Villette
is set in a French-speaking city, Jane Eyre has Adèle, The Professor is set in
Brussels -- so the corpus carries é 432, è 237, à 93, ê 79, ô 48 plus œ and æ. F02
measured Yarros at 0.0002% non-ASCII and derived an ASCII-fold for the name pool.
Under F02's own subset rule the Brontë pool may keep FRENCH accents and must still
exclude the Czech/Latvian/Slovak/Hungarian marks that never appear here. The fold is
per-work, and this is the first corpus where deriving it changes the answer.

Typography was inconsistent across works and it was the transcriber, not the author:
Shirley uses straight quotes and `--` with zero em-dashes while Jane Eyre and
Villette use curly and em-dash. Normalised toward what the text means.

Three defects, each found by running something rather than reasoning about it:
`Produced by` matched Brontë's own prose four times, which is the adjective-"minor"
shape again and is fixed by anchoring boilerplate patterns to line start; asserting
open/close quote counts must be equal is wrong, because 19th-century multi-paragraph
speech legitimately runs a surplus of opens, so the real error signature is that no
paragraph may begin with a closing quote; and The Professor's table of contents puts
two chapter names per line, so a bare regex returns 38 headings for a 25-chapter
novel and a minimum-gap filter still leaks its tail -- the rule that works is that
the body's "CHAPTER I" is the last one in the file.

Records the operator's pilot ruling: trial on Qwen3-0.6B-Base first, move up only if
it produces something useful.
This commit is contained in:
vh
2026-09-10 06:51:32 -07:00
parent 6cbc9c7a2c
commit ba8dac2c80
6 changed files with 644 additions and 4 deletions
+26 -1
View File
@@ -1,6 +1,6 @@
# Persistent memory — eshpfi-management
_Last updated: 2026-09-10 06:30 PT (**Pfish-6** = run-6 NVFP4 is the standing seat, ana-ml2 :8021 ONLY; run 7 PURGED ~139 GiB; pfi-gx10 is an experimental/TRAINING box and carries no serving seat; all five ERP adapters now MIRRORED to ana-ml2 and inside restic; **BabyBronte / R49 author-voice regime is in PREP on gx10, carrier SETTLED = dense Qwen3**; checkpoints AND superseded merges PURGED both boxes, **~573 GB total**; only merged-run06 + the v6 quant survive)_
_Last updated: 2026-09-10 07:05 PT (**Pfish-6** = run-6 NVFP4 is the standing seat, ana-ml2 :8021 ONLY; run 7 PURGED ~139 GiB; pfi-gx10 is an experimental/TRAINING box and carries no serving seat; all five ERP adapters now MIRRORED to ana-ml2 and inside restic; **BabyBronte / R49 author-voice regime is in PREP on gx10, carrier SETTLED = dense Qwen3**; checkpoints AND superseded merges PURGED both boxes, **~573 GB total**; only merged-run06 + the v6 quant survive)_
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
> `Written:` stamp is under an hour old, read it (it carries the in-flight
@@ -191,6 +191,31 @@ preserved verbatim in `archival-memory.md` § Superseded in-flight snapshots._
(`finish_reason: stop`, correct text), container `Up 4 hours (healthy)`.
⚠ `gx10:~/erp-tune/relaunch-trial-seat.sh` names a now-deleted model; NOT removed (its flags carry
the FlashInfer JIT/PATH trap + gpu-clear/never-pkill notes) but banner-marked RETIRED.
- **✅ R49 D1 CORPUS BUILT AND GREEN 2026-09-10.** `gx10:~/r49-corpus/`, instruments at
`scripts/r49-corpus/{build_corpus,verify_corpus}.py`. Charlotte only (ids verified against
gutenberg.org's own search): Jane Eyre 1260, Villette 9182, Shirley 30486, The Professor 1028 —
**680,291 words · 142 chapters · 950,974 Qwen3 tokens** (1.40 tok/word). All 11 acceptance checks
pass incl. lossless round-trip and **0 byte-fallback** on the pilot carrier's tokenizer.
Real projection at 6 copies × 3 epochs = 17.1M tokens: **0.6B 1.98 h**, 1.7B 3.36 h, 4B 6.63 h.
⚠⚠ **THE ALPHABET INVERTS THE YARROS RESULT — tell anyone re-pointing the name pool.** Brontë's
inventory is 75 letters, **23 non-ASCII**: `ÆÉÊËÔàâäæçèéêëîïôöùûüŒœ` (é 432, è 237, à 93, ê 79,
ô 48, + œ/æ). She writes French constantly — Villette, Adèle, Brussels. F02 measured Yarros at
0.0002% non-ASCII and derived an ASCII-fold; under F02's OWN subset rule the **Brontë pool may keep
FRENCH accents and must still exclude Czech/Latvian/Slovak/Hungarian marks.** The fold is per-work,
and this is the first corpus where deriving it changes the answer.
⚠ **Typography was inconsistent and it was the TRANSCRIBER, not the author**: Shirley = straight
quotes + `--` + ZERO em-dashes; Jane Eyre/Villette = curly + em-dash; The Professor = curly + `--`.
Normalised toward meaning (`--` → em dash, straight → curly paired per paragraph).
⚠ **Three defects, all found by running things rather than reasoning:** (a) `Produced by` matched
Brontë's OWN PROSE 4× ("a chilling effect produced by his steady announcement") — the adjective-
`minor` shape again, fixed by anchoring boilerplate patterns to line start; (b) asserting
open/close quote counts must be EQUAL is wrong — 19th-c multi-paragraph speech legitimately runs a
surplus of opens (+46/+49/+51 on untouched works), so the real signature is *no paragraph begins
with a closing quote* (0 of 14,230); (c) The Professor's TOC puts TWO chapter names per line, so a
bare regex returns 38 headings for a 25-chapter novel AND a min-gap filter still leaks the tail —
the rule that works is that the BODY's "CHAPTER I" is the LAST one in the file.
- **⭐ PILOT RULING (operator, 2026-09-10): trial on `Qwen3-0.6B-Base` FIRST, move up only if useful.**
Not a three-arm sweep up front — one ~2 h run, judged, then a decision.
- **🖋 BabyBronte / R49 author-voice LoRA regime — IN PREP on pfi-gx10, nothing training.** Plan +
every measured number: [`docs/pfi/author-voice-lora-regime.md`](docs/pfi/author-voice-lora-regime.md).
Research target is **brokkr-smithy R49** (`research/R49-author-voice-adapters/`) — brokkr owns