feat(r49): D1 corpus built and green — Charlotte Brontë, 680k words, 951k tokens
scripts/r49-corpus/{build_corpus,verify_corpus}.py; corpus staged at
gx10:~/r49-corpus/. Catalogue ids verified against gutenberg.org's own search
rather than recalled. Charlotte only -- the Bell poems are co-authored and the
Gaskell biography is a different hand, so neither belongs in a single-voice corpus.
Jane Eyre 1260 · Villette 9182 · Shirley 30486 · The Professor 1028
680,291 words · 142 chapters · 950,974 Qwen3 tokens (1.40 tok/word)
alphabet 75 letters, 23 non-ASCII · round-trip lossless · 0 byte-fallback
All 11 acceptance checks pass, including both tokenizer legs run against the pilot
carrier itself. With a real denominator the projections tighten: at 6 rename copies
x 3 epochs = 17.1M tokens, the 0.6B pilot is 1.98 h.
THE ALPHABET INVERTS THE YARROS RESULT. Brontë writes French constantly -- Villette
is set in a French-speaking city, Jane Eyre has Adèle, The Professor is set in
Brussels -- so the corpus carries é 432, è 237, à 93, ê 79, ô 48 plus œ and æ. F02
measured Yarros at 0.0002% non-ASCII and derived an ASCII-fold for the name pool.
Under F02's own subset rule the Brontë pool may keep FRENCH accents and must still
exclude the Czech/Latvian/Slovak/Hungarian marks that never appear here. The fold is
per-work, and this is the first corpus where deriving it changes the answer.
Typography was inconsistent across works and it was the transcriber, not the author:
Shirley uses straight quotes and `--` with zero em-dashes while Jane Eyre and
Villette use curly and em-dash. Normalised toward what the text means.
Three defects, each found by running something rather than reasoning about it:
`Produced by` matched Brontë's own prose four times, which is the adjective-"minor"
shape again and is fixed by anchoring boilerplate patterns to line start; asserting
open/close quote counts must be equal is wrong, because 19th-century multi-paragraph
speech legitimately runs a surplus of opens, so the real error signature is that no
paragraph may begin with a closing quote; and The Professor's table of contents puts
two chapter names per line, so a bare regex returns 38 headings for a 25-chapter
novel and a minimum-gap filter still leaks its tail -- the rule that works is that
the body's "CHAPTER I" is the last one in the file.
Records the operator's pilot ruling: trial on Qwen3-0.6B-Base first, move up only if
it produces something useful.
This commit is contained in:
+26
-1
@@ -1,6 +1,6 @@
|
||||
# Persistent memory — eshpfi-management
|
||||
|
||||
_Last updated: 2026-09-10 06:30 PT (**Pfish-6** = run-6 NVFP4 is the standing seat, ana-ml2 :8021 ONLY; run 7 PURGED ~139 GiB; pfi-gx10 is an experimental/TRAINING box and carries no serving seat; all five ERP adapters now MIRRORED to ana-ml2 and inside restic; **BabyBronte / R49 author-voice regime is in PREP on gx10, carrier SETTLED = dense Qwen3**; checkpoints AND superseded merges PURGED both boxes, **~573 GB total**; only merged-run06 + the v6 quant survive)_
|
||||
_Last updated: 2026-09-10 07:05 PT (**Pfish-6** = run-6 NVFP4 is the standing seat, ana-ml2 :8021 ONLY; run 7 PURGED ~139 GiB; pfi-gx10 is an experimental/TRAINING box and carries no serving seat; all five ERP adapters now MIRRORED to ana-ml2 and inside restic; **BabyBronte / R49 author-voice regime is in PREP on gx10, carrier SETTLED = dense Qwen3**; checkpoints AND superseded merges PURGED both boxes, **~573 GB total**; only merged-run06 + the v6 quant survive)_
|
||||
|
||||
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
|
||||
> `Written:` stamp is under an hour old, read it (it carries the in-flight
|
||||
@@ -191,6 +191,31 @@ preserved verbatim in `archival-memory.md` § Superseded in-flight snapshots._
|
||||
(`finish_reason: stop`, correct text), container `Up 4 hours (healthy)`.
|
||||
⚠ `gx10:~/erp-tune/relaunch-trial-seat.sh` names a now-deleted model; NOT removed (its flags carry
|
||||
the FlashInfer JIT/PATH trap + gpu-clear/never-pkill notes) but banner-marked RETIRED.
|
||||
- **✅ R49 D1 CORPUS BUILT AND GREEN 2026-09-10.** `gx10:~/r49-corpus/`, instruments at
|
||||
`scripts/r49-corpus/{build_corpus,verify_corpus}.py`. Charlotte only (ids verified against
|
||||
gutenberg.org's own search): Jane Eyre 1260, Villette 9182, Shirley 30486, The Professor 1028 —
|
||||
**680,291 words · 142 chapters · 950,974 Qwen3 tokens** (1.40 tok/word). All 11 acceptance checks
|
||||
pass incl. lossless round-trip and **0 byte-fallback** on the pilot carrier's tokenizer.
|
||||
Real projection at 6 copies × 3 epochs = 17.1M tokens: **0.6B 1.98 h**, 1.7B 3.36 h, 4B 6.63 h.
|
||||
⚠⚠ **THE ALPHABET INVERTS THE YARROS RESULT — tell anyone re-pointing the name pool.** Brontë's
|
||||
inventory is 75 letters, **23 non-ASCII**: `ÆÉÊËÔàâäæçèéêëîïôöùûüŒœ` (é 432, è 237, à 93, ê 79,
|
||||
ô 48, + œ/æ). She writes French constantly — Villette, Adèle, Brussels. F02 measured Yarros at
|
||||
0.0002% non-ASCII and derived an ASCII-fold; under F02's OWN subset rule the **Brontë pool may keep
|
||||
FRENCH accents and must still exclude Czech/Latvian/Slovak/Hungarian marks.** The fold is per-work,
|
||||
and this is the first corpus where deriving it changes the answer.
|
||||
⚠ **Typography was inconsistent and it was the TRANSCRIBER, not the author**: Shirley = straight
|
||||
quotes + `--` + ZERO em-dashes; Jane Eyre/Villette = curly + em-dash; The Professor = curly + `--`.
|
||||
Normalised toward meaning (`--` → em dash, straight → curly paired per paragraph).
|
||||
⚠ **Three defects, all found by running things rather than reasoning:** (a) `Produced by` matched
|
||||
Brontë's OWN PROSE 4× ("a chilling effect produced by his steady announcement") — the adjective-
|
||||
`minor` shape again, fixed by anchoring boilerplate patterns to line start; (b) asserting
|
||||
open/close quote counts must be EQUAL is wrong — 19th-c multi-paragraph speech legitimately runs a
|
||||
surplus of opens (+46/+49/+51 on untouched works), so the real signature is *no paragraph begins
|
||||
with a closing quote* (0 of 14,230); (c) The Professor's TOC puts TWO chapter names per line, so a
|
||||
bare regex returns 38 headings for a 25-chapter novel AND a min-gap filter still leaks the tail —
|
||||
the rule that works is that the BODY's "CHAPTER I" is the LAST one in the file.
|
||||
- **⭐ PILOT RULING (operator, 2026-09-10): trial on `Qwen3-0.6B-Base` FIRST, move up only if useful.**
|
||||
Not a three-arm sweep up front — one ~2 h run, judged, then a decision.
|
||||
- **🖋 BabyBronte / R49 author-voice LoRA regime — IN PREP on pfi-gx10, nothing training.** Plan +
|
||||
every measured number: [`docs/pfi/author-voice-lora-regime.md`](docs/pfi/author-voice-lora-regime.md).
|
||||
Research target is **brokkr-smithy R49** (`research/R49-author-voice-adapters/`) — brokkr owns
|
||||
|
||||
Reference in New Issue
Block a user