vh
1a5bc2ddf1
The v2 dense quant had failed four times. Attempt 5 lands it at 19 G. The blocker was not what it looked like. `AmbiguousGlobalPerLayerAttributeError` on `head_dim` read as a malformed upload -- DogOnKeyboard's config carries a `per_layer_config` key zerofata's canonical one lacks -- and the standing fix was to force `allow_global_per_layer_attribute_access=True`. Both halves were wrong. `pip install llmcompressor==0.13.0` downgrades transformers 5.16.1 -> 5.14.1. The config was serialized by 5.16.1, which materializes `per_layer_config` from `global_head_dim` + `layer_types`; 5.14.1 carries the heterogeneity guard but not the gemma4 resolver. Under the image's own transformers the same config loads fine. `:latest` was also re-pulled during attempt 4 and no earlier run, so the toolchain moved mid-diagnosis. Two things separated "malformed upload" from "moved toolchain": reproducing the real failing call (a bare AutoConfig load does not reproduce it; the trigger is reached through AutoTokenizer) and keeping zerofata's canonical tree, quantized cleanly on 2026-08-21, as a positive control. The fix drops `per_layer_config` rather than forcing global access. It is exactly redundant -- keys are precisely the ten full_attention layer indices, sole value (512, 4), verbatim the global fields -- and forcing instead would make `config.head_dim` answer 256 to the callers building the 512-wide layers. patch_perlayer.py re-proves that redundancy at apply time and refuses if it ever stops holding. Verified on the tensor table rather than the exit code: the output is identical family-for-family and count-for-count to the August canonical quant, with 356 BF16 vision-tower tensors preserved and input_activations=None. A GPU-free load leaves 0 tensors on meta and generates coherent prose. The section 4.4 serve test has NOT run -- GPU1 has 19.9 GB free against 19.5 GB of weights, so it needs a live seat displaced. Also fixes the A4B output, which had a truncation cap baked into its tokenizer (max_length 8192) from being quantized with the calibration corpus. Playbook gains section 3.17 for the pinned-transformers class and sharpens 3.16 to say drop the dataset outright for any A16 scheme.
120 lines
7.0 KiB
Markdown
120 lines
7.0 KiB
Markdown
# `[2026-09-10]` R49 / BabyBronte — D1–D3 built, carrier settled on dense Qwen3, and the 3-epoch pilot overfit
|
||
|
||
Operator opened the day with *"keep the adapter, then look into babybronte and start prepping
|
||
for a super lightweight small author voice training model regime."* By close the pipeline runs
|
||
end to end and one adapter exists.
|
||
|
||
## Carrier — SETTLED, and it inverted R49's own pin
|
||
|
||
Operator: *"use dense qwen3."* Sweep is **`Qwen3-{0.6,1.7,4}B-Base`**, which **overrides R49 H02's
|
||
stated arms** (the `Qwen3.5` trio). brokkr-smithy amended H02 accordingly.
|
||
|
||
The ruling came off a measurement, on pfi-gx10 (GB10), n=10/arm, seq 4096, r=32 on q,k,v,o+MLP,
|
||
bf16 sdpa, grad-ckpt on, spreads 0.3–2.6%:
|
||
|
||
Qwen3-0.6B-Base dense 0.616 B 2,399 tok/s
|
||
Qwen3-1.7B-Base dense 1.755 B 1,415 tok/s
|
||
Qwen3-4B-Base dense 4.089 B 717 tok/s
|
||
Qwen3.5-0.8B-Base hybrid 18 SSM/6 attn 0.765 B 540 tok/s
|
||
|
||
**The dense 4.089 B trains 33% FASTER than the hybrid 0.765 B**, on 5.3x the parameters. Cause:
|
||
Qwen3.5 is 18 linear-attention (Mamba) layers to 6 attention, and no fused kernel is installed
|
||
(`mamba_ssm`/`causal_conv1d`/`fla`/`kernels` all absent; triton 3.8 present). Grad-ckpt is not the
|
||
cause (19%, saves 2.6x memory). Batching is not the lever — both families sit at the box roofline
|
||
at batch 1. Projected per voice: hybrid 0.8B **12 h**, which is LONGER than the 7 h 26B-A4B tune the
|
||
regime exists to replace. The premise dies on that carrier.
|
||
|
||
Registry check closed the "is there something newer" question: **no official Qwen3.6/3.8 below 27B,
|
||
and neither family publishes a `-Base` checkpoint at any size.** Since the regime requires a
|
||
non-instruct carrier, dense Qwen3 was not a compromise — it was the only option.
|
||
|
||
⚠ Two Qwen3.5 landmines, measured not inferred: the *-Base releases **ship a vision tower** (153
|
||
`model.visual.*` Linear tensors at 0.8B) plus an MTP head, which `target_modules="all-linear"` would
|
||
train on text — `AutoModelForCausalLM` drops both, but that RENAMES modules relative to vLLM's
|
||
serving class, so adapter binding needs a sampled-target-changed check in the serving path. And
|
||
**cross-document packing is unsafe** because SSM state ignores the attention mask.
|
||
|
||
## D1 — corpus
|
||
|
||
`gx10:~/r49-corpus/`, instruments `scripts/r49-corpus/{build_corpus,verify_corpus}.py`. Charlotte
|
||
only (Jane Eyre 1260, Villette 9182, Shirley 30486, The Professor 1028; ids verified against
|
||
gutenberg.org's own search). **680,291 words · 142 chapters · 950,974 Qwen3 tokens.** 11 acceptance
|
||
checks pass including lossless round-trip and 0 byte-fallback.
|
||
|
||
⚠⚠ **The alphabet INVERTS the Yarros result.** Brontë writes French constantly (Villette, Adèle,
|
||
Brussels): 75 letters, **23 non-ASCII** — `ÆÉÊËÔàâäæçèéêëîïôöùûüŒœ`. F02 measured Yarros at 0.0002%
|
||
non-ASCII and derived an ASCII-fold. Under F02's own subset rule the Brontë pool **KEEPS French
|
||
accents** and **still excludes** Czech/Latvian/Slovak/Hungarian marks. First corpus where deriving
|
||
the alphabet per work changed the answer rather than confirming it.
|
||
|
||
⚠ Typography was inconsistent and it was the TRANSCRIBER, not the author: Shirley = straight quotes
|
||
+ `--` + zero em-dashes; Jane Eyre/Villette = curly + em-dash; The Professor = curly + `--`.
|
||
Normalised toward meaning.
|
||
|
||
## D2/D3 — entity resolution and rename
|
||
|
||
**Gender resolution is now TITLE-FIRST, a change from F02's method rather than a port of it.**
|
||
Pronoun proximity called **Jane male** — she narrates Jane Eyre, so her name appears mostly in
|
||
Rochester's dialogue surrounded by his pronouns. Titles have no such blind spot (`Miss Eyre`,
|
||
`Mrs. Fairfax`, `M. Paul`): **16 entities resolved, zero wrong**, every ambiguous case HELD.
|
||
**Held means ungendered, not unrenamed** — a held entity is still renamed from the neutral pool,
|
||
because holding a place leaks it (`Thornfield` × 100 = the `Riders Quadrant` case).
|
||
|
||
Pool: French + English per operator, locales `fr_FR, fr_BE, en_GB, en_IE` only (en_US/en_AU carry
|
||
modern surnames wrong for the 1840s), weighted per work by setting — Brussels novels 60% French,
|
||
Yorkshire 25%, alphabet-filtered.
|
||
|
||
⚠ **F02's collision filter does not carry across corpora, and whole-string filtering is not enough.**
|
||
The pool drew `Burns` and `Marie` (both Brontë characters), then after fixing that drew
|
||
**`Pierre-Yves`** while `Pierre` (Mademoiselle St. Pierre) is a Villette character. **Filter by
|
||
COMPONENT.** Final gate: 0 of 203 source entities survive in any of 24 copy-files.
|
||
|
||
## The pilot — and why 3 epochs was wrong
|
||
|
||
H02 is **pure continuation by design** (no beat annotation), which is why the pilot was launchable
|
||
without D4.
|
||
|
||
3-epoch run: train loss 3.310 → 2.574 while **held-out loss ROSE every epoch: 3.198 → 3.318 →
|
||
3.385.** Textbook overfit, and `save_strategy="no"` left nothing to fall back to. Operator: *"kill
|
||
and retrain 1 epoch."*
|
||
|
||
1-epoch rerun, same seed 4919 and byte-identical corpus (sha `3959036cf851bf62`), eval+save every
|
||
25 of 169 steps so the minimum is LOCATED not assumed:
|
||
|
||
step 25 3.2080 step 100 3.1764 step 169 3.1719 ← best, still descending
|
||
step 50 3.1891 step 125 3.1726
|
||
step 75 3.1786 step 150 3.1720
|
||
|
||
Monotonic descent, no turn. Better on held-out by 0.21 nats while train loss is HIGHER (3.147 vs
|
||
2.834) — the signature of the earlier run memorising. ⚠ Not strict subsets: cosine LR decays over
|
||
169 steps here vs 507 there.
|
||
|
||
Adapter verified bound: **196/196 lora_B tensors non-zero**. All three arms cut on one harness:
|
||
`base-unadapted`, `tuned-1ep-seed4919`, `incumbent-style-prompted`.
|
||
|
||
## Adjudication is now mine, and the independence is gone
|
||
|
||
Operator re-routed R49 entirely to infra-ops (*"leave babybronte to infra — concentrate on r50 and
|
||
the memory mechanism"*), relayed by brokkr who handed over the instrument and stepped off.
|
||
|
||
**I built the corpus, ran the training, and now judge whether it worked** — the exact shape of the
|
||
three inert gates R49's own journal records. Compensation: brokkr's rule **ratified verbatim and
|
||
frozen before any adapted text existed**, and I will not amend it after seeing numbers:
|
||
|
||
PASS iff adapted delta_cb <= 2.17 on >=3/4 works
|
||
AND below the base arm by more than the seed-to-seed spread
|
||
BEAT INCUMBENT iff below the style-prompted gen seat by more than that spread
|
||
|
||
⚠ Their controls came back **partly negative and that is what makes them useful**: real held-out
|
||
Charlotte scores delta_cb 1.65–2.17, Villette ties Thackeray at window level, and **Anne Brontë sits
|
||
at 2.374**, nearest-Charlotte of the whole panel. So `nearest` cannot separate the sister; only the
|
||
absolute band can. An adapter landing at 2.2 is not cleanly distinguishable from Anne.
|
||
|
||
Incumbent arm targets the CONCRETE model, not the alias: H02's `qwen3.6-35-a3b-heretic` is stale
|
||
(retired 2026-08-15); `gen` resolves to `hosted_vllm/qwen3.8-27b-uncensored` on ana-ml2:8015,
|
||
resolved at run start AND end.
|
||
|
||
Handoff bundle for adjudication at `/mnt/smithy/handoff/r49/`.
|
||
|
||
Related: [[2026-09-10-meromero-quants-and-the-pinned-transformers-trap]]
|