esh-pfi-infrastructure/docs/pfi/author-voice-lora-regime.md

Author-voice LoRA regime on pfi-gx10 — training-side prep

Written 2026-09-09 22:45 PT, revised 23:05 PT. Status: PREP. Nothing is training and nothing is queued. Both operator decisions from the first draft are now SETTLED (§6): carrier family is the dense Qwen3 line, and the intermediate checkpoints are purged.

The research target is brokkr-smithy R49 (research/R49-author-voice-adapters/), whose seed is the operator's BabyBronte design doc (R49/reference/babybronte-design-doc.md, 2026-08-25). R49 owns the hypotheses, the instruments and the adjudication. This document owns the other half — the box, the stack, the corpus staging, the trainer, the launcher, and the wall-clock — the same split that ran ERP-seat runs 3c through 7 on this box.

Read R49/target.md before touching the corpus design; several attractive ideas are already settled against there and re-proposing them is the failure mode this file exists to prevent.

---

1. What the regime is, in one paragraph

A voice is a LoRA adapter on a small non-instruct base model, trained on that author's real prose, steered at serving time by a terse beat line rather than a prompt. The completion in every training pair is 100% authentic author text; the only synthetic token in the corpus is the beat line, and beat lines are loss-masked, so the distribution the adapter learns to emit is the author's and the style ceiling is the author's own. Names and places are substituted deterministically — seeded RNG over a curated 23,398-name dictionary, never an LLM — which is what lets one work become 5–8 training copies without teaching plot memorisation.

The value being bought is marginal cost per voice, not inference latency. That framing is the operator's and it is load-bearing: the anchor for "expensive" is ERP run 2 at ~7 h for a single 26B-A4B tune on a harness whose audit found a blocking-mask defect, a vision tower a leaf-name regex would have trained on text, and 128 experts fused per layer. A dense sub-2B carrier has none of those failure modes, and at this size the methodology floor is finally cheaper than the shortcut — two seeds per arm and a re-run after every change are routine rather than unaffordable.

2. The box, and what is already staged

pfi-gx10 (10.100.50.60) — ASUS Ascent GX10, NVIDIA GB10, sm_121, aarch64, 121 GB unified memory, 916 GB NVMe, 470 GB free. Operator ruling 2026-09-09: experimental box, primarily for training, no serving seat. Its GPU is idle.

Training stack already present and current at /home/infra-ops/ml/.venv:

torch	2.14.0+cu130, torch.cuda.get_device_capability() == (12, 1)
transformers	5.16.1 (loads Qwen3_5Config natively)
peft / trl / accelerate	0.20.0 / 1.12.0 / 1.14.0
datasets / bitsandbytes	5.0.1 / 0.50.2
absent	mamba_ssm, causal_conv1d, fla, flash_attn, kernels (triton 3.8.0 present)

Staged on local NVMe under /home/infra-ops/carriers/ (pulled 2026-09-09, existence API-verified against the HF registry first, with a phantom repo run as the negative control):

CHOSEN — dense Qwen3          held, not chosen — hybrid Qwen3.5
Qwen3-0.6B-Base   1.2G        Qwen3.5-0.8B-Base   1.7G
Qwen3-1.7B-Base   3.3G        Qwen3.5-2B-Base     4.3G
Qwen3-4B-Base     ~8G         Qwen3.5-4B-Base     8.8G

Probes live at scripts/training-probes/{probe_carrier.py,bench_lora_step.py} with raw output in bench-lora-step-gx10-2026-09-09.jsonl, so every number below can be re-derived rather than taken on faith.

3. What the carriers actually are — measured, not read off the model card

R49 H02 names the Qwen3.5 trio. Probing the checkpoints rather than the config found three things worth knowing before writing a recipe.

They ship a vision tower and an MTP head. model.visual.* is 153 tensors on the 0.8B and 297 on the 2B — attn.qkv, attn.proj, mlp.linear_fc1/2, all nn.Linear leaves that target_modules="all-linear" would attach LoRA to and then train on pure text. This is the same defect the ERP harness audit caught on gemma-4. Mitigation is free: loading through AutoModelForCausalLM returns Qwen3_5ForCausalLM with the vision tower and MTP head dropped entirely — 0.752 B of text model, module paths model.layers.N.*.

⚠ That mitigation creates a serving trap. vLLM will load the full Qwen3_5ForConditionalGeneration, where the same weights live at model.language_model.layers.N.*. An adapter trained against the CausalLM prefix may not bind. Pre-flight: load the finished adapter in the serving path and confirm a sampled target tensor actually changed — the same silent-no-op check the ERP merge step already uses.

Three quarters of the layers are not attention. layer_types is 3× linear_attention + 1× full_attention repeating: 18 SSM / 6 attention at 0.8B and 2B, 24 / 8 at 4B. The SSM blocks carry conv1d, A_log, dt_bias and five Linear projections; mamba_ssm_dtype is float32.

region	0.8B	2B	4B	LoRA-able leaves
MLP	35.1%	48.1%	53.9%	gate_proj, up_proj, down_proj
embeddings (tied)	33.8%	27.0%	15.1%	lm_head — exclude
linear-attn (SSM)	25.2%	20.1%	24.0%	in_proj_{qkv,a,b,z}, out_proj
full attention	5.9%	4.7%	7.0%	q_proj, k_proj, v_proj, o_proj
total	0.752 B	1.882 B	4.206 B

The conventional q,k,v,o + MLP recipe therefore covers 41% of the 0.8B and leaves the SSM stack untouched in 18 of 24 layers. Adding the SSM Linears takes coverage to 66%. At this scale that is a cheap ablation, not an agonising choice.

⚠ Packing across document boundaries is unsafe on this architecture. An SSM layer carries recurrent state along the sequence and an attention mask does not reset it, so two renamed copies packed into one 8k window can bleed in 18 of 24 layers — which is precisely the per-copy name-consistency invariant the design doc calls sacred. Either one document per sequence, or prove the trainer's sequence-boundary signal is honoured by the linear-attn path. Under a dense carrier this problem does not exist.

The tied embedding is a third of the small carrier. vocab 248,320 × hidden 1024 = 254 M of the 0.752 B. The transformer body being tested at the small end is ~0.50 B, which matters when reporting "the carrier floor".

4. Throughput — and the newest carrier is the slow one

One forward+backward+AdamW microbatch, LoRA r=32/α=64 on q,k,v,o + MLP, bf16, sdpa, gradient checkpointing on, seq 4096, on gx10's GB10. n=10 measured after 3 warmup steps; median reported with the full spread.

carrier	architecture	params	s/step	tok/s	peak	spread
Qwen3.5-0.8B-Base	hybrid, 18 SSM / 6 attn	0.765 B	7.581	540	15.1 GiB	2.6%
Qwen3.5-0.8B-Base (no grad-ckpt)	"	0.765 B	6.364	644	38.9 GiB	1.5%
Qwen3-0.6B-Base	dense	0.616 B	1.707	2,399	9.8 GiB	0.6%
Qwen3-1.7B-Base	dense	1.755 B	2.895	1,415	12.2 GiB	0.8%
Qwen3-4B-Base	dense	4.089 B	5.714	717	17.2 GiB	0.3%
Qwen3-1.7B-Base, batch 4	dense	1.755 B	11.387	1,439	38.0 GiB	0.6%
Qwen3.5-0.8B-Base, batch 4	hybrid	0.765 B	30.030	546	55.5 GiB	0.7%

The dense 1.755 B carrier trains 2.6× faster than the hybrid 0.765 B one — on 2.3× the parameters, with more LoRA modules adapted (196 vs 96, because dense has real attention in every layer). Per parameter the dense path is ~6× more efficient. Spreads of 0.6–2.6% across n=10 put the instrument's noise an order of magnitude below the effect, so this is not variance.

The cause is almost certainly that no fused linear-attention kernel is installed (§2) so the SSM path runs a reference implementation. Grad checkpointing is not the culprit — turning it off recovers only 19% and costs 2.6× the memory, so leave it on. Batching is not the lever for either family: 1,415 → 1,439 tok/s dense and 540 → 546 tok/s hybrid from batch 1 to 4. Both architectures are already at this box's roofline at batch 1, which is a bandwidth story (GB10's unified LPDDR5X against an RTX PRO 6000's ~6.6× higher figure) — and it means the 2.6× gap is the kernel path, not a batching artefact.

What that does to the regime's premise. Projecting a Brontë-scale corpus (~1 M words ≈ 1.3 M tokens, × 6 rename copies, 3 epochs ≈ 23 M tokens):

carrier	projected wall-clock per voice
Qwen3-0.6B-Base dense	2.7 h
Qwen3-1.7B-Base dense	4.6 h
Qwen3-4B-Base dense	9.1 h
Qwen3.5-0.8B-Base hybrid	12 h

The sharpest way to put it: the dense 4.089 B carrier still trains 33% faster than the hybrid 0.765 B one, on 5.3× the parameters. The full three-arm dense sweep at two seeds each is ~33 h of GPU — about a day and a half for the whole H02 carrier question, and ~10 h if H03's ~300 k-word corpus floor holds.

The hybrid carrier would make a per-voice run longer than the 7 h 26B-A4B tune it exists to replace. At R49 H03's hoped-for corpus floor (~300 k words) the dense 1.7B lands near 1.4 h — a voice per afternoon, which is the regime the operator asked for. ⚠ These are projections from a synthetic-token throughput harness, not from a completed run; treat them as sizing, and re-measure on the first real corpus.

5. Prep remaining, in order

1. Carrier family decision — settled: dense Qwen3 (§6a).
2. Corpus D1 — BUILT AND GREEN 2026-09-10 (§8).
3. Re-point the R49 deterministic machinery at Brontë. The entity detector (corpus-level capitalised-vs-lowercase ratio), identity linking, gender resolution and the 23,398-name dictionary were all built and hardened against a Yarros sample. Per-work re-derivation needed: entity map, alphabet, and the UNRESOLVED_BLOCKING human pass (~20–40 entities per work).
4. Beat annotation (D4) via gen inverse-prompting, using F02's hardened prompt (banned meta-language, three PD worked examples, ≤20-word gate).
5. Trainer. erp_sft_harness is chat-shaped and carries ERP-specific eligibility machinery; the author-voice job is plain continuation with a masked prefix. Decision: a small purpose-built trainer that keeps the harness's §4 disciplines (provenance pin, order manifest, truncation report, cache key that sees semantic changes, recorded attention backend) rather than a fork of its corpus logic.
6. Pre-set the decision threshold before collecting data, wider than the measured seed-to-seed spread, per the R49 charter — and run the positive control the R49 journal's own lesson demands: confirm the stylometric instrument separates real Brontë from unadapted base output before it is asked to judge an adapter.
7. Launcher modelled on launch-run-07.sh — its guards were each bought with a past failure (GPU-clear assertion, pidfile not pgrep -f, refuse an existing log, free-space floor, setsid detach).

Deliberately out of scope here, per R49: the Director/critic loop, style arithmetic and the Pelican test, multi-LoRA arsenal serving, the Skaldsong integration contract, the modern in-copyright arsenal, inference latency.

6. Open for the operator

(a) Carrier family — SETTLED 2026-09-09, operator: "use dense qwen3". The sweep is Qwen3-{0.6,1.7,4}B-Base — the design doc's own original pin, and the family the measurements favour by 2.6–6×. This overrides R49 H02's stated arms, which name Qwen3.5-{0.8,2,4}B-Base; brokkr-smithy owns that file and has been told directly.

What the ruling buys, restated so it is not re-litigated: no vision tower and no MTP head to exclude, cross-document packing is safe again, sdpa/flash are both reachable, and the per-voice wall-clock is 2.7–4.6 h instead of 12 h. What it costs: one model generation of base quality. Reversible — the three Qwen3.5 checkpoints stay staged (14.8 GB), and an fla install (pure Triton, plausibly fine on aarch64) could revive that family as a follow-up experiment rather than a prerequisite. If anyone re-opens this, re-run bench_lora_step.py first; the argument is a measurement, not a preference.

Is there a newer small base to prefer instead? No — checked against the HF registry 2026-09-09, prompted by the operator asking brokkr-smithy the same question. Three facts, and together they close it:

1. No official Qwen3.6 or Qwen3.8 exists below 27B. The whole Qwen/ listing is Qwen3.6-{27B, 35B-A3B} and Qwen3.8-{27B, 2.4T-A95B, Flash-Next}.
2. Neither family publishes a -Base checkpoint at all. Every Base newer than Qwen3 is Qwen3.5 — Qwen3.5-{0.8B, 2B, 4B, 9B, 35B-A3B}-Base. Since the regime requires a non-instruct carrier, that alone rules the 3.6/3.8 lines out regardless of size.
3. The 27Bs are the same kernel path one size up. Qwen3.6-27B and Qwen3.8-27B both report model_type: qwen3_5, Qwen3_5ForConditionalGeneration, 64 layers as 16 full + 48 linear-attention, vision tower present — the exact shape measured slow above.

So the newest official small dense Qwen base is still the Qwen3 line, and the ruling is not a compromise against a better available option; it is the only dense option. (Third-party Qwen3.8-*-Distill checkpoints are Qwen3.5 hybrids distilled on 3.8 outputs — same kernel path, and unpinned provenance besides.)

⚠ Headroom worth knowing about, though H02 does not need it today: the dense Base line continues past 4B — Qwen3-8B-Base and Qwen3-14B-Base both exist. H02 caps at 4B by design ("sub-4B carrier"), and the projections say 4B already costs 9.1 h a voice, so this is not a proposal. It is insurance: if the 0.6/1.7/4B curve has not flattened at 4B, the sweep can be extended without changing family. (A Qwen/SAE-Res-*-Base-* row in a registry search is an interpretability sidecar, not a carrier — ignore those.)

(b) Intermediate checkpoints — PURGED 2026-09-09 23:00 PT, operator: "purge intermediate checkpoints". Seven checkpoints/ directories deleted with literal paths, one rm per line, after confirming none was a symlink and that every run's final adapter/ is an independent real directory:

pfi-gx10   run-03c 11G · run-04 16G · run-05 9.2G · run-06 9.2G   -> 45 GB
ana-ml2    run-01 12G · run-02 12G · run-03 5.9G                  -> 29 GB

gx10 419G→374G used (496 GB free); /tank/erp-tune 392G→363G, with zfs list -t snapshot empty so the space is genuinely returned rather than snapshot-held. All eight adapters re-verified by sha256 after the deletion, matching the values recorded during the mirror.

And the merged models followed at 23:08 PT, operator: "purge the merged models, keep run06 and the v6 quant." Eleven merges, literal paths, one rm per line, after proving what the live seat reads: docker inspect vllm-erp-seat shows it binds /tank/aimodels only and its model arg is /tank/aimodels/erp-tune-v6-nvfp4a16, so /tank/erp-tune/serve was never in the serving path at all. No container mounts it and no process held a file open under it (checked /proc/*/fd and /proc/*/maps, not lsof alone).

pfi-gx10   merged-run03c · merged-run04 · merged-run05      3 x 49 GiB -> 145 GB
ana-ml2    merged-final · merged-run02 · merged-run03 ·
           merged-run03-s{025,050,075} · merged-test ·
           nvfp4a16-test                                    8 dirs     -> 354 GB

All eleven were Gemma-4 merges from the run-1/2/3 era (Aug 24–26); the keeper is dated Sep 8 and lives in a different tree, so there was no ambiguity about which was which. gx10 382G→237G used (632 GB free); /tank/erp-tune/serve 354G→8.6M with zfs list -t snapshot empty, and the pool's raw FREE moved 4.80T→5.30T.

Kept, and verified after the deletion: gx10:~/erp-tune/serve/merged-run06 (49 GiB, 9 files), /tank/aimodels/erp-tune-v6-bf16 (49 G) and erp-tune-v6-nvfp4a16 (16 G, 9 files incl. recipe.yaml), all eight adapters sha256-unchanged, and /tank/erp-tune/serve's merge/quant tooling and logs (merge_lora.py, quant_nvfp4a16.py, the dry-run logs, base-arm, lora-scales — 8.6 MB total, the provenance for how every artifact above was built).

⚠ gx10:~/erp-tune/relaunch-trial-seat.sh now names a model that is gone. It was not deleted — its flags carry the FlashInfer JIT/PATH trap and the gpu-clear/never-pkill notes, each bought with a real failure — but it gained a RETIRED banner so nobody hits a confusing missing-model error later.

The live seat never bounced. After the deletion Pfish-6 answered a real completion (finish_reason: stop, 5 tokens, correct text), container still Up 4 hours (healthy).

7. Adapter disposition — settled, and made real

Operator, 2026-09-09: keep the adapter. As of 22:30 PT all five gx10-resident ERP adapters are mirrored to ana-ml2:/tank/erp-tune/run-<N>/adapter, matching the layout runs 01–03 already use there, byte-total identical on both sides and sha256 matching on every adapter_model.safetensors:

run-03c  run-04  run-05  run-06  run-07     315 MB each, 8 files each

/tank/* is deliberately excluded from ana-ml2's restic sources — terabytes of regenerable model weights. A trained adapter is the one thing under there upstream cannot hand back, so configs/restic/ana-ml2/profiles.yaml now carries a single documented carve-out, /tank/erp-tune/run-*/adapter, verified by resticprofile --dry-run to expand to exactly those eight paths and nothing else. The nightly 01:00 run picks them up.

---

8. D1 is built — the corpus, and three things it taught

scripts/r49-corpus/build_corpus.py --build and verify_corpus.py, corpus staged at gx10:~/r49-corpus/. All 11 acceptance checks pass, including the two tokenizer legs run against the pilot carrier itself.

works	Jane Eyre (1260) · Villette (9182) · Shirley (30486) · The Professor (1028)
size	680,291 words · 142 chapters · 950,974 tokens under the Qwen3 tokenizer (1.40 tok/word)
alphabet	75 letters, 23 of them non-ASCII
tokenizer	round-trip lossless, 0 byte-fallback pieces of 98,860

Catalogue ids were verified against gutenberg.org's own search, not recalled. Charlotte only — the Bell poems are co-authored and the Gaskell biography is a different hand, so neither belongs in a single-voice corpus.

Real denominator, so the projections tighten. At 6 rename copies × 3 epochs = 17.1M tokens: 0.6B → 1.98 h, 1.7B → 3.36 h, 4B → 6.63 h per voice. The pilot is a two-hour run.

⚠ The alphabet finding inverts the Yarros result, and brokkr's re-point depends on it

ÆÉÊËÔàâäæçèéêëîïôöùûüŒœ

F02 measured the Yarros corpus at 4 non-ASCII characters in 1.8M letters (0.0002%) and derived an ASCII-fold rule for the name pool. Charlotte Brontë is the opposite case: she writes French constantly — Villette is set in a French-speaking city, Jane Eyre has Adèle, The Professor is set in Brussels — and the corpus carries é 432 times, è 237, à 93, ê 79, ô 48, plus the œ and æ ligatures.

Under F02's own rule (the pool's character inventory must be a subset of the corpus's) the Brontë pool may keep French accents and must still exclude the Czech/Latvian/Slovak/Hungarian marks that never appear here. So the fold is not global — it is derived per work, which is exactly what the rule said, and this is the first corpus where the derivation changes the answer.

Typography was inconsistent, and it was the transcriber, not the author

work	quotes	dashes
Jane Eyre	curly	em-dash 2,058
Villette	curly	em-dash 2,272
Shirley	straight 9,115	-- 2,228, zero em-dashes
The Professor	curly	-- 964, zero em-dashes

Left alone the adapter would learn that this author "sometimes" writes each form — a false habit on the exact axis being trained. Normalised toward what the text means: -- is a transcription of an em-dash, so it becomes one; straight quotes are paired into curly per paragraph.

Two gate defects, both found by running the gate

1. Produced by matched Brontë's own prose — "a chilling effect produced by his steady announcement", three more like it. A hard rule on a phrase with a common innocent sense, manufacturing failures: the same shape as the drift detector that fired on the adjective "minor". Fixed by anchoring the boilerplate patterns to line start, where Gutenberg credits actually live.
2. Asserting open/close quote counts must be equal was wrong. Nineteenth-century convention runs a speech across paragraphs by opening each and closing only the last, so a surplus of opens is correct — measured +46 / +49 / +51 on the three works whose quotes were never touched. Replaced with the real error signature: no paragraph may begin with a closing quote, which convention never produces and a bad conversion does. 0 of 14,230 paragraphs.

A third, mine: the manifest baked absolute build-machine paths, so the corpus was unreadable the moment it moved to gx10. Paths are relative to the corpus root now. It failed loudly rather than silently reading nothing, which is why it was cheap.

Next

D2/D3 — re-point F02's entity detector, identity linking and gender resolution off the Yarros sample onto Brontë, with the alphabet above constraining the pool. Then D4 annotation, then the pilot.

Pilot ruling (operator, 2026-09-10): trial on Qwen3-0.6B-Base first and only move up if it produces something useful. So the sweep is not three arms up front — it is one ~2 h run, judged, and then a decision. That is the cheap ordering and it front-loads the kill signal: if voice does not transfer at 0.6B the question becomes how far up rather than whether at all, and if it does transfer the larger arms are a refinement rather than a gamble.