# Author-voice LoRA regime on pfi-gx10 — training-side prep _Written 2026-09-09 22:45 PT, revised 23:05 PT. Status: **PREP. Nothing is training and nothing is queued.** Both operator decisions from the first draft are now **SETTLED** (§6): carrier family is the **dense `Qwen3` line**, and the intermediate checkpoints are purged._ The research target is **brokkr-smithy R49** (`research/R49-author-voice-adapters/`), whose seed is the operator's **BabyBronte** design doc (`R49/reference/babybronte-design-doc.md`, 2026-08-25). R49 owns the hypotheses, the instruments and the adjudication. **This document owns the other half — the box, the stack, the corpus staging, the trainer, the launcher, and the wall-clock** — the same split that ran ERP-seat runs 3c through 7 on this box. Read `R49/target.md` before touching the corpus design; several attractive ideas are already settled *against* there and re-proposing them is the failure mode this file exists to prevent. --- ## 1. What the regime is, in one paragraph A voice is a **LoRA adapter on a small non-instruct base model**, trained on that author's real prose, steered at serving time by a terse beat line rather than a prompt. The completion in every training pair is 100% authentic author text; the only synthetic token in the corpus is the beat line, and beat lines are **loss-masked**, so the distribution the adapter learns to *emit* is the author's and the style ceiling is the author's own. Names and places are substituted **deterministically** — seeded RNG over a curated 23,398-name dictionary, never an LLM — which is what lets one work become 5–8 training copies without teaching plot memorisation. **The value being bought is marginal cost per voice, not inference latency.** That framing is the operator's and it is load-bearing: the anchor for "expensive" is ERP run 2 at ~7 h for a single 26B-A4B tune on a harness whose audit found a blocking-mask defect, a vision tower a leaf-name regex would have trained on text, and 128 experts fused per layer. A dense sub-2B carrier has none of those failure modes, and at this size the **methodology floor is finally cheaper than the shortcut** — two seeds per arm and a re-run after every change are routine rather than unaffordable. ## 2. The box, and what is already staged `pfi-gx10` (10.100.50.60) — ASUS Ascent GX10, NVIDIA **GB10**, `sm_121`, aarch64, **121 GB unified** memory, 916 GB NVMe, 470 GB free. Operator ruling 2026-09-09: **experimental box, primarily for training, no serving seat.** Its GPU is idle. Training stack already present and current at `/home/infra-ops/ml/.venv`: | | | |---|---| | torch | `2.14.0+cu130`, `torch.cuda.get_device_capability() == (12, 1)` | | transformers | 5.16.1 (loads `Qwen3_5Config` natively) | | peft / trl / accelerate | 0.20.0 / 1.12.0 / 1.14.0 | | datasets / bitsandbytes | 5.0.1 / 0.50.2 | | **absent** | `mamba_ssm`, `causal_conv1d`, `fla`, `flash_attn`, `kernels` (triton 3.8.0 present) | Staged on local NVMe under `/home/infra-ops/carriers/` (pulled 2026-09-09, existence API-verified against the HF registry first, with a phantom repo run as the negative control): CHOSEN — dense Qwen3 held, not chosen — hybrid Qwen3.5 Qwen3-0.6B-Base 1.2G Qwen3.5-0.8B-Base 1.7G Qwen3-1.7B-Base 3.3G Qwen3.5-2B-Base 4.3G Qwen3-4B-Base ~8G Qwen3.5-4B-Base 8.8G Probes live at `scripts/training-probes/{probe_carrier.py,bench_lora_step.py}` with raw output in `bench-lora-step-gx10-2026-09-09.jsonl`, so every number below can be re-derived rather than taken on faith. ## 3. What the carriers actually are — measured, not read off the model card R49 H02 names the `Qwen3.5` trio. Probing the checkpoints rather than the config found three things worth knowing before writing a recipe. **They ship a vision tower and an MTP head.** `model.visual.*` is 153 tensors on the 0.8B and 297 on the 2B — `attn.qkv`, `attn.proj`, `mlp.linear_fc1/2`, all `nn.Linear` leaves that `target_modules="all-linear"` would attach LoRA to and then train on pure text. This is the *same* defect the ERP harness audit caught on gemma-4. **Mitigation is free:** loading through `AutoModelForCausalLM` returns `Qwen3_5ForCausalLM` with the vision tower and MTP head dropped entirely — 0.752 B of text model, module paths `model.layers.N.*`. ⚠ **That mitigation creates a serving trap.** vLLM will load the full `Qwen3_5ForConditionalGeneration`, where the same weights live at `model.language_model.layers.N.*`. An adapter trained against the CausalLM prefix may not bind. **Pre-flight:** load the finished adapter in the serving path and confirm a sampled target tensor actually changed — the same silent-no-op check the ERP merge step already uses. **Three quarters of the layers are not attention.** `layer_types` is 3× `linear_attention` + 1× `full_attention` repeating: 18 SSM / 6 attention at 0.8B and 2B, 24 / 8 at 4B. The SSM blocks carry `conv1d`, `A_log`, `dt_bias` and five Linear projections; `mamba_ssm_dtype` is `float32`. | region | 0.8B | 2B | 4B | LoRA-able leaves | |---|---|---|---|---| | MLP | 35.1% | 48.1% | 53.9% | `gate_proj`, `up_proj`, `down_proj` | | embeddings (tied) | 33.8% | 27.0% | 15.1% | `lm_head` — exclude | | linear-attn (SSM) | 25.2% | 20.1% | 24.0% | `in_proj_{qkv,a,b,z}`, `out_proj` | | full attention | 5.9% | 4.7% | 7.0% | `q_proj`, `k_proj`, `v_proj`, `o_proj` | | **total** | **0.752 B** | **1.882 B** | **4.206 B** | | The conventional `q,k,v,o` + MLP recipe therefore covers **41%** of the 0.8B and leaves the SSM stack untouched in 18 of 24 layers. Adding the SSM Linears takes coverage to 66%. At this scale that is a cheap ablation, not an agonising choice. ⚠ **Packing across document boundaries is unsafe on this architecture.** An SSM layer carries recurrent state along the sequence and an attention mask does not reset it, so two renamed copies packed into one 8k window can bleed in 18 of 24 layers — which is precisely the per-copy name-consistency invariant the design doc calls sacred. Either one document per sequence, or prove the trainer's sequence-boundary signal is honoured by the linear-attn path. Under a dense carrier this problem does not exist. **The tied embedding is a third of the small carrier.** vocab 248,320 × hidden 1024 = 254 M of the 0.752 B. The transformer body being tested at the small end is ~0.50 B, which matters when reporting "the carrier floor". ## 4. Throughput — and the newest carrier is the slow one One forward+backward+AdamW microbatch, LoRA r=32/α=64 on `q,k,v,o` + MLP, bf16, `sdpa`, gradient checkpointing on, seq 4096, on gx10's GB10. n=10 measured after 3 warmup steps; median reported with the full spread. | carrier | architecture | params | s/step | tok/s | peak | spread | |---|---|---|---|---|---|---| | `Qwen3.5-0.8B-Base` | hybrid, 18 SSM / 6 attn | 0.765 B | 7.581 | **540** | 15.1 GiB | 2.6% | | `Qwen3.5-0.8B-Base` (no grad-ckpt) | " | 0.765 B | 6.364 | 644 | 38.9 GiB | 1.5% | | `Qwen3-0.6B-Base` | dense | 0.616 B | 1.707 | **2,399** | 9.8 GiB | 0.6% | | `Qwen3-1.7B-Base` | dense | 1.755 B | 2.895 | **1,415** | 12.2 GiB | 0.8% | | `Qwen3-4B-Base` | dense | 4.089 B | 5.714 | **717** | 17.2 GiB | 0.3% | | `Qwen3-1.7B-Base`, batch 4 | dense | 1.755 B | 11.387 | 1,439 | 38.0 GiB | 0.6% | | `Qwen3.5-0.8B-Base`, batch 4 | hybrid | 0.765 B | 30.030 | 546 | 55.5 GiB | 0.7% | **The dense 1.755 B carrier trains 2.6× faster than the hybrid 0.765 B one** — on 2.3× the parameters, with *more* LoRA modules adapted (196 vs 96, because dense has real attention in every layer). Per parameter the dense path is ~6× more efficient. Spreads of 0.6–2.6% across n=10 put the instrument's noise an order of magnitude below the effect, so this is not variance. The cause is almost certainly that **no fused linear-attention kernel is installed** (§2) so the SSM path runs a reference implementation. Grad checkpointing is *not* the culprit — turning it off recovers only 19% and costs 2.6× the memory, so leave it on. Batching is not the lever for either family: 1,415 → 1,439 tok/s dense and 540 → 546 tok/s hybrid from batch 1 to 4. **Both architectures are already at this box's roofline at batch 1**, which is a bandwidth story (GB10's unified LPDDR5X against an RTX PRO 6000's ~6.6× higher figure) — and it means the 2.6× gap is the kernel path, not a batching artefact. **What that does to the regime's premise.** Projecting a Brontë-scale corpus (~1 M words ≈ 1.3 M tokens, × 6 rename copies, 3 epochs ≈ 23 M tokens): | carrier | projected wall-clock per voice | |---|---| | `Qwen3-0.6B-Base` dense | **2.7 h** | | `Qwen3-1.7B-Base` dense | **4.6 h** | | `Qwen3-4B-Base` dense | **9.1 h** | | `Qwen3.5-0.8B-Base` hybrid | **12 h** | The sharpest way to put it: **the dense 4.089 B carrier still trains 33% faster than the hybrid 0.765 B one**, on 5.3× the parameters. The full three-arm dense sweep at two seeds each is ~33 h of GPU — about a day and a half for the whole H02 carrier question, and ~10 h if H03's ~300 k-word corpus floor holds. The hybrid carrier would make a per-voice run *longer than the 7 h 26B-A4B tune it exists to replace.* At R49 H03's hoped-for corpus floor (~300 k words) the dense 1.7B lands near **1.4 h** — a voice per afternoon, which is the regime the operator asked for. ⚠ These are projections from a synthetic-token throughput harness, not from a completed run; treat them as sizing, and re-measure on the first real corpus. ## 5. Prep remaining, in order 1. ~~Carrier family decision~~ — **settled: dense `Qwen3`** (§6a). 2. **Corpus D1** — Gutenberg Brontë (Jane Eyre, Villette, Shirley, The Professor), boilerplate stripped, chapter-segmented, typography normalised, character inventory recorded. Public domain, clean under any disposition. 3. **Re-point the R49 deterministic machinery at Brontë.** The entity detector (corpus-level capitalised-vs-lowercase ratio), identity linking, gender resolution and the 23,398-name dictionary were all built and hardened against a *Yarros* sample. Per-work re-derivation needed: entity map, alphabet, and the `UNRESOLVED_BLOCKING` human pass (~20–40 entities per work). 4. **Beat annotation (D4)** via `gen` inverse-prompting, using F02's hardened prompt (banned meta-language, three PD worked examples, ≤20-word gate). 5. **Trainer.** `erp_sft_harness` is chat-shaped and carries ERP-specific eligibility machinery; the author-voice job is plain continuation with a masked prefix. Decision: a small purpose-built trainer that *keeps* the harness's §4 disciplines (provenance pin, order manifest, truncation report, cache key that sees semantic changes, recorded attention backend) rather than a fork of its corpus logic. 6. **Pre-set the decision threshold before collecting data**, wider than the measured seed-to-seed spread, per the R49 charter — and run the positive control the R49 journal's own lesson demands: confirm the stylometric instrument separates real Brontë from unadapted base output *before* it is asked to judge an adapter. 7. **Launcher** modelled on `launch-run-07.sh` — its guards were each bought with a past failure (GPU-clear assertion, pidfile not `pgrep -f`, refuse an existing log, free-space floor, `setsid` detach). Deliberately out of scope here, per R49: the Director/critic loop, style arithmetic and the Pelican test, multi-LoRA arsenal serving, the Skaldsong integration contract, the modern in-copyright arsenal, inference latency. ## 6. Open for the operator **(a) Carrier family — SETTLED 2026-09-09, operator: _"use dense qwen3"._** The sweep is **`Qwen3-{0.6,1.7,4}B-Base`** — the design doc's own original pin, and the family the measurements favour by 2.6–6×. **This overrides R49 H02's stated arms**, which name `Qwen3.5-{0.8,2,4}B-Base`; brokkr-smithy owns that file and has been told directly. What the ruling buys, restated so it is not re-litigated: no vision tower and no MTP head to exclude, cross-document packing is safe again, `sdpa`/flash are both reachable, and the per-voice wall-clock is 2.7–4.6 h instead of 12 h. What it costs: one model generation of base quality. **Reversible** — the three Qwen3.5 checkpoints stay staged (14.8 GB), and an `fla` install (pure Triton, plausibly fine on aarch64) could revive that family as a follow-up experiment rather than a prerequisite. If anyone re-opens this, re-run `bench_lora_step.py` first; the argument is a measurement, not a preference. **Is there a newer small base to prefer instead? No — checked against the HF registry 2026-09-09, prompted by the operator asking brokkr-smithy the same question.** Three facts, and together they close it: 1. **No official Qwen3.6 or Qwen3.8 exists below 27B.** The whole `Qwen/` listing is `Qwen3.6-{27B, 35B-A3B}` and `Qwen3.8-{27B, 2.4T-A95B, Flash-Next}`. 2. **Neither family publishes a `-Base` checkpoint at all.** Every Base newer than Qwen3 is Qwen3.5 — `Qwen3.5-{0.8B, 2B, 4B, 9B, 35B-A3B}-Base`. Since the regime requires a *non-instruct* carrier, that alone rules the 3.6/3.8 lines out regardless of size. 3. **The 27Bs are the same kernel path one size up.** `Qwen3.6-27B` and `Qwen3.8-27B` both report `model_type: qwen3_5`, `Qwen3_5ForConditionalGeneration`, 64 layers as **16 full + 48 linear-attention**, vision tower present — the exact shape measured slow above. So the newest official small **dense** Qwen base is still the `Qwen3` line, and the ruling is not a compromise against a better available option; it is the only dense option. (Third-party `Qwen3.8-*-Distill` checkpoints are Qwen3.5 hybrids distilled on 3.8 outputs — same kernel path, and unpinned provenance besides.) ⚠ **Headroom worth knowing about, though H02 does not need it today:** the dense Base line continues past 4B — `Qwen3-8B-Base` and `Qwen3-14B-Base` both exist. H02 caps at 4B by design ("sub-4B carrier"), and the projections say 4B already costs 9.1 h a voice, so this is not a proposal. It is insurance: if the 0.6/1.7/4B curve has *not* flattened at 4B, the sweep can be extended without changing family. (A `Qwen/SAE-Res-*-Base-*` row in a registry search is an interpretability sidecar, not a carrier — ignore those.) **(b) Intermediate checkpoints — PURGED 2026-09-09 23:00 PT, operator: _"purge intermediate checkpoints"._** Seven `checkpoints/` directories deleted with literal paths, one `rm` per line, after confirming none was a symlink and that every run's final `adapter/` is an independent real directory: pfi-gx10 run-03c 11G · run-04 16G · run-05 9.2G · run-06 9.2G -> 45 GB ana-ml2 run-01 12G · run-02 12G · run-03 5.9G -> 29 GB gx10 419G→374G used (496 GB free); `/tank/erp-tune` 392G→363G, with `zfs list -t snapshot` empty so the space is genuinely returned rather than snapshot-held. **All eight adapters re-verified by `sha256` after the deletion**, matching the values recorded during the mirror. **And the merged models followed at 23:08 PT**, operator: _"purge the merged models, keep run06 and the v6 quant."_ Eleven merges, literal paths, one `rm` per line, after proving what the live seat reads: `docker inspect vllm-erp-seat` shows it binds **`/tank/aimodels` only** and its model arg is `/tank/aimodels/erp-tune-v6-nvfp4a16`, so `/tank/erp-tune/serve` was never in the serving path at all. No container mounts it and no process held a file open under it (checked `/proc/*/fd` and `/proc/*/maps`, not `lsof` alone). pfi-gx10 merged-run03c · merged-run04 · merged-run05 3 x 49 GiB -> 145 GB ana-ml2 merged-final · merged-run02 · merged-run03 · merged-run03-s{025,050,075} · merged-test · nvfp4a16-test 8 dirs -> 354 GB All eleven were Gemma-4 merges from the run-1/2/3 era (Aug 24–26); the keeper is dated Sep 8 and lives in a different tree, so there was no ambiguity about which was which. gx10 382G→237G used (**632 GB free**); `/tank/erp-tune/serve` 354G→8.6M with `zfs list -t snapshot` empty, and the pool's raw FREE moved 4.80T→5.30T. **Kept, and verified after the deletion:** `gx10:~/erp-tune/serve/merged-run06` (49 GiB, 9 files), `/tank/aimodels/erp-tune-v6-bf16` (49 G) and `erp-tune-v6-nvfp4a16` (16 G, 9 files incl. `recipe.yaml`), all eight adapters sha256-unchanged, and `/tank/erp-tune/serve`'s merge/quant tooling and logs (`merge_lora.py`, `quant_nvfp4a16.py`, the dry-run logs, `base-arm`, `lora-scales` — 8.6 MB total, the provenance for how every artifact above was built). ⚠ **`gx10:~/erp-tune/relaunch-trial-seat.sh` now names a model that is gone.** It was not deleted — its flags carry the FlashInfer JIT/PATH trap and the gpu-clear/never-pkill notes, each bought with a real failure — but it gained a RETIRED banner so nobody hits a confusing missing-model error later. **The live seat never bounced.** After the deletion `Pfish-6` answered a real completion (`finish_reason: stop`, 5 tokens, correct text), container still `Up 4 hours (healthy)`. ## 7. Adapter disposition — settled, and made real Operator, 2026-09-09: **keep the adapter.** As of 22:30 PT all five gx10-resident ERP adapters are mirrored to `ana-ml2:/tank/erp-tune/run-/adapter`, matching the layout runs 01–03 already use there, byte-total identical on both sides and `sha256` matching on every `adapter_model.safetensors`: run-03c run-04 run-05 run-06 run-07 315 MB each, 8 files each `/tank/*` is deliberately **excluded** from ana-ml2's restic sources — terabytes of regenerable model weights. A trained adapter is the one thing under there upstream cannot hand back, so `configs/restic/ana-ml2/profiles.yaml` now carries a single documented carve-out, `/tank/erp-tune/run-*/adapter`, verified by `resticprofile --dry-run` to expand to exactly those eight paths and nothing else. The nightly 01:00 run picks them up.