esh-pfi-infrastructure/docs/pfi/author-voice-lora-regime.md

# Author-voice LoRA regime on pfi-gx10 — training-side prep

_Written 2026-09-09 22:45 PT, revised 23:05 PT. Status: **PREP. Nothing is
training and nothing is queued.** Both operator decisions from the first draft are
now **SETTLED** (§6): carrier family is the **dense `Qwen3` line**, and the
intermediate checkpoints are purged._

The research target is **brokkr-smithy R49** (`research/R49-author-voice-adapters/`),
whose seed is the operator's **BabyBronte** design doc
(`R49/reference/babybronte-design-doc.md`, 2026-08-25). R49 owns the hypotheses,
the instruments and the adjudication. **This document owns the other half — the
box, the stack, the corpus staging, the trainer, the launcher, and the
wall-clock** — the same split that ran ERP-seat runs 3c through 7 on this box.

Read `R49/target.md` before touching the corpus design; several attractive ideas
are already settled *against* there and re-proposing them is the failure mode
this file exists to prevent.

---

## 1. What the regime is, in one paragraph

A voice is a **LoRA adapter on a small non-instruct base model**, trained on that
author's real prose, steered at serving time by a terse beat line rather than a
prompt. The completion in every training pair is 100% authentic author text; the
only synthetic token in the corpus is the beat line, and beat lines are
**loss-masked**, so the distribution the adapter learns to *emit* is the author's
and the style ceiling is the author's own. Names and places are substituted
**deterministically** — seeded RNG over a curated 23,398-name dictionary, never
an LLM — which is what lets one work become 5–8 training copies without teaching
plot memorisation.

**The value being bought is marginal cost per voice, not inference latency.**
That framing is the operator's and it is load-bearing: the anchor for
"expensive" is ERP run 2 at ~7 h for a single 26B-A4B tune on a harness whose
audit found a blocking-mask defect, a vision tower a leaf-name regex would have
trained on text, and 128 experts fused per layer. A dense sub-2B carrier has
none of those failure modes, and at this size the **methodology floor is finally
cheaper than the shortcut** — two seeds per arm and a re-run after every change
are routine rather than unaffordable.

## 2. The box, and what is already staged

`pfi-gx10` (10.100.50.60) — ASUS Ascent GX10, NVIDIA **GB10**, `sm_121`,
aarch64, **121 GB unified** memory, 916 GB NVMe, 470 GB free. Operator ruling
2026-09-09: **experimental box, primarily for training, no serving seat.** Its
GPU is idle.

Training stack already present and current at `/home/infra-ops/ml/.venv`:

| | |
|---|---|
| torch | `2.14.0+cu130`, `torch.cuda.get_device_capability() == (12, 1)` |
| transformers | 5.16.1 (loads `Qwen3_5Config` natively) |
| peft / trl / accelerate | 0.20.0 / 1.12.0 / 1.14.0 |
| datasets / bitsandbytes | 5.0.1 / 0.50.2 |
| **absent** | `mamba_ssm`, `causal_conv1d`, `fla`, `flash_attn`, `kernels` (triton 3.8.0 present) |

Staged on local NVMe under `/home/infra-ops/carriers/` (pulled 2026-09-09,
existence API-verified against the HF registry first, with a phantom repo run as
the negative control):

    CHOSEN — dense Qwen3          held, not chosen — hybrid Qwen3.5
    Qwen3-0.6B-Base   1.2G        Qwen3.5-0.8B-Base   1.7G
    Qwen3-1.7B-Base   3.3G        Qwen3.5-2B-Base     4.3G
    Qwen3-4B-Base     ~8G         Qwen3.5-4B-Base     8.8G

Probes live at `scripts/training-probes/{probe_carrier.py,bench_lora_step.py}`
with raw output in `bench-lora-step-gx10-2026-09-09.jsonl`, so every number below
can be re-derived rather than taken on faith.

## 3. What the carriers actually are — measured, not read off the model card

R49 H02 names the `Qwen3.5` trio. Probing the checkpoints rather than the config
found three things worth knowing before writing a recipe.

**They ship a vision tower and an MTP head.** `model.visual.*` is 153 tensors on
the 0.8B and 297 on the 2B — `attn.qkv`, `attn.proj`, `mlp.linear_fc1/2`, all
`nn.Linear` leaves that `target_modules="all-linear"` would attach LoRA to and
then train on pure text. This is the *same* defect the ERP harness audit caught
on gemma-4. **Mitigation is free:** loading through `AutoModelForCausalLM`
returns `Qwen3_5ForCausalLM` with the vision tower and MTP head dropped
entirely — 0.752 B of text model, module paths `model.layers.N.*`.

⚠ **That mitigation creates a serving trap.** vLLM will load the full
`Qwen3_5ForConditionalGeneration`, where the same weights live at
`model.language_model.layers.N.*`. An adapter trained against the CausalLM
prefix may not bind. **Pre-flight:** load the finished adapter in the serving
path and confirm a sampled target tensor actually changed — the same silent-no-op
check the ERP merge step already uses.

**Three quarters of the layers are not attention.** `layer_types` is 3×
`linear_attention` + 1× `full_attention` repeating: 18 SSM / 6 attention at
0.8B and 2B, 24 / 8 at 4B. The SSM blocks carry `conv1d`, `A_log`, `dt_bias` and
five Linear projections; `mamba_ssm_dtype` is `float32`.

| region | 0.8B | 2B | 4B | LoRA-able leaves |
|---|---|---|---|---|
| MLP | 35.1% | 48.1% | 53.9% | `gate_proj`, `up_proj`, `down_proj` |
| embeddings (tied) | 33.8% | 27.0% | 15.1% | `lm_head` — exclude |
| linear-attn (SSM) | 25.2% | 20.1% | 24.0% | `in_proj_{qkv,a,b,z}`, `out_proj` |
| full attention | 5.9% | 4.7% | 7.0% | `q_proj`, `k_proj`, `v_proj`, `o_proj` |
| **total** | **0.752 B** | **1.882 B** | **4.206 B** | |

The conventional `q,k,v,o` + MLP recipe therefore covers **41%** of the 0.8B and
leaves the SSM stack untouched in 18 of 24 layers. Adding the SSM Linears takes
coverage to 66%. At this scale that is a cheap ablation, not an agonising choice.

⚠ **Packing across document boundaries is unsafe on this architecture.** An SSM
layer carries recurrent state along the sequence and an attention mask does not
reset it, so two renamed copies packed into one 8k window can bleed in 18 of 24
layers — which is precisely the per-copy name-consistency invariant the design
doc calls sacred. Either one document per sequence, or prove the trainer's
sequence-boundary signal is honoured by the linear-attn path. Under a dense
carrier this problem does not exist.

**The tied embedding is a third of the small carrier.** vocab 248,320 × hidden
1024 = 254 M of the 0.752 B. The transformer body being tested at the small end
is ~0.50 B, which matters when reporting "the carrier floor".

## 4. Throughput — and the newest carrier is the slow one

One forward+backward+AdamW microbatch, LoRA r=32/α=64 on `q,k,v,o` + MLP,
bf16, `sdpa`, gradient checkpointing on, seq 4096, on gx10's GB10. n=10
measured after 3 warmup steps; median reported with the full spread.

| carrier | architecture | params | s/step | tok/s | peak | spread |
|---|---|---|---|---|---|---|
| `Qwen3.5-0.8B-Base` | hybrid, 18 SSM / 6 attn | 0.765 B | 7.581 | **540** | 15.1 GiB | 2.6% |
| `Qwen3.5-0.8B-Base` (no grad-ckpt) | " | 0.765 B | 6.364 | 644 | 38.9 GiB | 1.5% |
| `Qwen3-0.6B-Base` | dense | 0.616 B | 1.707 | **2,399** | 9.8 GiB | 0.6% |
| `Qwen3-1.7B-Base` | dense | 1.755 B | 2.895 | **1,415** | 12.2 GiB | 0.8% |
| `Qwen3-4B-Base` | dense | 4.089 B | 5.714 | **717** | 17.2 GiB | 0.3% |
| `Qwen3-1.7B-Base`, batch 4 | dense | 1.755 B | 11.387 | 1,439 | 38.0 GiB | 0.6% |
| `Qwen3.5-0.8B-Base`, batch 4 | hybrid | 0.765 B | 30.030 | 546 | 55.5 GiB | 0.7% |

**The dense 1.755 B carrier trains 2.6× faster than the hybrid 0.765 B one** — on
2.3× the parameters, with *more* LoRA modules adapted (196 vs 96, because dense
has real attention in every layer). Per parameter the dense path is ~6× more
efficient. Spreads of 0.6–2.6% across n=10 put the instrument's noise an order of
magnitude below the effect, so this is not variance.

The cause is almost certainly that **no fused linear-attention kernel is
installed** (§2) so the SSM path runs a reference implementation. Grad
checkpointing is *not* the culprit — turning it off recovers only 19% and costs
2.6× the memory, so leave it on. Batching is not the lever for either family: 1,415 → 1,439
tok/s dense and 540 → 546 tok/s hybrid from batch 1 to 4. **Both architectures
are already at this box's roofline at batch 1**, which is a bandwidth story
(GB10's unified LPDDR5X against an RTX PRO 6000's ~6.6× higher figure) — and it
means the 2.6× gap is the kernel path, not a batching artefact.

**What that does to the regime's premise.** Projecting a Brontë-scale corpus
(~1 M words ≈ 1.3 M tokens, × 6 rename copies, 3 epochs ≈ 23 M tokens):

| carrier | projected wall-clock per voice |
|---|---|
| `Qwen3-0.6B-Base` dense | **2.7 h** |
| `Qwen3-1.7B-Base` dense | **4.6 h** |
| `Qwen3-4B-Base` dense | **9.1 h** |
| `Qwen3.5-0.8B-Base` hybrid | **12 h** |

The sharpest way to put it: **the dense 4.089 B carrier still trains 33% faster
than the hybrid 0.765 B one**, on 5.3× the parameters. The full three-arm dense
sweep at two seeds each is ~33 h of GPU — about a day and a half for the whole
H02 carrier question, and ~10 h if H03's ~300 k-word corpus floor holds.

The hybrid carrier would make a per-voice run *longer than the 7 h 26B-A4B tune
it exists to replace.* At R49 H03's hoped-for corpus floor (~300 k words) the
dense 1.7B lands near **1.4 h** — a voice per afternoon, which is the regime the
operator asked for. ⚠ These are projections from a synthetic-token throughput
harness, not from a completed run; treat them as sizing, and re-measure on the
first real corpus.

## 5. Prep remaining, in order

1. ~~Carrier family decision~~ — **settled: dense `Qwen3`** (§6a).
2. **Corpus D1** — Gutenberg Brontë (Jane Eyre, Villette, Shirley, The
   Professor), boilerplate stripped, chapter-segmented, typography normalised,
   character inventory recorded. Public domain, clean under any disposition.
3. **Re-point the R49 deterministic machinery at Brontë.** The entity detector
   (corpus-level capitalised-vs-lowercase ratio), identity linking, gender
   resolution and the 23,398-name dictionary were all built and hardened against
   a *Yarros* sample. Per-work re-derivation needed: entity map, alphabet, and
   the `UNRESOLVED_BLOCKING` human pass (~20–40 entities per work).
4. **Beat annotation (D4)** via `gen` inverse-prompting, using F02's hardened
   prompt (banned meta-language, three PD worked examples, ≤20-word gate).
5. **Trainer.** `erp_sft_harness` is chat-shaped and carries ERP-specific
   eligibility machinery; the author-voice job is plain continuation with a
   masked prefix. Decision: a small purpose-built trainer that *keeps* the
   harness's §4 disciplines (provenance pin, order manifest, truncation report,
   cache key that sees semantic changes, recorded attention backend) rather than
   a fork of its corpus logic.
6. **Pre-set the decision threshold before collecting data**, wider than the
   measured seed-to-seed spread, per the R49 charter — and run the positive
   control the R49 journal's own lesson demands: confirm the stylometric
   instrument separates real Brontë from unadapted base output *before* it is
   asked to judge an adapter.
7. **Launcher** modelled on `launch-run-07.sh` — its guards were each bought with
   a past failure (GPU-clear assertion, pidfile not `pgrep -f`, refuse an
   existing log, free-space floor, `setsid` detach).

Deliberately out of scope here, per R49: the Director/critic loop, style
arithmetic and the Pelican test, multi-LoRA arsenal serving, the Skaldsong
integration contract, the modern in-copyright arsenal, inference latency.

## 6. Open for the operator

**(a) Carrier family — SETTLED 2026-09-09, operator: _"use dense qwen3"._** The
sweep is **`Qwen3-{0.6,1.7,4}B-Base`** — the design doc's own original pin, and
the family the measurements favour by 2.6–6×. **This overrides R49 H02's stated
arms**, which name `Qwen3.5-{0.8,2,4}B-Base`; brokkr-smithy owns that file and
has been told directly.

What the ruling buys, restated so it is not re-litigated: no vision tower and no
MTP head to exclude, cross-document packing is safe again, `sdpa`/flash are both
reachable, and the per-voice wall-clock is 2.7–4.6 h instead of 12 h. What it
costs: one model generation of base quality. **Reversible** — the three Qwen3.5
checkpoints stay staged (14.8 GB), and an `fla` install (pure Triton, plausibly
fine on aarch64) could revive that family as a follow-up experiment rather than a
prerequisite. If anyone re-opens this, re-run `bench_lora_step.py` first; the
argument is a measurement, not a preference.

**Is there a newer small base to prefer instead? No — checked against the HF
registry 2026-09-09, prompted by the operator asking brokkr-smithy the same
question.** Three facts, and together they close it:

1. **No official Qwen3.6 or Qwen3.8 exists below 27B.** The whole `Qwen/` listing
   is `Qwen3.6-{27B, 35B-A3B}` and `Qwen3.8-{27B, 2.4T-A95B, Flash-Next}`.
2. **Neither family publishes a `-Base` checkpoint at all.** Every Base newer than
   Qwen3 is Qwen3.5 — `Qwen3.5-{0.8B, 2B, 4B, 9B, 35B-A3B}-Base`. Since the
   regime requires a *non-instruct* carrier, that alone rules the 3.6/3.8 lines
   out regardless of size.
3. **The 27Bs are the same kernel path one size up.** `Qwen3.6-27B` and
   `Qwen3.8-27B` both report `model_type: qwen3_5`,
   `Qwen3_5ForConditionalGeneration`, 64 layers as **16 full + 48
   linear-attention**, vision tower present — the exact shape measured slow above.

So the newest official small **dense** Qwen base is still the `Qwen3` line, and
the ruling is not a compromise against a better available option; it is the only
dense option. (Third-party `Qwen3.8-*-Distill` checkpoints are Qwen3.5 hybrids
distilled on 3.8 outputs — same kernel path, and unpinned provenance besides.)

⚠ **Headroom worth knowing about, though H02 does not need it today:** the dense
Base line continues past 4B — `Qwen3-8B-Base` and `Qwen3-14B-Base` both exist. H02
caps at 4B by design ("sub-4B carrier"), and the projections say 4B already costs
9.1 h a voice, so this is not a proposal. It is insurance: if the 0.6/1.7/4B curve
has *not* flattened at 4B, the sweep can be extended without changing family.
(A `Qwen/SAE-Res-*-Base-*` row in a registry search is an interpretability
sidecar, not a carrier — ignore those.)

**(b) Intermediate checkpoints — PURGED 2026-09-09 23:00 PT, operator:
_"purge intermediate checkpoints"._** Seven `checkpoints/` directories deleted
with literal paths, one `rm` per line, after confirming none was a symlink and
that every run's final `adapter/` is an independent real directory:

    pfi-gx10   run-03c 11G · run-04 16G · run-05 9.2G · run-06 9.2G   -> 45 GB
    ana-ml2    run-01 12G · run-02 12G · run-03 5.9G                  -> 29 GB

gx10 419G→374G used (496 GB free); `/tank/erp-tune` 392G→363G, with `zfs list -t
snapshot` empty so the space is genuinely returned rather than snapshot-held.
**All eight adapters re-verified by `sha256` after the deletion**, matching the
values recorded during the mirror.

**And the merged models followed at 23:08 PT**, operator: _"purge the merged
models, keep run06 and the v6 quant."_ Eleven merges, literal paths, one `rm` per
line, after proving what the live seat reads: `docker inspect vllm-erp-seat` shows
it binds **`/tank/aimodels` only** and its model arg is
`/tank/aimodels/erp-tune-v6-nvfp4a16`, so `/tank/erp-tune/serve` was never in the
serving path at all. No container mounts it and no process held a file open under
it (checked `/proc/*/fd` and `/proc/*/maps`, not `lsof` alone).

    pfi-gx10   merged-run03c · merged-run04 · merged-run05      3 x 49 GiB -> 145 GB
    ana-ml2    merged-final · merged-run02 · merged-run03 ·
               merged-run03-s{025,050,075} · merged-test ·
               nvfp4a16-test                                    8 dirs     -> 354 GB

All eleven were Gemma-4 merges from the run-1/2/3 era (Aug 24–26); the keeper is
dated Sep 8 and lives in a different tree, so there was no ambiguity about which
was which. gx10 382G→237G used (**632 GB free**); `/tank/erp-tune/serve` 354G→8.6M
with `zfs list -t snapshot` empty, and the pool's raw FREE moved 4.80T→5.30T.

**Kept, and verified after the deletion:** `gx10:~/erp-tune/serve/merged-run06`
(49 GiB, 9 files), `/tank/aimodels/erp-tune-v6-bf16` (49 G) and
`erp-tune-v6-nvfp4a16` (16 G, 9 files incl. `recipe.yaml`), all eight adapters
sha256-unchanged, and `/tank/erp-tune/serve`'s merge/quant tooling and logs
(`merge_lora.py`, `quant_nvfp4a16.py`, the dry-run logs, `base-arm`, `lora-scales`
— 8.6 MB total, the provenance for how every artifact above was built).

⚠ **`gx10:~/erp-tune/relaunch-trial-seat.sh` now names a model that is gone.** It
was not deleted — its flags carry the FlashInfer JIT/PATH trap and the
gpu-clear/never-pkill notes, each bought with a real failure — but it gained a
RETIRED banner so nobody hits a confusing missing-model error later.

**The live seat never bounced.** After the deletion `Pfish-6` answered a real
completion (`finish_reason: stop`, 5 tokens, correct text), container still
`Up 4 hours (healthy)`.

## 7. Adapter disposition — settled, and made real

Operator, 2026-09-09: **keep the adapter.** As of 22:30 PT all five
gx10-resident ERP adapters are mirrored to `ana-ml2:/tank/erp-tune/run-<N>/adapter`,
matching the layout runs 01–03 already use there, byte-total identical on both
sides and `sha256` matching on every `adapter_model.safetensors`:

    run-03c  run-04  run-05  run-06  run-07     315 MB each, 8 files each

`/tank/*` is deliberately **excluded** from ana-ml2's restic sources — terabytes
of regenerable model weights. A trained adapter is the one thing under there
upstream cannot hand back, so `configs/restic/ana-ml2/profiles.yaml` now carries
a single documented carve-out, `/tank/erp-tune/run-*/adapter`, verified by
`resticprofile --dry-run` to expand to exactly those eight paths and nothing
else. The nightly 01:00 run picks them up.