diff --git a/docs/pfi/author-voice-lora-regime.md b/docs/pfi/author-voice-lora-regime.md index 70a8c7a..bfb4927 100644 --- a/docs/pfi/author-voice-lora-regime.md +++ b/docs/pfi/author-voice-lora-regime.md @@ -220,6 +220,34 @@ fine on aarch64) could revive that family as a follow-up experiment rather than prerequisite. If anyone re-opens this, re-run `bench_lora_step.py` first; the argument is a measurement, not a preference. +**Is there a newer small base to prefer instead? No — checked against the HF +registry 2026-09-09, prompted by the operator asking brokkr-smithy the same +question.** Three facts, and together they close it: + +1. **No official Qwen3.6 or Qwen3.8 exists below 27B.** The whole `Qwen/` listing + is `Qwen3.6-{27B, 35B-A3B}` and `Qwen3.8-{27B, 2.4T-A95B, Flash-Next}`. +2. **Neither family publishes a `-Base` checkpoint at all.** Every Base newer than + Qwen3 is Qwen3.5 — `Qwen3.5-{0.8B, 2B, 4B, 9B, 35B-A3B}-Base`. Since the + regime requires a *non-instruct* carrier, that alone rules the 3.6/3.8 lines + out regardless of size. +3. **The 27Bs are the same kernel path one size up.** `Qwen3.6-27B` and + `Qwen3.8-27B` both report `model_type: qwen3_5`, + `Qwen3_5ForConditionalGeneration`, 64 layers as **16 full + 48 + linear-attention**, vision tower present — the exact shape measured slow above. + +So the newest official small **dense** Qwen base is still the `Qwen3` line, and +the ruling is not a compromise against a better available option; it is the only +dense option. (Third-party `Qwen3.8-*-Distill` checkpoints are Qwen3.5 hybrids +distilled on 3.8 outputs — same kernel path, and unpinned provenance besides.) + +⚠ **Headroom worth knowing about, though H02 does not need it today:** the dense +Base line continues past 4B — `Qwen3-8B-Base` and `Qwen3-14B-Base` both exist. H02 +caps at 4B by design ("sub-4B carrier"), and the projections say 4B already costs +9.1 h a voice, so this is not a proposal. It is insurance: if the 0.6/1.7/4B curve +has *not* flattened at 4B, the sweep can be extended without changing family. +(A `Qwen/SAE-Res-*-Base-*` row in a registry search is an interpretability +sidecar, not a carrier — ignore those.) + **(b) Intermediate checkpoints — PURGED 2026-09-09 23:00 PT, operator: _"purge intermediate checkpoints"._** Seven `checkpoints/` directories deleted with literal paths, one `rm` per line, after confirming none was a symlink and