docs(r49-prep): close the "is there a newer small base?" question against the registry
The operator asked brokkr-smithy to look for a smaller Qwen3.6/3.8; brokkr reported
none exists and that both 27Bs are the same hybrid kernel path one size up. Verified
independently against the HF API rather than adopted, and it holds -- with two facts
brokkr's note did not carry.
1. No official Qwen3.6 or Qwen3.8 below 27B. The listing is Qwen3.6-{27B, 35B-A3B}
and Qwen3.8-{27B, 2.4T-A95B, Flash-Next}.
2. Neither family publishes a -Base checkpoint AT ALL. Every Base newer than Qwen3
is Qwen3.5. Since the regime requires a non-instruct carrier, that rules the
3.6/3.8 lines out on its own, independent of size.
3. Qwen3.6-27B and Qwen3.8-27B both report model_type qwen3_5, 64 layers as 16 full
+ 48 linear-attention, vision tower present -- confirmed as stated.
So the dense Qwen3 ruling is not a compromise against a better available option; it
is the only dense Base option. Also recorded as insurance, not a proposal:
Qwen3-8B-Base and Qwen3-14B-Base exist, so the sweep can extend past 4B without
changing family if the carrier curve has not flattened there.
This commit is contained in:
@@ -220,6 +220,34 @@ fine on aarch64) could revive that family as a follow-up experiment rather than
|
||||
prerequisite. If anyone re-opens this, re-run `bench_lora_step.py` first; the
|
||||
argument is a measurement, not a preference.
|
||||
|
||||
**Is there a newer small base to prefer instead? No — checked against the HF
|
||||
registry 2026-09-09, prompted by the operator asking brokkr-smithy the same
|
||||
question.** Three facts, and together they close it:
|
||||
|
||||
1. **No official Qwen3.6 or Qwen3.8 exists below 27B.** The whole `Qwen/` listing
|
||||
is `Qwen3.6-{27B, 35B-A3B}` and `Qwen3.8-{27B, 2.4T-A95B, Flash-Next}`.
|
||||
2. **Neither family publishes a `-Base` checkpoint at all.** Every Base newer than
|
||||
Qwen3 is Qwen3.5 — `Qwen3.5-{0.8B, 2B, 4B, 9B, 35B-A3B}-Base`. Since the
|
||||
regime requires a *non-instruct* carrier, that alone rules the 3.6/3.8 lines
|
||||
out regardless of size.
|
||||
3. **The 27Bs are the same kernel path one size up.** `Qwen3.6-27B` and
|
||||
`Qwen3.8-27B` both report `model_type: qwen3_5`,
|
||||
`Qwen3_5ForConditionalGeneration`, 64 layers as **16 full + 48
|
||||
linear-attention**, vision tower present — the exact shape measured slow above.
|
||||
|
||||
So the newest official small **dense** Qwen base is still the `Qwen3` line, and
|
||||
the ruling is not a compromise against a better available option; it is the only
|
||||
dense option. (Third-party `Qwen3.8-*-Distill` checkpoints are Qwen3.5 hybrids
|
||||
distilled on 3.8 outputs — same kernel path, and unpinned provenance besides.)
|
||||
|
||||
⚠ **Headroom worth knowing about, though H02 does not need it today:** the dense
|
||||
Base line continues past 4B — `Qwen3-8B-Base` and `Qwen3-14B-Base` both exist. H02
|
||||
caps at 4B by design ("sub-4B carrier"), and the projections say 4B already costs
|
||||
9.1 h a voice, so this is not a proposal. It is insurance: if the 0.6/1.7/4B curve
|
||||
has *not* flattened at 4B, the sweep can be extended without changing family.
|
||||
(A `Qwen/SAE-Res-*-Base-*` row in a registry search is an interpretability
|
||||
sidecar, not a carrier — ignore those.)
|
||||
|
||||
**(b) Intermediate checkpoints — PURGED 2026-09-09 23:00 PT, operator:
|
||||
_"purge intermediate checkpoints"._** Seven `checkpoints/` directories deleted
|
||||
with literal paths, one `rm` per line, after confirming none was a symlink and
|
||||
|
||||
Reference in New Issue
Block a user