Operator: "purge the merged models, keep run06 and the v6 quant." Eleven merges
removed with literal paths, one rm per line.
pfi-gx10 merged-run03c, merged-run04, merged-run05 3 x 49 GiB -> 145 GB
ana-ml2 merged-final, merged-run02, merged-run03,
merged-run03-s{025,050,075}, merged-test,
nvfp4a16-test 8 dirs -> 354 GB
gx10 382G->237G used, 632 GB free. /tank/erp-tune/serve 354G->8.6M with no
snapshots holding the blocks; the pool's raw FREE moved 4.80T->5.30T. Combined
with the earlier checkpoint purge, ~573 GB reclaimed tonight.
The check that made this safe: docker inspect on the live vllm-erp-seat shows it
binds /tank/aimodels ONLY, with model arg /tank/aimodels/erp-tune-v6-nvfp4a16, so
/tank/erp-tune/serve was never in the serving path. Also confirmed no container
mounts that tree and no process held a file open under it, reading /proc/*/fd and
/proc/*/maps rather than trusting an empty lsof. All eleven were run-1/2/3-era
Gemma-4 merges dated Aug 24-26; the keeper is dated Sep 8 and lives in a different
tree, so there was no ambiguity about which was which.
Kept and re-verified after the deletion: gx10 serve/merged-run06,
/tank/aimodels/erp-tune-v6-{bf16,nvfp4a16}, all eight adapters sha256-unchanged,
and the merge/quant tooling and logs under /tank/erp-tune/serve (8.6 MB) that
document how the artifacts were built.
The live seat never bounced -- Pfish-6 answered a real completion after the delete
with finish_reason stop and correct text, container still healthy at 4 h uptime.
relaunch-trial-seat.sh now names a deleted model; it was banner-marked RETIRED
rather than removed, because its flags carry the FlashInfer JIT/PATH trap and the
gpu-clear / never-pkill notes.
The operator asked brokkr-smithy to look for a smaller Qwen3.6/3.8; brokkr reported
none exists and that both 27Bs are the same hybrid kernel path one size up. Verified
independently against the HF API rather than adopted, and it holds -- with two facts
brokkr's note did not carry.
1. No official Qwen3.6 or Qwen3.8 below 27B. The listing is Qwen3.6-{27B, 35B-A3B}
and Qwen3.8-{27B, 2.4T-A95B, Flash-Next}.
2. Neither family publishes a -Base checkpoint AT ALL. Every Base newer than Qwen3
is Qwen3.5. Since the regime requires a non-instruct carrier, that rules the
3.6/3.8 lines out on its own, independent of size.
3. Qwen3.6-27B and Qwen3.8-27B both report model_type qwen3_5, 64 layers as 16 full
+ 48 linear-attention, vision tower present -- confirmed as stated.
So the dense Qwen3 ruling is not a compromise against a better available option; it
is the only dense Base option. Also recorded as insurance, not a proposal:
Qwen3-8B-Base and Qwen3-14B-Base exist, so the sweep can extend past 4B without
changing family if the carrier curve has not flattened there.
Two operator rulings, 2026-09-09.
"purge intermediate checkpoints" -- seven checkpoints/ directories removed with
literal paths, one rm per line, after confirming none was a symlink and that
every run's final adapter/ is an independent real directory:
pfi-gx10 run-03c 11G run-04 16G run-05 9.2G run-06 9.2G = 45 GB
ana-ml2 run-01 12G run-02 12G run-03 5.9G = 29 GB
gx10 419G->374G used, 496 GB free. /tank/erp-tune 392G->363G with zfs list -t
snapshot empty, so the space is genuinely returned rather than snapshot-held. All
eight adapters re-verified by sha256 after the deletion, matching the values
recorded during the mirror. Merged artifacts deliberately untouched -- they are
not checkpoints, and the ~550 GB of superseded merges stays a separate call.
"use dense qwen3" -- the R49 H02 carrier sweep becomes Qwen3-{0.6,1.7,4}B-Base,
which overrides the Qwen3.5 arms H02 names; brokkr-smithy owns that file and was
told directly. Qwen3-4B-Base staged and benched to complete the family:
Qwen3-0.6B-Base 0.616 B 1.707 s/step 2,399 tok/s spread 0.6%
Qwen3-1.7B-Base 1.755 B 2.895 s/step 1,415 tok/s spread 0.8%
Qwen3-4B-Base 4.089 B 5.714 s/step 717 tok/s spread 0.3%
The dense 4.089 B carrier still trains 33% faster than the hybrid 0.765 B one.
Projected per voice 2.7 / 4.6 / 9.1 h; the three-arm sweep at two seeds is ~33 h
of GPU, ~10 h if H03's corpus floor holds. The three Qwen3.5 checkpoints stay
staged so the decision is reversible behind an fla install.
Also recorded: verified at 22:45-22:48 PT that nothing is training on gx10,
ana-ml2, nh3-dev or irv-ml1, and that brokkr's own run07-gate close states
"Nothing is owed. No battery to run." Run 7 has no servable artifact left. And a
correction to a standing lesson -- the bracketed-class trick does not defeat a
wrapper's argv, since the invoking shell's command line carries the literal
pattern; observe the artifact instead.
Prep for the BabyBronte / brokkr-smithy R49 author-voice adapter regime, plus
the operator's "keep the adapter" ruling made durable.
Measured on pfi-gx10 (GB10, sm_121), n=10 per arm after 3 warmup steps, seq
4096, LoRA r=32 on q/k/v/o + MLP, bf16, sdpa, grad-checkpointing on:
Qwen3-0.6B-Base dense 0.616 B 1.707 s/step 2,399 tok/s
Qwen3-1.7B-Base dense 1.755 B 2.895 s/step 1,415 tok/s
Qwen3.5-0.8B-Base hybrid 0.765 B 7.581 s/step 540 tok/s
The dense 1.755 B carrier trains 2.6x faster than the hybrid 0.765 B one on 2.3x
the parameters (~6x per parameter), with more LoRA modules adapted (196 vs 96).
Spreads of 0.6-2.6% put instrument noise an order of magnitude below the effect.
Cause: Qwen3.5 is 18 linear-attention (SSM) layers to 6 attention, and no fused
linear-attention kernel is installed on the box. Grad checkpointing is not the
culprit (19%, and saves 2.6x memory). Batching is not the lever for either
family -- both sit at this box's roofline at batch 1.
Projected per voice on a Brontë-scale corpus: dense 0.6B 2.7 h, dense 1.7B
4.6 h, hybrid 0.8B 12 h. The hybrid would take longer than the 7 h 26B-A4B tune
the regime exists to replace, so the carrier family is now an open decision with
a recommendation for the dense Qwen3 line -- the design doc's original pin.
Two further Qwen3.5 findings, both measured rather than read off the config: the
Base checkpoints ship a vision tower (153/297 model.visual.* Linear tensors that
target_modules="all-linear" would train on text) and an MTP head, both dropped
for free by loading through AutoModelForCausalLM -- which renames modules
relative to the vLLM serving path, so adapter binding needs the
sampled-target-changed check on the serving side; and cross-document packing is
unsafe because SSM state ignores the attention mask, breaking the per-copy
name-consistency invariant the design doc calls sacred. Neither exists on dense.
Adapter disposition, per the operator's ruling: all five gx10-resident ERP
adapters (run-03c/04/05/06/07) mirrored to ana-ml2:/tank/erp-tune/run-<N>/adapter
matching the layout runs 01-03 already used, byte-totals identical both sides and
sha256 matching on every adapter_model.safetensors. /tank/* is deliberately
excluded from ana-ml2's restic sources, so the profile gains one documented
carve-out for /tank/erp-tune/run-*/adapter, verified by resticprofile --dry-run
to expand to exactly those eight paths.
Nothing is training and nothing is queued.
vh