esh-pfi-infrastructure/docs/pfi/gen-seat-candidate-eval-2026-08-21.md

Gen-seat candidate evaluation — 2026-08-21

Cold-Fusion was abandoned (see persistent-memory.md); the seat is on qwen38-27b-heresy-nvfp4-mixed. Two replacement candidates were put up. All facts below come from the HF registry and from reading the artifacts directly — the safetensors headers were fetched with HTTP Range requests, so the tensor census cost about a megabyte rather than a 20 GB download.

The candidates

	orcarouter/Qwen3.8-27B-Uncensored	preetpatel/…-NVFP4
what	BF16 source weights	NVFP4 quant of orcarouter
size	55.6 GB	19.7 GB
base	Qwen/Qwen3.8-27B (stock Qwen)	orcarouter
MTP tensors	15 ✓	0 ✗
visual tensors	333 ✓	333 ✓
scheme	n/a (bf16)	NVFP4 W4A4 ✗
re:^mtp.* in ignore	n/a	absent ✗
traction	3,278 dl / 60 likes	36 dl / 0 likes
gated	yes — our token already has access	no
chat template	sha c3cf9e34 — byte-identical to the live heresy seat	same

Verdict: preetpatel is disqualified, on two independent hard failures

1. Zero MTP tensors. Read straight from the safetensors header: 2,672 tensors, none matching mtp.*. The author's own recipe.yaml asks to ignore re:.*mtp.*, but the written config.json contains no mtp ignore entry at all — while re:.*visual.* expanded to 110 explicit entries. That asymmetry is the signature of llm-compressor pruning an ignore pattern that matched nothing, i.e. the MTP head was never loaded and never quantized. It is the same re:^mtp.*-pruning trap documented in the playbook, seen from the outside.

Cost: no speculative decoding. Our seat runs MTP at ~59% acceptance and 118 tok/s; without it, roughly half the decode throughput.

2. NVFP4 W4A4 — 4-bit activations. input_activations: num_bits 4, type float. This is precisely the AEON failure mode we spent a multi-day saga diagnosing and purging: the activation-fidelity gradient is W4A4 < W4+FP8 < W4+bf16, W4A4 was responsible for ~15-20% stochastic degeneration, and W4A4 collapses past ~30k context. The gen seat serves 262K.

Either failure alone would rule it out. It is also one day old with 36 downloads.

orcarouter checks out as a quant source

Stock-Qwen base (not a reasoning-compression finetune — the trait that sank Cold-Fusion), Arditi-et-al. single-direction abliteration, MTP and vision both explicitly preserved and verified at 15/333, chat template byte-identical to the build we are serving right now, and the gate is already accepted on our token.

Third option, noted and not recommended

orcarouter/Qwen3.8-27B-Uncensored-FP8 — 76,109 downloads, 693 likes, far more traction than either candidate. But 30.9 GB against NVFP4's 22 GB, and GPU0 is zero-sum with meromero co-resident: +9 GB of weights comes straight out of the KV pool, taking it from ~14.4 GiB / 403k tokens to roughly 5 GiB / ~150k — which breaks 262K context at 1.5x concurrency. Viable only if the seat gives up long context or meromero moves.

The imatrix constraint — read before committing to it

The operator asked for imatrix if we quant ourselves. This is not a switch.

quant_mixed_nvfp4.py already sets observer="imatrix_mse" on the W4A4 group and has never once used it — llm-compressor logs no importance data available. Falling back to uniform MSE and proceeds. Playbook §3.13 documents this and warns explicitly: do not "fix" it by assuming an imatrix would help; verify first that your llm-compressor version can consume an externally supplied importance matrix at all, and in what format. Parked as park/…imatrix-mse… (id 42) with the calibration corpus that would feed it.

Also note the W4A16 portions of the mixed recipe are data-free by construction — llm-compressor infers DataFreePipeline for weight-only quantization and ignores calibration data entirely. Imatrix can only ever bite on the W4A4 MLP group.

So "quant with imatrix" is two projects: an unscoped capability investigation, and then the ~2h quant. Recommendation is to decouple them — ship the proven recipe first, run imatrix as its own bounded experiment. Every A/B we hold is uniform-MSE-to-uniform-MSE, so a non-imatrix build stays directly comparable to heresy's PPL 6.910 / 47.2% acceptance.

Mandatory step if we pull

Run services/gen-seat-mixed-quant/bench/think-leak/think_prior.py on the bf16 before any GPU time. It is a ~10s CPU measurement and it is the gate that would have disqualified Cold-Fusion before its 300-trial study ever ran. Prior is favourable — stock-Qwen base, template identical to heresy, which measures <0.002 against Cold-Fusion's 0.185 — but measure, don't assume.

---

Addendum — M.O.G.-SEC pen-test model (same night)

Two Blackfrost-Research/M.O.G.-SEC-27B-1M-CTX candidates for the pen-test project: a BF16 and a pre-made NVFP4. Same verdict as gen-seat: pull the BF16, quant ourselves. Read directly off the artifacts via HTTP Range.

	BF16	pre-made NVFP4
MTP tensors	15 ✓	0 ✗
scheme	n/a	ModelOpt W4A4 ✗
context	native 262K (config), 1M claimed	same

The pre-made NVFP4 is disqualified on three grounds, one unique to this model: ModelOpt W4A4 (4-bit activations — the AEON degradation mode), zero MTP, and — the sharp one — W4A4 on a 1M-context model is self-defeating, since W4A4 fidelity collapses past ~30k. A long-context model quanted on the activation scheme that fails hardest at long context works against itself.

The BF16 quanted cleanly (mog-sec-27b-nvfp4-mixed, 23.4 GB) and is served in the retired fable slot (ana-ml2 GPU1 :8019, aliases mog-sec / mog-sec-reasoning). Gates: format screen 1.11e-05, surface 6/6, MTP 55.3%, vision 7/3/1, and a capability smoke 4/4 (it delivers offensive-security content, does not refuse).

The 1M is not real on our path. rope_scaling: None in the weights' config (native Qwen3.8 is 262K), and the repo's 1M is an SGLang/DFlash2 deployment kit. We serve native 262K. A true 1M seat would be a separate SGLang project — flagged, not attempted.