feat(erp-seat): erp-tune-v6-nvfp4a16 quantized (data-free W4A16, ~90 s) and serving on ana-ml2 GPU1 :8021; playbook §3.16 (data-free NVFP4A16 still bakes the tokenizer cap); memory: gate state, brokkr after-window asks, ana-ml2 non-persistent mesh routes

2026-09-08 22:14:09 -07:00
parent 911ff20356
commit 8512dd4d31
2 changed files with 34 additions and 0 deletions
@@ -457,6 +457,20 @@ fallback, the incumbent-vs-candidate A/Bs (47.2% acceptance, PPL 6.910, and the
 2026-08-20 Heretic-300 build) are apples-to-apples. This is unrealised upside, not a
 correction to past numbers.

+### 3.16 Weight-only NVFP4A16 with a minmax observer is DATA-FREE — your calibration corpus is ignored, but its tokenizer side-effect is not
+
+Measured 2026-09-08 (Gemma-4 26B-A4B MoE, ERP run 6, llm-compressor 0.13): with
+`scheme="NVFP4A16"` (default `memoryless_minmax` weights, no activation quant) llm-compressor
+logs `Inferred DataFreePipeline for QuantizationModifier` and never touches the dataset — the
+whole 26B quant ran in ~90 s on one Blackwell. Two consequences: (1) do not budget calibration
+time or believe a corpus "shaped" the result — only `imatrix_mse`/activation observers consume
+data; (2) building the calibration set still calls the fast tokenizer with
+`truncation=True, max_length=N`, so §3.14's baked cap (`max_length: 8192` here) lands in the
+saved `tokenizer.json` **even though no calibration happened**. The §4.3 post-step caught it.
+Reference: `services/erp-seat-quant/quant_nvfp4a16_gemma4_moe.py` (linearize_moe + assert
+11,520 expert Linears + post-steps; the published `prithivMLmods/gemma-4-26B-A4B-it-NVFP4A16`
+recipe replicated, 222→252 ignore entries with audio/norm/router regexes added).
+
 ### 3.14 ⭐⭐ Calibration BAKES a truncation cap into the shipped tokenizer

 **Symptom (on a newer transformers, at startup, on a vision model):**