feat(heretic2-nvfp4): WORKING modelopt NVFP4+MTP seat + full recipe runbook

The fast char-rp-reasoning seat works: ~77 tok/s (vs GGUF ~59.5, base NVFP4 ~53),
MTP draft-acceptance 32-40%, mean acceptance length 2.19. Same Heretic2/NEO-CODE
model, NVFP4 + native qwen3_5_mtp spec-decode.

Full end-to-end recipe + the four landmines in docs/runbooks/heretic2-nvfp4-mtp-seat.md:
(1) load as AutoModelForImageTextToText not AutoModelForCausalLM (namespace/gibberish);
(2) modelopt format not compressed-tensors (compressed-tensors MTP = 0% accept);
(3) modelopt 0.45 <-> transformers 5.12.1 FusedMoE crash (guarded in quant_modelopt.py);
(4) vLLM 0.24.0 does NOT propagate modelopt exclude_modules to the spec-decode draft
model -> BF16 mtp head gets quantized -> shape crash; no checkpoint config fixes it
(is_layer_skipped is exact-membership not glob) -> fix is a mounted sitecustomize that
force-skips mtp.* in is_layer_skipped (upstream vLLM bug to report).

Scripts: quant_modelopt.py (FusedMoE guard + single-shard export + multimodal load),
finalize_modelopt_mtp.py (splice bf16 mtp), serve_modelopt_mtp.sh, run_quant_modelopt.sh,
sitecustomize-mtp-workaround.py.

persistent-memory.md

@@ -104,7 +104,34 @@ no longer deployed sidecars here. See Recent decisions.)
 
 _As of 2026-07-14 — ONE active task: the NVFP4 fast char-rp-reasoning seat. GIBBERISH RESOLVED (quant-namespace bug — NVFP4 now serves coherent) and the speed premise DISPROVEN on the llm-compressor/compressed-tensors format (base NVFP4 ≈ GGUF at batch-1; MTP 0%-accept). PIVOTED (operator) to a **modelopt-format re-quant for working MTP** — scoped + de-risked, needs `quant_modelopt.py` + one more GPU0 window (see ★ section). Everything else this session LANDED: the char-rp-reasoning Deckard→NEO-CODE swap (#355 resolved), the Worldtree deploy-speed PR, and the soong-lab webhook fix — see Recent decisions + git; the old #355/deploy-speed detail below is kept as history (both DONE)._
 
-### ✅ RESOLVED (gibberish) → ⏭ PIVOT: modelopt-format re-quant for working MTP
+### ✅✅ DONE — modelopt NVFP4 + MTP fast char-rp-reasoning seat WORKS (2026-07-14)
+
+**WORKING at ~77 tok/s** (vs GGUF NEO-CODE ~59.5, base NVFP4 ~53 → **~1.3× over GGUF**), MTP
+draft-acceptance **32–40%**, mean acceptance length **2.19**. Same Heretic2/NEO-CODE model, NVFP4
++ native MTP. **★ FULL RECIPE + all gotchas: `docs/runbooks/heretic2-nvfp4-mtp-seat.md`** (the
+fire-drill-killer the operator demanded). Scripts: eshpfi `services/heretic2-nvfp4-quant/`
+(`quant_modelopt.py`, `finalize_modelopt_mtp.py`, `serve_modelopt_mtp.sh`,
+`sitecustomize-mtp-workaround.py`). Seat artifact: ana-ml2
+`/tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4-mtp`.
+
+**The four landmines (each cost hours — full detail in the runbook):** (1) load as
+`AutoModelForImageTextToText` not `AutoModelForCausalLM` (namespace → else `!!!!`); (2) MODELOPT
+format not compressed-tensors (compressed-tensors MTP = 0% accept; base NVFP4 alone ≈ GGUF, MTP is
+the whole win); (3) modelopt 0.45↔transformers 5.12.1 FusedMoE crash — guarded in `quant_modelopt.py`;
+(4) **⭐ vLLM 0.24.0 does NOT propagate modelopt `exclude_modules` to the spec-decode DRAFT model** →
+the BF16 mtp head gets quantized → shape crash; **no checkpoint config fixes it** (`is_layer_skipped`
+uses exact membership not glob, and the drafter never sees the mtp excludes) → **fix = a mounted
+`sitecustomize` that force-skips `mtp.*` in `is_layer_skipped`** (report upstream as a vLLM bug).
+
+**Operator's memory was right:** MTP ran before on the MoE `gen` (qwopus-122B +12% single-stream,
+archival 2026-07-01) — dropped for `gen` because MTP HURTS concurrency; it belongs on the
+single-stream RP seats. **NOT-YET-DONE (productionization):** compose stack with the workaround baked
+in, brokkr P00 (soong 9-tool k5 — same base as GGUF so R36 should carry, confirm NVFP4-vs-Q5 quality),
+gateway `char-rp-reasoning` repoint, wt-dev heads-up, file the vLLM upstream bug.
+
+---
+
+### (historical) RESOLVED (gibberish) → PIVOT: modelopt-format re-quant for working MTP
 
 **2026-07-14 WINDOW OUTCOME (this session).** Ran the diagnostic ladder in one clean ~40-min GPU0 window.
 - **GIBBERISH ROOT CAUSE = quant NAMESPACE (found from config diffs + vLLM source, ZERO GPU time).** `quant_nvfp4.py` loaded via `AutoModelForCausalLM` → resolves qwen3_5 to the text-only `Qwen3_5ForCausalLM` → weight keys born flat `model.layers.*` (no vision). But vLLM 0.24 registers ONLY `Qwen3_5ForConditionalGeneration` (registry.py:566), whose `hf_to_vllm_mapper` (qwen3_vl.py:1692) remaps `model.language_model.*`→`language_model.model.*` and has NO rule for a bare `model.layers.` prefix → every transformer-layer weight fails to match → uninitialized → `!!!!`. The step-4 config-merge to ConditionalGeneration was a doomed patch over a wrong-namespace checkpoint. **FIX = load as `AutoModelForImageTextToText`** (resolves qwen3_5 → `Qwen3_5ForConditionalGeneration` → keys born `model.language_model.*` + `model.visual.*`, pantheon namespace). One-class swap; committed to `quant_nvfp4.py`.