2 Commits

Author	SHA1	Message	Date
vh	aca45393c2	fix(heretic2-nvfp4): quant as ConditionalGeneration (namespace fix) + modelopt recipe for working MTP ... Root-caused the NVFP4 gibberish to a quant-namespace bug: quant_nvfp4.py loaded via AutoModelForCausalLM -> text-only Qwen3_5ForCausalLM -> flat model.layers.* keys, but vLLM 0.24 serves only Qwen3_5ForConditionalGeneration (whose weight mapper needs model.language_model.*). Fixed by loading as AutoModelForImageTextToText; NVFP4 now serves coherent (validated greedy on ana-ml2 GPU0). Base NVFP4 (compressed-tensors) measured ~53 tok/s (~= GGUF at batch-1, no single-stream win) and its MTP is 0% acceptance (vLLM's Qwen3_5MTP drafter loads the bf16 mtp head only off a modelopt main-model checkpoint). Added quant_modelopt.py (nvidia-modelopt PTQ, matches AEON's NVFP4 W4A4 g16 + lm_head/linear_attn/visual exclusions) as the path to working native MTP; graft + splice + serve otherwise unchanged.	2026-07-14 13:10:25 -07:00
vh	920f9a3709	feat(heretic2-nvfp4): MTP-graft + NVFP4 quant scripts + pipeline README (fire-ready) ... graft_mtp.py: grafts the 15 base-Qwen3.6 MTP tensors into Heretic2 BF16 (CPU-only). quant_nvfp4.py: llm-compressor NVFP4 (Linear only; GDN/vision/lm-head/norms/MTP kept BF16 per robbatt's deckard recipe + brokkr's spec); text (AEON-baseline) or chat (production, apply_chat_template renders qwen3_coder XML) calib modes. README: fire sequence + gates (GPU window, production calib) + artifacts. Spike gated only on: (1) off-peak Blackwell GPU window, (2) brokkr's production calib.	2026-07-14 08:57:25 -07:00