### verify_quant.py — new v2 heretic quant vs the 2026-08-21 known-good canonical quant $ sudo -n python3 verify_quant.py ====================================================================== /tank/aimodels/G4-MeroMero-v2-31B-heretic-NVFP4A16 group_0: weights num_bits=4 type=float strategy=tensor_group | input_activations=None (WEIGHT-ONLY) format=nvfp4-pack-quantized kv_cache_scheme=None status=compressed text_config: per_layer_config=absent head_dim=256 global_head_dim=512 num_key_value_heads=16 num_global_key_value_heads=4 tensor dtypes by family: embeddings BF16x1 language_model BF16x60, F32x410, F8_E4M3x410, U8x410 [820 packed/scale tensors] norms BF16x361 vision_tower BF16x356 ====================================================================== /tank/aimodels/meromero-v2-nvfp4-work/G4-MeroMero-v2-31B-NVFP4A16 group_0: weights num_bits=4 type=float strategy=tensor_group | input_activations=None (WEIGHT-ONLY) format=nvfp4-pack-quantized kv_cache_scheme=None status=compressed text_config: per_layer_config=absent head_dim=256 global_head_dim=512 num_key_value_heads=16 num_global_key_value_heads=4 tensor dtypes by family: embeddings BF16x1 language_model BF16x60, F32x410, F8_E4M3x410, U8x410 [820 packed/scale tensors] norms BF16x361 vision_tower BF16x356 ### post_quant_gemma4.py --check — v2 heretic output [CHECK] src=/tank/aimodels/G4-MeroMero-v2-31B-heretic-bf16 [CHECK] out=/tank/aimodels/G4-MeroMero-v2-31B-heretic-NVFP4A16 -- step 1: MTP graft N/A for Gemma-4 (no MTP head). mtp tensors in output index: 0; mtp entries in ignore list: 0 -- step 2: restore processor_config.json + preprocessor_config.json processor_config.json already present and identical to source preprocessor_config.json already present and correct -- step 4: confirm saved tokenizer.json has truncation: null truncation is null -- clean [CHECK] done rc=0 ### post_quant_gemma4.py --check — A4B output (after the truncation fix) [CHECK] src=/tank/aimodels/G4-MeroMero-26B-A4B-it-uncensored-heretic-bf16 [CHECK] out=/tank/aimodels/G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16 -- step 1: MTP graft N/A for Gemma-4 (no MTP head). mtp tensors in output index: 0; mtp entries in ignore list: 0 -- step 2: restore processor_config.json + preprocessor_config.json processor_config.json already present and identical to source preprocessor_config.json already present and correct -- step 4: confirm saved tokenizer.json has truncation: null truncation is null -- clean [CHECK] done rc=0 ### post_quant_gemma4.py --check — 2026-08-21 known-good tree (POSITIVE CONTROL, must be clean) [CHECK] src=/tank/aimodels/meromero-v2-nvfp4-work/src [CHECK] out=/tank/aimodels/meromero-v2-nvfp4-work/G4-MeroMero-v2-31B-NVFP4A16 -- step 1: MTP graft N/A for Gemma-4 (no MTP head). mtp tensors in output index: 0; mtp entries in ignore list: 0 -- step 2: restore processor_config.json + preprocessor_config.json processor_config.json already present and identical to source preprocessor_config.json already present and correct -- step 4: confirm saved tokenizer.json has truncation: null truncation is null -- clean [CHECK] done rc=0 ### output tree total 19762161 drwxr-xr-x 2 infra-ops infra-ops 13 Sep 10 10:43 . drwxrwxr-x 62 llmuser llm 99 Sep 10 08:59 .. -rw-r--r-- 1 root root 16934 Sep 10 10:43 chat_template.jinja -rw-r--r-- 1 root root 19419 Sep 10 10:43 config.json -rw-r--r-- 1 root root 204 Sep 10 10:43 generation_config.json -rw------- 1 root root 19994044576 Sep 10 10:43 model-00001-of-00002.safetensors -rw------- 1 root root 452731960 Sep 10 10:43 model-00002-of-00002.safetensors -rw-r--r-- 1 root root 209808 Sep 10 10:43 model.safetensors.index.json -rw-r--r-- 1 root root 375 Sep 10 10:43 preprocessor_config.json -rw-r--r-- 1 root root 1689 Aug 12 02:28 processor_config.json -rw-r--r-- 1 root root 430 Sep 10 10:43 recipe.yaml -rw-r--r-- 1 root root 2819 Sep 10 10:43 tokenizer_config.json -rw-r--r-- 1 root root 32169780 Sep 10 10:43 tokenizer.json 19G /tank/aimodels/G4-MeroMero-v2-31B-heretic-NVFP4A16