esh-pfi-infrastructure/persistent-memory.d/2026-09-10-meromero-acquisition-and-four-quant-failures.md

[2026-09-10] MeroMero acquisition — the A4B quant landed, the v2 dense has failed FOUR times

Operator wanted a MeroMero seat. Getting there cost four quant attempts and corrected two of my own wrong hypotheses, so the failure chain is the durable part.

The family, because I got it wrong first

I called the unsuffixed 31B "v1" because v2 is v2-31B. The operator caught it. Creation dates from the upstream author settle it:

2026-04-15  zerofata/G4-MeroMero-26B-A4B   30 layers, kv 8   MoE 128 experts  ← the ORIGINAL
2026-04-30  zerofata/G4-MeroMero-31B       60 layers, kv 16  DENSE            ← a separate line
2026-07-31  zerofata/G4-MeroMero-v2-31B    60 layers, kv 16  DENSE

Not a v1→v2 sequence at one size: an A4B MoE first, then a dense 31B line that later got a v2. ⚠ The -v1- repo 401s, and I read that as "v1 must be the unsuffixed one" instead of checking dates.

The geometry is why it matters. The A4B is 30 layers / kv 8 — identical to Pfish-6 (a gemma4 26B-A4B) — so it drops into the existing KV budget at 262k. The dense 31B is 60 layers / kv 16, ~4x the KV per token, and did NOT fit 262k on GPU1 alongside the other seats.

Published quants: none of them are our scheme

pekkAi/G4-MeroMero-31B-NVFP4                        true v1 dense, W4A4
llmfan46/G4-MeroMero-31B-uncensored-heretic-NVFP4   abliterated dense, W4A16_NVFP4  (pulled, 19 GB)
pekkAi/G4-MeroMero-26B-A4B-it-...-heretic-NVFP4     abliterated A4B, W4A4 + FP8 KV baked in
NVFP4 of v2-31B                                     DOES NOT EXIST, from anyone (0 of 27 v2 repos)

W4A4 carries our own measured long-context collapse (Granite, 30k ctx), which is why every seat we run is A16. So the operator ruled: pull both ablits bf16 and quantize in-house at W4A16.

MTP — checked on the SOURCES, not inferred

Gemma-4 ships no MTP head at all: 0 mtp tensors and no mtp config keys in the A4B bf16, the v2 bf16, Pfish-6, or the published quants. The "mtp if applicable" clause is a no-op here; the MTP work in our history is Qwen-family.

A4B — SUCCEEDED

G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16, 16 G, 7 files, 2m08s. That time confirms playbook §3.16: weight-only A16 runs a DataFreePipeline and never touches the dataset. ⚠ Output is missing preprocessor_config.json — §4.3 post-step 2 still owed before serving.

v2 dense — FOUR failures, each a different layer

1. num_key_value_heads is None at Gemma4TextAttention.__init__. I hypothesised a transformers version deadlock (config authored by 5.16.1, container on 5.12.1) — wrong: all three configs construct fine on 5.12.1. Then I hypothesised the stale torch_dtype kwarg reshaping the config — also wrong: every kwarg combination returns 16. ⚠ A 2-layer truncation PASSED and hid it. The failing local is chosen per layer type; only full_attention layers take the branch. Testing each layer type individually found it in seconds. Real cause: DogOnKeyboard's config sets attention_k_eq_v: true but omits num_global_key_value_heads and global_head_dim — a malformed upload, not a toolchain issue. Patched from zerofata's canonical values (4 / 512) after shape-verifying the checkpoint: full-attn k_proj [2048,5376] = 4×512, sliding [4096,5376] = 16×256, identical to canonical. Original at config.json.bak-pre-kvfields-20260910.
2. initialize model processor ... required when a dataset is provided — DogOnKeyboard ships no processor_config.json (the A4B does; that is why it succeeded). Copied from canonical, AND dropped the dataset entirely, since §3.16 says A16 ignores it and §3.14 says building it bakes a truncation cap into the shipped tokenizer. Removing it kills both for zero loss.
3. My own bug: the reference module runs argparse with required=True at IMPORT, so blanking sys.argv still exited 2. Placeholder args, real argv restored after.
4. AmbiguousGlobalPerLayerAttributeError: 'head_dim' is a per-layer attribute — OPEN. The DogOnKeyboard config carries a per_layer_config key that zerofata's lacks (it was the one key in the diff I noted and did not chase). transformers refuses global access to head_dim on a heterogeneous config; the fix is likely allow_global_per_layer_attribute_access=True, with the warning that a caller assuming homogeneity may then read the wrong value.

⚠ My wrapper reported rc=0 on a failed run because it read $? after an echo. A wrapper that reports success on failure is the false-reassurance class; fixed to capture $? immediately.

Related: 2026-09-10-r49-babybronte-d1-d3-and-the-1-epoch-pilot