4 Commits

Author	SHA1	Message	Date
vh	8d6a9390de	docs(gemma4-charrp): RETRACT the contradiction-deficit claim — the item was ill-posed ... Supersedes what commit 3446367 recorded. That message stated the A16 control "settles a question" and quoted 12% contradiction detection against gen's 81%. The quantization half of it stands; the deficit it was measuring does not exist. brokkr-smithy-dev retracted the finding after the operator asked to see the individual items. The task presented two mutually contradicting statements and asked for "the contradicting statement" — but CONTRADICTION IS SYMMETRIC. Neither statement was more the contradicting one, the model had no way to know which had been inserted, and it consistently named the absolute claim: a defensible reading that the labelling scored wrong on every single item. The tell was there and both of us walked past it: 12% on a five-option task is BELOW THE 20% CHANCE FLOOR. A below-chance score indicts the instrument before it indicts the model. That should be the first reaction to a below-chance result, not a late one, and it is now written into the README as such. Retracted: "the model owns the contradiction deficit"; "domain tuning costs 43 points of contradiction detection" (which on a sound instrument does not shrink but REVERSES); and every pre-fix T2 number for Gemma-4, MeroMero-v2, sec and gen. A second defect surfaced during the fix — all generators shared one RNG, so rewriting one task reshuffled every task after it. What survives is real and worth separating out: the A16 control result holds. Activation precision is close to free on this workload, every other task identical across the W4A4 and W4A16 builds. The two staging confounds caught before the run — the two Hub repos named NVFP4A16 that declare 4-bit activations, and the stale chat template — were independent of the item defect and remain load-bearing. On the corrected instrument Gemma leads the very axis it was suspected of failing (94% against sec's 81% and gen's 50%); its actual weak axis with thinking off is spatial composition at 69%. Recorded as a dated superseded-claims table rather than a silent edit, per the repo's quant-work convention, so notes elsewhere stop misleading people.	2026-08-24 13:07:58 -07:00
vh	24e8826219	docs(gemma4-charrp): the A16 control needs a chat-template override, not just a path swap ... Pre-flighting the staged A16 build before handing it to brokkr-smithy-dev's battery found a second axis hiding inside what was supposed to be a single-variable control. The A16 build ships a STALE chat template. Verified by hash against the upstream weights on the same disk: google/gemma-4-26B-A4B-it is 390 lines, the RedHatAI A4 build's is 389 and byte-identical to upstream once trailing newlines are normalised, and the prithivMLmods A16 build's is 266 and is not. The delta is not cosmetic — upstream and A4 open the thinking path with `{%- set enable_thinking = enable_thinking | default(false) -%}` and branch off it, while the A16 template has no such set and guards with `enable_thinking is defined and enable_thinking` instead. tokenizer_config.json corroborates: A4's response_schema carries a `thinking` property, A16's has only role and content. That build was quantized from an older revision of the checkpoint. Served with its own template, the A16 arm would render a different prompt for identical messages, and a contradiction-detection delta could be attributed to activation precision when it was the template. That is the same failure class as the misnamed-A16 repos — a field nobody validated, believed because the name looked right — one layer further down, and it would have produced a result that looked like a finding. Overriding is safe because the tokenizers agree: vocab identical at 262,144 entries, added_tokens identical, so the same template over the same vocab renders the same token ids. Everything else pre-flights clean — both artifacts complete with no missing shards, generation_config.json byte-identical. Seat NOT flipped; displacing production for the bench window is the operator's call and is still open.	2026-08-24 12:32:03 -07:00
vh	f509668e45	docs(gemma4-charrp): record the A16 activation-axis control and how to run it ... brokkr-smithy-dev's first battery on the new seat scored 12% on contradiction detection with CoT off, against gen's 81%, while state tracking, deterministic constraint following, long-context state and the confabulation control all sat at 100%. That is not general degradation — it is the shape 4-bit input activations produce on the most reasoning-dense task, which is exactly the confound flagged when the W4A4 quant was chosen. They have recorded the finding as CONFOUNDED rather than reporting it as a property of Gemma, and asked for an A16 build to separate the two. No quant run was needed: a genuine A16 build of the same checkpoint already exists on the Hub and is now at /tank/aimodels/gemma4-26b-a4b-it-nvfp4a16 (prithivMLmods, compressed-tensors, nvfp4-pack-quantized, input_activations null, 17 GB). Same weights, same loader as the live seat, one axis moved. ⚠ Two other repos would have answered the question wrongly and the note in the env template says so: bg-digitalservices and ManniX-ITA both publish "Gemma-4-26B-A4B-it-NVFP4A16" whose config declares input_activations num_bits 4 — W4A4 under an A16 name, via modelopt, both 16.46 GB against the real one's 17.93. Using either and seeing T2 stay at 12% would have wrongly convicted the model. Running it is a one-line GEMMA4_MODEL swap plus a recreate, because there is no room for a concurrent seat: GPU0 has 3.5 GiB free with gen and the A4 seat on it, and GPU1's 19.4 GiB against ~18 GiB of weights is the same sub-GiB headroom that crash-looped the predecessor this morning. Port and both aliases are unchanged either way, so no consumer config moves.	2026-08-24 12:28:17 -07:00
vh	27155c0f3b	feat(char-rp): swap the seat to the Gemma-4 26B-A4B MoE, NVFP4, same port ... Straight-across replacement of the dense G4-MeroMero-v2-31B-NVFP4A16 seat with google/gemma-4-26B-A4B-it on ana-ml2 GPU0. Port, served-model-names and every gateway route are unchanged, so no consumer sees a difference in addressing: `char-rp` -> hosted_vllm/char-rp and `char-rp-reasoning` -> hosted_vllm/char-rp-thinking, both still :8016. The seat's requirements now include chain-of-thought, which makes throughput more critical rather than less — the user waits through the whole reasoning block before the first visible token, and the MoE measures ~114 tok/s @32K against the dense 31B's ~40.7. Both artifacts are on disk and they are NOT interchangeable. The BF16 weights (/tank/aimodels/gemma4-26b-a4b-it-bf16, 49 GB) are the QLoRA tuning base, since QLoRA does its own quantization. They CANNOT be served here: 48.10 GiB of weights against ~49 GiB of free GPU0 leaves nothing for KV cache, and the engine would die at allocation exactly the way the predecessor did this afternoon. The serving copy is RedHatAI/gemma-4-26B-A4B-it-NVFP4 (16 GB), chosen over the other -it quants because it is compressed-tensors (nvfp4-pack-quantized) — the same loader path the outgoing seat used — from the llm-compressor team at 357k downloads. The nvidia/ repo is the base rather than -it, and the thinking channel lives in the instruction-tuned weights. Smaller weights at the same 0.47 memory budget buy a much larger KV pool: 27.37 GiB and 1,724,110 tokens, against the predecessor's 371,023 at the same budget. That is 6.5 full-length 262K sequences concurrent rather than 1.4. The gemma4 tool-call parser, reasoning parser and the enable_thinking:false default all carry over unchanged — they are architecture-level, not checkpoint-level. The --chat-template override does NOT carry over: MeroMero pointed at a jinja hand-patched against that checkpoint, and this model ships its own. Verified that dropping it did not reintroduce the failure that flag existed to prevent — non-thinking prose lands in content with reasoning_content empty, and the thinking alias populates reasoning_content with content carrying the answer. ⚠ Scheme differs from the incumbent and the bench should say so: this quant declares 4-bit input activations (W4A4) where the outgoing seat was NVFP4A16. Faster, and not like-for-like on the activation axis. meromero-charrp is retained stopped in `created` state and relabelled to AI - Dormant, per the house rollback pattern. Both stacks want :8016, so rolling back means stopping the gemma4 seat first.	2026-08-24 12:02:59 -07:00