From f509668e458e1c22b2d0718f9a56f96a089e9329 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 24 Aug 2026 12:28:17 -0700 Subject: [PATCH] docs(gemma4-charrp): record the A16 activation-axis control and how to run it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit brokkr-smithy-dev's first battery on the new seat scored 12% on contradiction detection with CoT off, against gen's 81%, while state tracking, deterministic constraint following, long-context state and the confabulation control all sat at 100%. That is not general degradation — it is the shape 4-bit input activations produce on the most reasoning-dense task, which is exactly the confound flagged when the W4A4 quant was chosen. They have recorded the finding as CONFOUNDED rather than reporting it as a property of Gemma, and asked for an A16 build to separate the two. No quant run was needed: a genuine A16 build of the same checkpoint already exists on the Hub and is now at /tank/aimodels/gemma4-26b-a4b-it-nvfp4a16 (prithivMLmods, compressed-tensors, nvfp4-pack-quantized, input_activations null, 17 GB). Same weights, same loader as the live seat, one axis moved. ⚠ Two other repos would have answered the question wrongly and the note in the env template says so: bg-digitalservices and ManniX-ITA both publish "Gemma-4-26B-A4B-it-NVFP4A16" whose config declares input_activations num_bits 4 — W4A4 under an A16 name, via modelopt, both 16.46 GB against the real one's 17.93. Using either and seeing T2 stay at 12% would have wrongly convicted the model. Running it is a one-line GEMMA4_MODEL swap plus a recreate, because there is no room for a concurrent seat: GPU0 has 3.5 GiB free with gen and the A4 seat on it, and GPU1's 19.4 GiB against ~18 GiB of weights is the same sub-GiB headroom that crash-looped the predecessor this morning. Port and both aliases are unchanged either way, so no consumer config moves. --- stacks/gemma4-charrp/.env.example | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/stacks/gemma4-charrp/.env.example b/stacks/gemma4-charrp/.env.example index 03e8638..3b3a8ab 100644 --- a/stacks/gemma4-charrp/.env.example +++ b/stacks/gemma4-charrp/.env.example @@ -22,6 +22,21 @@ GEMMA4_MAX_NUM_SEQS=32 # ⚠ THE NVFP4 QUANT, NOT THE BF16. /tank/aimodels/gemma4-26b-a4b-it-bf16 is the # QLoRA tuning base and is 48.10 GiB of weights — it does not fit beside gen. GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4 +# +# ACTIVATION-AXIS CONTROL, for one-off benching — swap this line, recreate: +# GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4a16 +# Same NVFP4 weights and the same compressed-tensors loader, but 16-bit +# activations. brokkr-smithy-dev's 2026-08-24 battery scored 12% on +# contradiction detection with CoT off against gen's 81%, while every other +# task sat at 100% — the shape 4-bit input activations would produce on the +# most reasoning-dense task, so the finding is CONFOUNDED until this is run. +# Two minutes down each way; port and both aliases are unchanged, so no +# consumer config moves. +# +# ⚠ Only prithivMLmods/gemma-4-26B-A4B-it-NVFP4A16 is genuinely A16. +# bg-digitalservices and ManniX-ITA both publish repos NAMED "NVFP4A16" whose +# config.json declares input_activations num_bits 4. Check the field, not the +# name, before ever substituting a different repo. GEMMA4_PORT=8016 GEMMA4_GPU_ID=0