docs(gemma4-charrp): record the A16 activation-axis control and how to run it

brokkr-smithy-dev's first battery on the new seat scored 12% on contradiction
detection with CoT off, against gen's 81%, while state tracking, deterministic
constraint following, long-context state and the confabulation control all sat
at 100%. That is not general degradation — it is the shape 4-bit input
activations produce on the most reasoning-dense task, which is exactly the
confound flagged when the W4A4 quant was chosen. They have recorded the finding
as CONFOUNDED rather than reporting it as a property of Gemma, and asked for an
A16 build to separate the two.

No quant run was needed: a genuine A16 build of the same checkpoint already
exists on the Hub and is now at /tank/aimodels/gemma4-26b-a4b-it-nvfp4a16
(prithivMLmods, compressed-tensors, nvfp4-pack-quantized, input_activations
null, 17 GB). Same weights, same loader as the live seat, one axis moved.

⚠ Two other repos would have answered the question wrongly and the note in the
env template says so: bg-digitalservices and ManniX-ITA both publish
"Gemma-4-26B-A4B-it-NVFP4A16" whose config declares input_activations num_bits
4 — W4A4 under an A16 name, via modelopt, both 16.46 GB against the real one's
17.93. Using either and seeing T2 stay at 12% would have wrongly convicted the
model.

Running it is a one-line GEMMA4_MODEL swap plus a recreate, because there is no
room for a concurrent seat: GPU0 has 3.5 GiB free with gen and the A4 seat on
it, and GPU1's 19.4 GiB against ~18 GiB of weights is the same sub-GiB headroom
that crash-looped the predecessor this morning. Port and both aliases are
unchanged either way, so no consumer config moves.
This commit is contained in:
2026-08-24 12:28:17 -07:00
parent 27155c0f3b
commit f509668e45
+15
View File
@@ -22,6 +22,21 @@ GEMMA4_MAX_NUM_SEQS=32
# ⚠ THE NVFP4 QUANT, NOT THE BF16. /tank/aimodels/gemma4-26b-a4b-it-bf16 is the
# QLoRA tuning base and is 48.10 GiB of weights — it does not fit beside gen.
GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4
#
# ACTIVATION-AXIS CONTROL, for one-off benching — swap this line, recreate:
# GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4a16
# Same NVFP4 weights and the same compressed-tensors loader, but 16-bit
# activations. brokkr-smithy-dev's 2026-08-24 battery scored 12% on
# contradiction detection with CoT off against gen's 81%, while every other
# task sat at 100% — the shape 4-bit input activations would produce on the
# most reasoning-dense task, so the finding is CONFOUNDED until this is run.
# Two minutes down each way; port and both aliases are unchanged, so no
# consumer config moves.
#
# ⚠ Only prithivMLmods/gemma-4-26B-A4B-it-NVFP4A16 is genuinely A16.
# bg-digitalservices and ManniX-ITA both publish repos NAMED "NVFP4A16" whose
# config.json declares input_activations num_bits 4. Check the field, not the
# name, before ever substituting a different repo.
GEMMA4_PORT=8016
GEMMA4_GPU_ID=0