docs(gemma4-charrp): record the A16 activation-axis control and how to run it
brokkr-smithy-dev's first battery on the new seat scored 12% on contradiction detection with CoT off, against gen's 81%, while state tracking, deterministic constraint following, long-context state and the confabulation control all sat at 100%. That is not general degradation — it is the shape 4-bit input activations produce on the most reasoning-dense task, which is exactly the confound flagged when the W4A4 quant was chosen. They have recorded the finding as CONFOUNDED rather than reporting it as a property of Gemma, and asked for an A16 build to separate the two. No quant run was needed: a genuine A16 build of the same checkpoint already exists on the Hub and is now at /tank/aimodels/gemma4-26b-a4b-it-nvfp4a16 (prithivMLmods, compressed-tensors, nvfp4-pack-quantized, input_activations null, 17 GB). Same weights, same loader as the live seat, one axis moved. ⚠ Two other repos would have answered the question wrongly and the note in the env template says so: bg-digitalservices and ManniX-ITA both publish "Gemma-4-26B-A4B-it-NVFP4A16" whose config declares input_activations num_bits 4 — W4A4 under an A16 name, via modelopt, both 16.46 GB against the real one's 17.93. Using either and seeing T2 stay at 12% would have wrongly convicted the model. Running it is a one-line GEMMA4_MODEL swap plus a recreate, because there is no room for a concurrent seat: GPU0 has 3.5 GiB free with gen and the A4 seat on it, and GPU1's 19.4 GiB against ~18 GiB of weights is the same sub-GiB headroom that crash-looped the predecessor this morning. Port and both aliases are unchanged either way, so no consumer config moves.
This commit is contained in:
@@ -22,6 +22,21 @@ GEMMA4_MAX_NUM_SEQS=32
|
||||
# ⚠ THE NVFP4 QUANT, NOT THE BF16. /tank/aimodels/gemma4-26b-a4b-it-bf16 is the
|
||||
# QLoRA tuning base and is 48.10 GiB of weights — it does not fit beside gen.
|
||||
GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4
|
||||
#
|
||||
# ACTIVATION-AXIS CONTROL, for one-off benching — swap this line, recreate:
|
||||
# GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4a16
|
||||
# Same NVFP4 weights and the same compressed-tensors loader, but 16-bit
|
||||
# activations. brokkr-smithy-dev's 2026-08-24 battery scored 12% on
|
||||
# contradiction detection with CoT off against gen's 81%, while every other
|
||||
# task sat at 100% — the shape 4-bit input activations would produce on the
|
||||
# most reasoning-dense task, so the finding is CONFOUNDED until this is run.
|
||||
# Two minutes down each way; port and both aliases are unchanged, so no
|
||||
# consumer config moves.
|
||||
#
|
||||
# ⚠ Only prithivMLmods/gemma-4-26B-A4B-it-NVFP4A16 is genuinely A16.
|
||||
# bg-digitalservices and ManniX-ITA both publish repos NAMED "NVFP4A16" whose
|
||||
# config.json declares input_activations num_bits 4. Check the field, not the
|
||||
# name, before ever substituting a different repo.
|
||||
|
||||
GEMMA4_PORT=8016
|
||||
GEMMA4_GPU_ID=0
|
||||
|
||||
Reference in New Issue
Block a user