docs(gemma4-charrp): the A16 control needs a chat-template override, not just a path swap
Pre-flighting the staged A16 build before handing it to brokkr-smithy-dev's
battery found a second axis hiding inside what was supposed to be a
single-variable control.
The A16 build ships a STALE chat template. Verified by hash against the upstream
weights on the same disk: google/gemma-4-26B-A4B-it is 390 lines, the RedHatAI
A4 build's is 389 and byte-identical to upstream once trailing newlines are
normalised, and the prithivMLmods A16 build's is 266 and is not. The delta is
not cosmetic — upstream and A4 open the thinking path with
`{%- set enable_thinking = enable_thinking | default(false) -%}` and branch off
it, while the A16 template has no such set and guards with
`enable_thinking is defined and enable_thinking` instead. tokenizer_config.json
corroborates: A4's response_schema carries a `thinking` property, A16's has only
role and content. That build was quantized from an older revision of the
checkpoint.
Served with its own template, the A16 arm would render a different prompt for
identical messages, and a contradiction-detection delta could be attributed to
activation precision when it was the template. That is the same failure class as
the misnamed-A16 repos — a field nobody validated, believed because the name
looked right — one layer further down, and it would have produced a result that
looked like a finding.
Overriding is safe because the tokenizers agree: vocab identical at 262,144
entries, added_tokens identical, so the same template over the same vocab
renders the same token ids. Everything else pre-flights clean — both artifacts
complete with no missing shards, generation_config.json byte-identical.
Seat NOT flipped; displacing production for the bench window is the operator's
call and is still open.
This commit is contained in:
@@ -37,6 +37,22 @@ GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4
|
||||
# bg-digitalservices and ManniX-ITA both publish repos NAMED "NVFP4A16" whose
|
||||
# config.json declares input_activations num_bits 4. Check the field, not the
|
||||
# name, before ever substituting a different repo.
|
||||
#
|
||||
# ⚠⚠ THE A16 BUILD ALSO SHIPS A STALE CHAT TEMPLATE — the control run MUST
|
||||
# override it or it moves two axes instead of one:
|
||||
# --chat-template /tank/aimodels/gemma4-26b-a4b-it-nvfp4/chat_template.jinja
|
||||
# Verified by hash on 2026-08-24: upstream google/gemma-4-26B-A4B-it is 390
|
||||
# lines, the A4 build's is 389 and byte-identical to it once trailing newlines
|
||||
# are normalised, and the A16 build's is 266 and is NOT. The thinking machinery
|
||||
# is built differently too — upstream and A4 set
|
||||
# `enable_thinking | default(false)` at line 186, the A16 template has no such
|
||||
# set — and its tokenizer_config response_schema has no `thinking` property.
|
||||
# It was quantized from an older revision. Served with its own template the two
|
||||
# arms would render DIFFERENT PROMPTS, and a score delta could be the template
|
||||
# rather than the activations.
|
||||
# Safe to override because the tokenizers agree: vocab identical at 262,144
|
||||
# entries, added_tokens identical. Same template over the same vocab renders
|
||||
# the same token ids.
|
||||
|
||||
GEMMA4_PORT=8016
|
||||
GEMMA4_GPU_ID=0
|
||||
|
||||
Reference in New Issue
Block a user