feat(gemma4-charrp): pin the chat template; A16 control run executed and reverted
The template is now passed explicitly, defaulting to the A4 build's chat_template.jinja. That is a no-op for what is served — the A4 build ships that exact file, byte-identical to upstream google/gemma-4-26B-A4B-it once trailing newlines are normalised — and it permanently closes the class of bug found while staging the control: the A16 build ships a stale 266-line template against upstream's 390, with the thinking path built differently and no `thinking` property in its tokenizer_config response_schema. Serving each build with its own template would have moved a second axis. The control ran on the operator's greenlight and has been reverted. Seat is back on the W4A4 build, healthy, RestartCount 0, both aliases verified through the gateway — char-rp returns content with reasoning_content empty, char-rp-reasoning returns both. Result, since it settles a question this repo's config now encodes: activation precision does NOT explain the contradiction-detection deficit. Contradiction detection moved 12% -> 19% between W4A4 and W4A16, which at n=16 is 2/16 -> 3/16 — one item — against gen's 81% on identical items. Every other task is identical across the two builds and the core difference is 2.6 points carried almost entirely by two single items. brokkr-smithy-dev pre-registered that a null result would be the robust branch, because a hidden third axis would tend to create a delta rather than suppress one, so the conclusion survives the residual doubt neither side could close without a dequantization pass. The practical upshot for future scheme choices: W4A4 costs less on this workload than the caution warranted. The caution was still correct to have. Displaced production for 3.7 seconds of measurement plus two container recreates. The A16 build and the BF16 tuning base both stay on disk with the runbook in the stack README, so re-running is a two-minute flip.
This commit is contained in:
@@ -75,12 +75,29 @@ services:
|
||||
# `char-rp-reasoning` alias exercises.
|
||||
- --default-chat-template-kwargs
|
||||
- '{"enable_thinking": false}'
|
||||
# ⚠ NO --chat-template. MeroMero pointed at a hand-patched jinja under
|
||||
# meromero-v2-nvfp4-work/; that patch was cut against THAT checkpoint and
|
||||
# does not describe this one. This model ships its own chat_template.jinja
|
||||
# and vLLM picks it up. If thinking mode ever misbehaves, diff this
|
||||
# model's template against the MeroMero patch before reaching for a
|
||||
# copy — do not assume the patch transfers.
|
||||
# TEMPLATE PINNED EXPLICITLY, AND IT IS A NO-OP FOR THE DEFAULT MODEL.
|
||||
# The A4 build ships this exact file — verified byte-identical to upstream
|
||||
# google/gemma-4-26B-A4B-it once trailing newlines are normalised (390 vs
|
||||
# 389 lines, same hash) — so pinning it changes nothing about what is
|
||||
# served and permanently removes a real class of bug.
|
||||
#
|
||||
# ⚠ WHY IT IS PINNED AT ALL: the A16 control build at
|
||||
# /tank/aimodels/gemma4-26b-a4b-it-nvfp4a16 ships a STALE 266-line
|
||||
# template. Upstream and A4 open the thinking path with
|
||||
# `{%- set enable_thinking = enable_thinking | default(false) -%}`; the
|
||||
# A16 one has no such set, and its tokenizer_config response_schema lacks
|
||||
# the `thinking` property. Serving it with its own template would render a
|
||||
# DIFFERENT PROMPT, turning a one-axis activation-precision control into a
|
||||
# two-axis comparison — a result that would look like a finding.
|
||||
# Safe to force across both builds because the tokenizers are identical:
|
||||
# vocab 262,144 entries, added_tokens identical.
|
||||
#
|
||||
# ⚠ IF GEMMA4_MODEL EVER POINTS AT A DIFFERENT CHECKPOINT, THIS DEFAULT
|
||||
# MUST MOVE WITH IT. A pinned template is only correct for the checkpoint
|
||||
# it came from. That is the inverse of the mistake the outgoing MeroMero
|
||||
# seat warned about, where a hand-patched template was assumed to transfer.
|
||||
- --chat-template
|
||||
- ${GEMMA4_CHAT_TEMPLATE:-/tank/aimodels/gemma4-26b-a4b-it-nvfp4/chat_template.jinja}
|
||||
- --max-model-len
|
||||
- "${GEMMA4_MAX_MODEL_LEN:-262144}"
|
||||
- --max-num-seqs
|
||||
|
||||
Reference in New Issue
Block a user