feat(gemma4-charrp): pin the chat template; A16 control run executed and reverted

The template is now passed explicitly, defaulting to the A4 build's
chat_template.jinja. That is a no-op for what is served — the A4 build ships
that exact file, byte-identical to upstream google/gemma-4-26B-A4B-it once
trailing newlines are normalised — and it permanently closes the class of bug
found while staging the control: the A16 build ships a stale 266-line template
against upstream's 390, with the thinking path built differently and no
`thinking` property in its tokenizer_config response_schema. Serving each build
with its own template would have moved a second axis.

The control ran on the operator's greenlight and has been reverted. Seat is back
on the W4A4 build, healthy, RestartCount 0, both aliases verified through the
gateway — char-rp returns content with reasoning_content empty,
char-rp-reasoning returns both.

Result, since it settles a question this repo's config now encodes: activation
precision does NOT explain the contradiction-detection deficit. Contradiction
detection moved 12% -> 19% between W4A4 and W4A16, which at n=16 is 2/16 -> 3/16
— one item — against gen's 81% on identical items. Every other task is identical
across the two builds and the core difference is 2.6 points carried almost
entirely by two single items. brokkr-smithy-dev pre-registered that a null
result would be the robust branch, because a hidden third axis would tend to
create a delta rather than suppress one, so the conclusion survives the residual
doubt neither side could close without a dequantization pass.

The practical upshot for future scheme choices: W4A4 costs less on this workload
than the caution warranted. The caution was still correct to have.

Displaced production for 3.7 seconds of measurement plus two container
recreates. The A16 build and the BF16 tuning base both stay on disk with the
runbook in the stack README, so re-running is a two-minute flip.
This commit is contained in:
2026-08-24 13:03:19 -07:00
parent 1bd90eaacc
commit 3446367d5e
+23 -6
View File
@@ -75,12 +75,29 @@ services:
# `char-rp-reasoning` alias exercises.
- --default-chat-template-kwargs
- '{"enable_thinking": false}'
# ⚠ NO --chat-template. MeroMero pointed at a hand-patched jinja under
# meromero-v2-nvfp4-work/; that patch was cut against THAT checkpoint and
# does not describe this one. This model ships its own chat_template.jinja
# and vLLM picks it up. If thinking mode ever misbehaves, diff this
# model's template against the MeroMero patch before reaching for a
# copy — do not assume the patch transfers.
# TEMPLATE PINNED EXPLICITLY, AND IT IS A NO-OP FOR THE DEFAULT MODEL.
# The A4 build ships this exact file — verified byte-identical to upstream
# google/gemma-4-26B-A4B-it once trailing newlines are normalised (390 vs
# 389 lines, same hash) — so pinning it changes nothing about what is
# served and permanently removes a real class of bug.
#
# ⚠ WHY IT IS PINNED AT ALL: the A16 control build at
# /tank/aimodels/gemma4-26b-a4b-it-nvfp4a16 ships a STALE 266-line
# template. Upstream and A4 open the thinking path with
# `{%- set enable_thinking = enable_thinking | default(false) -%}`; the
# A16 one has no such set, and its tokenizer_config response_schema lacks
# the `thinking` property. Serving it with its own template would render a
# DIFFERENT PROMPT, turning a one-axis activation-precision control into a
# two-axis comparison — a result that would look like a finding.
# Safe to force across both builds because the tokenizers are identical:
# vocab 262,144 entries, added_tokens identical.
#
# ⚠ IF GEMMA4_MODEL EVER POINTS AT A DIFFERENT CHECKPOINT, THIS DEFAULT
# MUST MOVE WITH IT. A pinned template is only correct for the checkpoint
# it came from. That is the inverse of the mistake the outgoing MeroMero
# seat warned about, where a hand-patched template was assumed to transfer.
- --chat-template
- ${GEMMA4_CHAT_TEMPLATE:-/tank/aimodels/gemma4-26b-a4b-it-nvfp4/chat_template.jinja}
- --max-model-len
- "${GEMMA4_MAX_MODEL_LEN:-262144}"
- --max-num-seqs