The stack had no README and now carries three model directories that look
interchangeable and are not: the BF16 QLoRA base that cannot be served here, the
W4A4 quant that is served, and the W4A16 build that exists solely as an
activation-axis control. Writing down which is which, and why, before someone
"simplifies" the compose to the BF16 path and rediscovers the OOM.
Also captures the A16 control procedure end to end, including the two confounds
found while staging it — the two Hub repos named NVFP4A16 that declare 4-bit
activations, and the stale chat template the real one ships — and the fact that
overriding the template is safe because the tokenizers are identical. Both sides
have now cross-checked this: brokkr independently diffed every non-quantization
config field of both builds against the upstream BF16 and found only
transformers_version differing. Residual risk recorded rather than hidden:
config identity is not weight identity and nobody has done a dequantization
pass.
The Gemma-4 flags are documented as architecture-level rather than
checkpoint-level, since that is why they survived the seat swap unchanged, and
the enable_thinking:false pin is called out as mandatory rather than stylistic —
without it every plain prose turn lands in reasoning_content with a null content
and every consumer breaks.
Notes the non-termination defect with thinking on (32 of 96 calls truncating at
12k tokens, all 16 constraint items among them, reasoning sound right up to the
point it fails to stop) and why VLLM_USE_V2_MODEL_RUNNER=0 is deliberately not
applied to a seat whose production mode is thinking-off.
No live change: the seat is still serving the W4A4 build. Displacing it for the
control run is an operator decision and is still open.
vh
1bd90eaacc