# gemma4-charrp — char-rp seat on ana-ml2 GPU0. Real .env lives on the host.

# ⚠ GPU0 IS SHARED WITH `vllm-gen`. gen runs at --gpu-memory-utilization 0.43
# but actually holds ~45.6 GiB of the 94.97 GiB card — that flag sizes the KV
# cache and does NOT cover CUDA context, graphs and non-torch overhead. The
# predecessor seat sat at 0.51, the pair summed to 0.94, and on 2026-08-24 it
# stopped fitting and crash-looped 13 times with
# `torch.OutOfMemoryError: ... 195.19 MiB is free`.
#
# 0.47 keeps ~4.8 GiB of real margin. This model's weights are only ~15.3 GiB
# (NVFP4) against the predecessor's ~19.5 GiB, so the same budget buys MORE KV
# cache than before, not less. Raising this means lowering gen's in the same
# change — and check the real numbers, not the flags:
#   nvidia-smi --query-compute-apps=pid,used_memory --format=csv
GEMMA4_GPU_MEM_UTIL=0.47

# Native context. config.json declares max_position_embeddings 262144, same as
# the outgoing seat, so this is a straight-across swap on context too.
GEMMA4_MAX_MODEL_LEN=262144
GEMMA4_MAX_NUM_SEQS=32

# ⚠ THE NVFP4 QUANT, NOT THE BF16. /tank/aimodels/gemma4-26b-a4b-it-bf16 is the
# QLoRA tuning base and is 48.10 GiB of weights — it does not fit beside gen.
GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4
#
# ACTIVATION-AXIS CONTROL, for one-off benching — swap this line, recreate:
#   GEMMA4_MODEL=/tank/aimodels/gemma4-26b-a4b-it-nvfp4a16
# Same NVFP4 weights and the same compressed-tensors loader, but 16-bit
# activations.
#
# ⚠ THE ORIGINAL MOTIVATION FOR THIS CONTROL WAS RETRACTED — see the README's
# "Superseded claims" section. The A16 run itself HELD and is the reason this
# path is documented: activation precision is close to free on this battery,
# every other task identical across builds. But the "12% contradiction
# detection" number that prompted it was an artifact of an ill-posed benchmark
# item, not a model property. Keep the path; do not repeat the reasoning.
# Two minutes down each way; port and both aliases are unchanged, so no
# consumer config moves.
#
# ⚠ Only prithivMLmods/gemma-4-26B-A4B-it-NVFP4A16 is genuinely A16.
# bg-digitalservices and ManniX-ITA both publish repos NAMED "NVFP4A16" whose
# config.json declares input_activations num_bits 4. Check the field, not the
# name, before ever substituting a different repo.
#
# ⚠⚠ THE A16 BUILD ALSO SHIPS A STALE CHAT TEMPLATE — the control run MUST
# override it or it moves two axes instead of one:
#   --chat-template /tank/aimodels/gemma4-26b-a4b-it-nvfp4/chat_template.jinja
# Verified by hash on 2026-08-24: upstream google/gemma-4-26B-A4B-it is 390
# lines, the A4 build's is 389 and byte-identical to it once trailing newlines
# are normalised, and the A16 build's is 266 and is NOT. The thinking machinery
# is built differently too — upstream and A4 set
# `enable_thinking | default(false)` at line 186, the A16 template has no such
# set — and its tokenizer_config response_schema has no `thinking` property.
# It was quantized from an older revision. Served with its own template the two
# arms would render DIFFERENT PROMPTS, and a score delta could be the template
# rather than the activations.
# Safe to override because the tokenizers agree: vocab identical at 262,144
# entries, added_tokens identical. Same template over the same vocab renders
# the same token ids.

GEMMA4_PORT=8016
GEMMA4_GPU_ID=0
GEMMA4_CONTAINER=vllm-gemma4-charrp

# Same value as every other vLLM seat on this host — the gateway presents it.
API_KEY=