tune(vllm): GPU-1 rebalance — granite 131k ctx, qwen 65k ctx, ~3.5GB free

Reclaimed Qwen3.5-9B's over-provisioned KV (20x conc @ 32k) and handed it
to granite. granite: 51200->131072 ctx (305k-token pool, 2.33x worst-case;
PagedAttention => ~2.2x more short-request concurrency from the bigger pool),
util 0.36->0.35. qwen: 32768->65536 ctx (8.13x), util 0.40->0.35. Trio
unchanged (chunked inputs, 8k plenty). Leaves ~3.7GB free on the shared
card. Start-order matters (trim qwen first, then grow granite) — vLLM
requires free>=util*total at startup.

stacks/qwen35-vl/.env.example

@@ -15,10 +15,13 @@ QWEN_PORT=8007
 # GPU 0 is kept free for hot-reloading large models.
 QWEN_GPU_ID=1
 
-# 0.40 (~38 GB) — above the ~34 GB start floor, ~10 GB card headroom over prod.
-# On this shared card vLLM needs free >= util*total, so util is capped ~0.51.
-QWEN_GPU_MEM_UTIL=0.40
-QWEN_MAX_MODEL_LEN=32768
+# util 0.35 (~33.6 GB) / max-len 65536 — GPU-1 rebalance 2026-06-13. Qwen was
+# wildly over-provisioned (20x conc @ 32k); trimmed to free room for granite while
+# DOUBLING qwen's own context (32k->65k, still ~8x conc). On this shared card vLLM
+# needs free >= util*total (cap ~0.51 here); floor to start at 65k is ~0.34.
+# Raising max-len is free for short requests (PagedAttention = KV per actual token).
+QWEN_GPU_MEM_UTIL=0.35
+QWEN_MAX_MODEL_LEN=65536
 
 # Optional
 HF_TOKEN=

stacks/vllm/.env.example

@@ -83,13 +83,17 @@ GRANITE_GPU_ID=1
 GRANITE_MODEL=ibm-granite/granite-4.1-8b-fp8
 GRANITE_QUANT=compressed-tensors
 GRANITE_SERVED_NAME=granite-4.1-8b
-# 50K ctx (51200) — matches phi4's window. Deliberately NOT maxed: leaves ~3 GB
-# on GPU 1 as a hedge for future Granite-LoRA serving (--enable-lora needs ~2-3 GB)
-# / a holistic GPU-1 retune of the embed/rerank/reward trio. KV @50K fp8 ≈ 4.2 GB;
-# +9.6 weights +~2.4 overhead ≈ 16.2 GB. Granite 4.1 supports up to 131072.
-GRANITE_MAX_MODEL_LEN=51200
+# 131072 ctx — MAXED 2026-06-13 (was 51200/50K). GPU-1 rebalance: granite shares
+# the card with the trio + qwen35-vl (vision). Qwen was over-provisioned on KV
+# (20x conc @ 32k), so trimming it freed room for granite's ~305k-token pool here.
+# PagedAttention allocates KV per ACTUAL token, so 131072 is only a CEILING — a 1k
+# summarize turn uses ~1k tokens, so the pool holds ~300 concurrently; the "2.33x"
+# headline is worst-case (every request maxing 131k). Granite 4.1 supports 131072.
+GRANITE_MAX_MODEL_LEN=131072
 # FP8 KV cache (native on Ada cc 8.9). At 50K ≈ ~4.2 GB (vs ~8.4 GB at fp16).
 GRANITE_KV_CACHE_DTYPE=fp8
-# Target ~16.2 GB of GPU 1's ~19 GB free: weights 9.6 + KV 4.2 + overhead ~2.4.
-# 0.36×46≈16.6 GB budget (covers it with slack). Confirm vs vLLM's first-boot KV log.
-GRANITE_GPU_MEM_UTIL=0.36
+# util 0.35 (~33.6 GB) — tuned 2026-06-13 to leave ~3.5 GB free on GPU 1 alongside
+# the trio + qwen co-tenants. On this shared card vLLM needs free >= util*total at
+# startup, and START ORDER matters: trim qwen FIRST, then grow granite, else granite
+# OOMs against the full card. (0.37 overshot to 1.7 GB free; 0.35 lands ~3.7 GB.)
+GRANITE_GPU_MEM_UTIL=0.35