fix(vllm): right-size granite util 0.34->0.18 + max-len ->65536 (free ~10.5GB GPU1)

Operator-directed 2026-07-16: free ~10GB on ana-ml2 GPU1 to relocate a GPU0
model onto GPU1. granite-4.1-8b (fleet summarizer) was over-provisioned at
util 0.34 / max-model-len 131072 with a flat 0.0% KV usage.

Set GRANITE_GPU_MEM_UTIL 0.34 -> 0.18 and GRANITE_MAX_MODEL_LEN 131072 -> 65536
on the live /opt/docker/compose/vllm/.env (backup .env.bak-pre-granite-rightsize-
20260716), recreated vllm-granite ONLY (shared stack). Result: GPU1 62,641 ->
51,897 MiB used (~10.5GB freed, ~45GB free now); KV 6.45 GiB / 84,528 tok /
1.29x concurrency @ 65536; summarizer verified healthy.

The util drop required the max-len drop: on this shared card the effective KV
slope is ~950 MiB per 0.01 util, and vLLM refuses to start unless the KV pool
holds >= 1x max-model-len -- util 0.15 undershot (est max-len 47184 < 65536,
crash-loop, ~2-3 min summarizer outage) before 0.18 landed. 65536 is granite's
precedented summarizer ctx; a summarizer doesn't need 131072.

.env.example updated to the new util (max-len was already 65536 in the template;
live had drifted to 131072). persistent-memory.md updated (parked item closed).

persistent-memory.md

@@ -119,7 +119,7 @@ _As of 2026-07-15 — a long fleet-ops + infra session. Everything below either
 
 **PARKED (operator-directed holds):**
 - **image-bench (qwen-image-bench, GPU1) eviction — DONE 2026-07-15 (operator-directed, same-day as the switch).** Stopped `vllm-qwen-image-bench` on ana-ml2 (Exited(0), revert via `docker compose start`) → GPU1 freed ~32 GB (95→63 GB used). LiteLLM `image-judge` + `qwen-image-bench` aliases repointed to gen (:8015, qwen3.6-35b-a3b-heretic), deterministic judge sampling (temp0/top_k1/rep_pen1.05) + `enable_thinking:false` kept; verified live with :8014 down. comfy-dev pinged as regression watcher (NSFW-anatomy + tight-grounding; thread 01KXMRDJ). Backstop = the stopped stack (revert ~90 s) + live config backup `config.yaml.bak-pre-imagejudge-20260715`. Full record: auto-memory `project_arbo_gen_switch_imagebench_evict`.
-- **granite right-size — un-started GPU1 win.** granite (GPU1) is ~29 GB (util 0.34 -> ~16.5 GB oversized KV for an 8B summarizer). Right-sizing util 0.34 -> ~0.13 frees ~16 GB WHILE keeping the summarizer (fleet's dedicated summarizer/classifier; aliases granite-4.1-8b/summarizer/classifier). Independent of image-bench; operator was freeing GPU1 for a TBD "something else".
+- **granite right-size — DONE 2026-07-16 (~10.5 GB freed).** util 0.34→0.18 + max-model-len 131072→65536 on `vllm-granite` (GPU1, `/opt/docker/compose/vllm` .env — recreate `vllm-granite` ONLY, it's a shared stack w/ embed/rerank/reward). GPU1 62,641→51,897 MiB (~45 GB free now); KV 6.45 GiB / 84,528 tok / 1.29x@65536, summarizer healthy. Effective slope ~950 MiB KV per 0.01 util + KV must hold ≥1× max-len; util 0.15 undershot (crash-loop, ~2-3 min summarizer outage) before 0.18 landed. Frees GPU1 to relocate a GPU0 model — operator's next call: which (gen :8015 / char-rp :8016 / char-rp-reasoning :8018). Backup `.env.bak-pre-granite-rightsize-20260716`.
 
 **Open loops for the fresh session:**
 - **`stacks/heretic2-charrp-reasoning/` is UNTRACKED in git** (`?? stacks/heretic2-charrp-reasoning/`) — the char-rp-reasoning compose stack is deployed but never committed; commit it (canonical mirror of the live seat).
@@ -128,6 +128,8 @@ _As of 2026-07-15 — a long fleet-ops + infra session. Everything below either
 
 ## Recent decisions
 
+- `[2026-07-16]` **granite right-sized → ~10.5 GB freed on GPU1** (util 0.34→0.18 + max-len 131072→65536; KV 6.45 GiB / 1.29x@65536, summarizer healthy). GPU1 now ~45 GB free to relocate a GPU0 model. LESSON: ~950 MiB KV per 0.01 util here + KV must hold ≥1× max-len — util 0.15 crash-looped (est max-len 47184<65536, ~2-3 min summarizer blip) before 0.18 landed. `.env`-only, recreate `vllm-granite` alone (shared stack).
+
 - `[2026-07-15]` **image-bench eviction DONE (parked item closed).** Stopped vllm-qwen-image-bench (ana-ml2 GPU1, ~32 GB freed); LiteLLM `image-judge`+`qwen-image-bench` → gen :8015 (judge samplers + thinking-off), verified with :8014 down; comfy-dev pinged; also backfilled the canonical char-rp-reasoning litellm block (was lagging live). Revert ~90 s. auto-memory `project_arbo_gen_switch_imagebench_evict`.
 
 - `[2026-07-15]` arbo fully switched off image-judge (qwen-image-bench) -> gen; image-bench pending eviction post-bake → `persistent-memory.d/2026-07-15-arbo-fully-switched-off-image-judge-qwen-image.md`

stacks/vllm/.env.example

@@ -107,4 +107,10 @@ GRANITE_KV_CACHE_DTYPE=fp8
 # qwen36-vl 0.46 + granite 0.24 + reward 0.10 + embed/rerank 0.03 ≈ 0.90 total,
 # ~7.5 GB headroom (the OOM buffer; held under 20-concurrent load test). granite
 # gets a 169K-token KV pool = 2.58x concurrency @ 64K. Bring qwen36 up LAST.
-GRANITE_GPU_MEM_UTIL=0.24
+# 0.18 — RIGHT-SIZED 2026-07-16 (was 0.34 live; qwen36-vl no longer a GPU-1 tenant,
+# image-bench evicted) to free ~10.5 GB for relocating a GPU0 model onto GPU1. KV
+# 6.45 GiB = 84,528 tokens = 1.29x concurrency @ 65536 (summarizer = short parallel
+# calls; ample). Effective slope on this shared card ≈ 950 MiB KV per 0.01 util, and
+# KV must hold >= 1x max-model-len — util 0.15 undershot (crash: est max-len 47184 <
+# 65536), 0.18 lands the target cleanly.
+GRANITE_GPU_MEM_UTIL=0.18