feat(qwen36-vl): swap FP8→NVFP4 + GPU1 rebalance (granite restored)

The nvidia ModelOpt NVFP4 MoE that failed on vLLM 0.19.1/0.22.0 (#44081)
loads clean on 0.23.0. Cut prod qwen36 FP8→NVFP4: ~20.4 GiB weights vs
~34 (~40% lighter, ~13 GB reclaimed on GPU 1), faster single-stream on
Blackwell FP4 cores, vision tower preserved (comfy-dev real anatomy-judge
A/B on 16 prod images: PASS; brokkr text/speed: parity bar a minor
multi-step-chained-reasoning slip that doesn't bite the judge role).

- compose: pin image by 0.23.0 digest, drop --kv-cache-dtype fp8 (fp16 KV
  — the freed room buys full-precision KV), util 0.46→0.32.
- GPU1 rebalance (pinned): granite restored 0.24→0.34 / 65536→131072
  (undoes the FP8-era sacrifice); trio unchanged; total ~0.82, ~24 GB free.
- gateway model name qwen3.6-35b-a3b unchanged (now NVFP4 behind it);
  thinking-split (enable_thinking=false default) intact — the judge needs it.

stacks/qwen36-vl/compose.yaml

@@ -1,19 +1,22 @@
-# qwen36-vl — Qwen3.6-35B-A3B vision-language MoE (official FP8) on ana-ml2.
+# qwen36-vl — Qwen3.6-35B-A3B vision-language MoE (official NVFP4) on ana-ml2.
 #
 # Replaces the qwen35-vl stack (Qwen3.5-9B) 2026-06-14. Co-located on GPU 1 with
-# the granite summarizer + embed/rerank/reward trio (GPU 0 stays free for the
-# llama-swap creative-writing hot-swap card). Serves on :8007.
+# the granite summarizer + embed/rerank/reward trio. Serves on :8007.
 #
-# WHY official FP8 (not NVFP4): NVFP4 (nvidia/Qwen3.6-35B-A3B-NVFP4, ~21 GB) is the
-# lighter fit but its vLLM ModelOpt-MoE loader is BROKEN as of 0.19.1/0.22.0
-# (KeyError w2_input_scale / lm_head.input_scale — vLLM #44081). The official
-# Qwen pre-quantized FP8 (~34 GB weights) loads clean on :latest and — unlike
-# the old qwen35-vl — needs NO pinned nightly digest: that hack existed because
-# vLLM DYNAMIC `--quantization fp8` quantized the vision tower to noise. This
-# checkpoint is PRE-quantized, so we OMIT --quantization (vLLM auto-detects the
-# checkpoint's own fp8) and the vision tower is preserved. Validated 2026-06-14
-# on GPU 0: loads in 34.2 GiB, image test returns correct ("Blue"). Revisit
-# NVFP4 (frees ~13 GB) once vLLM's loader is fixed.
+# WHY NVFP4 now (swapped FROM FP8 2026-06-15): the nvidia ModelOpt NVFP4 MoE that
+# was BROKEN on vLLM 0.19.1/0.22.0 (#44081, lm_head.input_scale) loads clean on
+# 0.23.0 — the ModelOpt lm_head fix landed. So we cut FP8→NVFP4: ~20.4 GiB weights
+# vs FP8's ~34 GiB (~40% lighter, ~13 GB reclaimed on GPU 1), faster single-stream
+# on Blackwell's FP4 tensor cores, and the freed room funds fp16 KV + a granite
+# context restore (see the rebalance note below). The NVFP4 checkpoint preserves
+# the vision tower (ModelOpt leaves it high-precision) — VALIDATED by comfy-dev's
+# real anatomy-judge A/B on 16 prod images: PASS, holds the load-bearing
+# discrimination (gross-deformity reject + clean-pass), only shuffles already-
+# unreliable sub-ceiling borderline-hand calls. brokkr's text/speed arm: parity
+# except a minor multi-step chained-numeric-reasoning slip (W4A4 tell) — doesn't
+# bite the vision-judge role; flag for any gateway consumer doing chained math.
+# Requires vLLM >= 0.23.0 (pinned by digest in .env). NO --quantization flag
+# (vLLM auto-detects the checkpoint's NVFP4).
 #
 # NAMING: served ONLY as its TRUE name `qwen3.6-35b-a3b`. A model is never aliased
 # under a prior model's name — a caller asking for `qwen3.5-9b-fp8` (a 9B dense)
@@ -22,10 +25,14 @@
 # hero-judge, stacks/arbo v0.11.3+) migrate to `qwen3.6-35b-a3b` — they 404 on the
 # old name until they repoint, which is the correct loud signal (notified 2026-06-14).
 #
-# WHY util 0.42 / max-len 131072: hybrid attn (10 of 40 layers full-attn, ~10 KB/
-# tok KV) → KV is cheap, so big context is nearly free; the 34 GB weights are the
-# cost. 0.42 (~40 GB) = weights + graph + generous KV. Granite drops to 0.25/64K
-# to make room (the FP8-vs-maxed-granite tradeoff, operator-approved 2026-06-14).
+# GPU-1 REBALANCE (2026-06-15, pinned): the NVFP4 swap freed ~13 GB, redistributed —
+#   qwen36 NVFP4: util 0.46→0.32 (~31 GB: 20.4 GB weights + fp16 KV + graph).
+#     fp16 KV (we DROPPED --kv-cache-dtype fp8) — the freed room buys back full-
+#     precision KV; hybrid attn (10/40 full-attn) keeps even fp16 KV affordable.
+#   granite: RESTORED 0.24→0.34, max-len 65536→131072 (gives back the context
+#     sacrificed for FP8 qwen — the FP8-vs-maxed-granite tradeoff is now undone).
+#   trio (embed/rerank/reward) unchanged at floor.
+#   Total GPU-1 util ~0.82 → ~17 GB headroom (was a tight ~5 GB).
 #
 # THINKING TOGGLE: this is ONE hybrid checkpoint (not separate Instruct/Thinking
 # downloads) with a Qwen3-style per-request `enable_thinking` switch. The chat
@@ -60,8 +67,8 @@ services:
       - VLLM_API_KEY=${API_KEY:-}
     command:
       - ${QWEN_MODEL}
-      # Pre-quantized FP8 checkpoint → NO --quantization (vLLM auto-detects; a
-      # forced flag would re-quantize the vision tower to noise, see header).
+      # Pre-quantized NVFP4 (ModelOpt) checkpoint → NO --quantization (vLLM auto-
+      # detects; the vision tower is left high-precision by the producer).
       - --served-model-name
       - qwen3.6-35b-a3b
       - --host
@@ -78,8 +85,8 @@ services:
       # vision endpoint (the summarizer carries the concurrency, not this).
       - --max-num-seqs
       - ${QWEN_MAX_NUM_SEQS}
-      - --kv-cache-dtype
-      - fp8
+      # fp16 KV (no --kv-cache-dtype): the NVFP4 swap freed enough room to run
+      # full-precision KV — better than the fp8 KV the FP8 build needed to fit.
       - --trust-remote-code
       - --dtype
       - auto
@@ -110,9 +117,9 @@ services:
       - tnet
     labels:
       - homepage.group=AI Systems
-      - homepage.name=Qwen3.6-35B-A3B VL (FP8)
+      - homepage.name=Qwen3.6-35B-A3B VL (NVFP4)
       - homepage.icon=mdi-image-search
-      - homepage.description=Qwen3.6-35B-A3B vision-language MoE (FP8) via vLLM (ana-ml2)
+      - homepage.description=Qwen3.6-35B-A3B vision-language MoE (NVFP4) via vLLM (ana-ml2)
       - homepage.href=http://10.250.50.54:${QWEN_PORT}/docs
 
 networks: