memory: snapshot refresh — breeze settled, util does not predict residency, handoff defect

Incremental over 653f7fb, covering the three turns since. breeze-tts stays on irv-ml1 and the TTS-stack move is parked at id 75. Adds the full fv-ml1 per-seat residency table and the shuffle arithmetic behind the recommendation: every arrangement that makes real room on GPU 1 spends the GPU 3 reserve anyway, so if breeze ever moves it should go straight to GPU 3 rather than evacuating three working seats for the same cost plus a migration. The finding worth keeping is that --gpu-memory-utilization does not predict resident VRAM, and is wrong in both directions: cyberprev at util 0.40 holds 8 GB more than its fraction, gen-small at 0.48 holds 10 GB less. An hour earlier in the session I reasoned about placement from those fractions and would have been 8 GB out. Read nvidia-smi --query-compute-apps instead. Also records a characterized defect in this skill's own handoff generator. Across two runs in one session it converted operator-deferred items into an imperative next-steps list and twice invited the next session to commit files marked as predating it. It fails in the blind spot of the documented fail-loud posture: the output is structurally valid and exits 0 while inverting the intent. That matters more than an ordinary bad summary because the handoff is the one artifact a fresh context reads as instruction. Both runs were corrected in-session; not yet reported to the skill's owner.
2026-09-15 10:08:57 -07:00
parent 7a33bd9f09
commit 9d36c74572
3 changed files with 75 additions and 6 deletions
@@ -73,6 +73,43 @@ GPU 1 is the constrained card at **0.975 committed / 4,336 MiB free**; GPU 0 has
 and `tts-gateway` reach each other by name only through `extra_hosts` pins. Those pins
 travel with them and need re-pointing at the new host.

+## fv-ml1 per-seat residency, measured 2026-09-15 (97,887 MiB per card)
+
+| GPU | seats (resident MiB) | resident | free |
+|---|---|---|---|
+| **0** | cyberprev 47,124 · gen-small 36,942 · parakeet 1,178 | 85,244 | **11,982** |
+| **1** | meromero 44,056 · erp 27,748 · reward 9,192 · coder 6,292 · embed 3,472 · rerank 2,114 | 92,874 | **4,336** |
+| **2** | flash-next 95,344 | 95,344 | 1,767 |
+| **3** | — (deliberate reserve) | 2 | **97,247** |
+
+## ⚠⚠ `--gpu-memory-utilization` DOES NOT PREDICT RESIDENCY — measure, don't compute
+
+Capacity planning off the configured util fractions is wrong **in both directions**:
+
+    vllm-cyberprev   util 0.40 -> expect ~39,155 MiB   ACTUAL 47,124   (+8.0 GB OVER)
+    vllm-gen-small   util 0.48 -> expect ~46,986 MiB   ACTUAL 36,942   (-10.0 GB UNDER)
+
+An hour earlier in this session I reasoned about placement from the util fractions and
+would have been ~8 GB wrong on one card. ⭐ **`util` is a ceiling the engine may not
+reach and evidently can exceed** (weights, activations and non-KV allocations sit
+outside the KV budget the fraction governs). Always read
+`nvidia-smi --query-compute-apps` for the real number.
+
+## Does a shuffle make room for breeze? — worked through
+
+- **GPU 0 today: YES, no shuffle.** 11,982 free vs breeze 10,316 → **1.7 GB margin**,
+  against a tenant that grew +1.1 GB in an hour, on the live chat path. Fits; thin.
+- **GPU 1 today: NO.** 4,336 free.
+- **Evacuate embed+rerank+reward (14,778 MiB) → GPU 1 has 19,114 free**, breeze fits
+  with ~8.8 GB headroom. **But there is nowhere to put those three except GPU 3** —
+  14,778 does not fit GPU 0's 11,982, and GPU 2 has 1,767.
+- **Partial shuffle fails:** embed+rerank (5,586) to GPU 0 leaves GPU 1 at 9,922 free —
+  **~400 MB short** of breeze, and breeze is growing.
+
+⭐ **Conclusion: every arrangement that makes real room on GPU 1 spends the GPU 3
+reserve anyway.** So if breeze ever goes to FV, put it **straight on GPU 3** rather than
+evacuating three working seats to achieve the same cost plus a migration.
+
 ## If consolidation onto FV is the goal anyway

 **GPU 3** fits it comfortably (97,247 MiB free) and has none of the margin problem — but