memory: snapshot refresh — breeze settled, util does not predict residency, handoff defect

Incremental over 653f7fb, covering the three turns since.

breeze-tts stays on irv-ml1 and the TTS-stack move is parked at id 75. Adds the
full fv-ml1 per-seat residency table and the shuffle arithmetic behind the
recommendation: every arrangement that makes real room on GPU 1 spends the GPU 3
reserve anyway, so if breeze ever moves it should go straight to GPU 3 rather than
evacuating three working seats for the same cost plus a migration.

The finding worth keeping is that --gpu-memory-utilization does not predict resident
VRAM, and is wrong in both directions: cyberprev at util 0.40 holds 8 GB more than
its fraction, gen-small at 0.48 holds 10 GB less. An hour earlier in the session I
reasoned about placement from those fractions and would have been 8 GB out. Read
nvidia-smi --query-compute-apps instead.

Also records a characterized defect in this skill's own handoff generator. Across
two runs in one session it converted operator-deferred items into an imperative
next-steps list and twice invited the next session to commit files marked as
predating it. It fails in the blind spot of the documented fail-loud posture: the
output is structurally valid and exits 0 while inverting the intent. That matters
more than an ordinary bad summary because the handoff is the one artifact a fresh
context reads as instruction. Both runs were corrected in-session; not yet reported
to the skill's owner.
This commit is contained in:
vh
2026-09-15 10:08:57 -07:00
parent 7a33bd9f09
commit 9d36c74572
3 changed files with 75 additions and 6 deletions
@@ -73,6 +73,43 @@ GPU 1 is the constrained card at **0.975 committed / 4,336 MiB free**; GPU 0 has
and `tts-gateway` reach each other by name only through `extra_hosts` pins. Those pins
travel with them and need re-pointing at the new host.
## fv-ml1 per-seat residency, measured 2026-09-15 (97,887 MiB per card)
| GPU | seats (resident MiB) | resident | free |
|---|---|---|---|
| **0** | cyberprev 47,124 · gen-small 36,942 · parakeet 1,178 | 85,244 | **11,982** |
| **1** | meromero 44,056 · erp 27,748 · reward 9,192 · coder 6,292 · embed 3,472 · rerank 2,114 | 92,874 | **4,336** |
| **2** | flash-next 95,344 | 95,344 | 1,767 |
| **3** | — (deliberate reserve) | 2 | **97,247** |
## ⚠⚠ `--gpu-memory-utilization` DOES NOT PREDICT RESIDENCY — measure, don't compute
Capacity planning off the configured util fractions is wrong **in both directions**:
vllm-cyberprev util 0.40 -> expect ~39,155 MiB ACTUAL 47,124 (+8.0 GB OVER)
vllm-gen-small util 0.48 -> expect ~46,986 MiB ACTUAL 36,942 (-10.0 GB UNDER)
An hour earlier in this session I reasoned about placement from the util fractions and
would have been ~8 GB wrong on one card. ⭐ **`util` is a ceiling the engine may not
reach and evidently can exceed** (weights, activations and non-KV allocations sit
outside the KV budget the fraction governs). Always read
`nvidia-smi --query-compute-apps` for the real number.
## Does a shuffle make room for breeze? — worked through
- **GPU 0 today: YES, no shuffle.** 11,982 free vs breeze 10,316 → **1.7 GB margin**,
against a tenant that grew +1.1 GB in an hour, on the live chat path. Fits; thin.
- **GPU 1 today: NO.** 4,336 free.
- **Evacuate embed+rerank+reward (14,778 MiB) → GPU 1 has 19,114 free**, breeze fits
with ~8.8 GB headroom. **But there is nowhere to put those three except GPU 3** —
14,778 does not fit GPU 0's 11,982, and GPU 2 has 1,767.
- **Partial shuffle fails:** embed+rerank (5,586) to GPU 0 leaves GPU 1 at 9,922 free —
**~400 MB short** of breeze, and breeze is growing.
⭐ **Conclusion: every arrangement that makes real room on GPU 1 spends the GPU 3
reserve anyway.** So if breeze ever goes to FV, put it **straight on GPU 3** rather than
evacuating three working seats to achieve the same cost plus a migration.
## If consolidation onto FV is the goal anyway
**GPU 3** fits it comfortably (97,247 MiB free) and has none of the margin problem — but
@@ -111,6 +111,34 @@ possible answer, and it was the right one here.
problem; a refusal naming permission rules or auto-mode is a harness one. Different
fixes, and neither applies when nothing failed.
## ⚠ CHARACTERIZED DEFECT: `/snapshot`'s handoff generator turns deferred items into orders
**n=2, same session, reproducible.** `snapshot_handoff.py` (gen-small) reliably converts
"open, operator-deferred, not blocking" into an imperative **Next steps** list, and twice
invited the next session to commit files explicitly marked as predating the session.
run 1: "Execute deferred operator tasks: AI-tab Dormant regrouping, nconnect=8,
fused MoE (park id 47)" + "Commit graphify-out/… if they are ready"
run 2: six next-steps, FIVE of them deferred/parked items presented as actions,
+ the same commit invitation
⚠ **It fails silently in the skill's blind spot.** The documented failure posture is
fail-loud-fall-back — unreachable gateway, timeout, truncation, missing section → write
nothing, exit non-zero. **A structurally valid handoff whose content inverts the
operator's intent passes every one of those checks** and exits 0.
⚠ **And this is the one artifact a fresh context inherits as instruction.** It is read
immediately after `/clear`, before any other framing, and its Next steps read as a
mandate. A wrong one here is not a bad summary; it is a fresh session going and doing
belayed work.
**Mitigation until fixed: read the generated handoff before accepting it**, and invert
any deferred item into an explicit *do NOT*. Both runs this session were corrected
in-session. Owner is the `/snapshot` skill (galdrabok); not reported yet.
⭐ Same family as everything above — the instrument produced a plausible artifact and
the plausibility is exactly what makes it dangerous.
## Related
`2026-09-15-talk-v10-deploy.md` (#2, and the gate built for it),