fix(fv-ml1): pin vLLM KV in bytes on meromero (22 GiB) and coder; gen-small KV 8 -> 5.5 GiB

meromero-rp sized its KV from --gpu-memory-utilization, and vLLM's profile
counts co-booting neighbours as its own non-torch memory. Three boots gave
20.37 / 26.74 / 29.04 GiB of KV, and the 2026-10-05 Docker-move restart cut
GPU 1 free memory to 3,981 MiB, below intern-decision's 32k-token budget.
Coder read negative KV on every co-boot. Both are now pinned, so util only
gates the boot check (meromero util 0.52 -> 0.47, just above its footprint).

GPU 0 plateaus at ~60 MiB free 1-2 days after a boot (Beszel), so
gen-small's KV drops to 5.5 GiB (461,215 tokens, 1.76x) and parakeet-nemo
stays on GPU 0 with room for its 3,840 MiB cap. GPU 3 stays clear.

Applied with playbooks/fv-ml1-kv-pin-20261005.yaml: GPU 1 free 3,981 ->
11,801 MiB, GPU 0 495 -> 5,039 MiB right after boot; all nine seats healthy.

stacks/gen-small-seat/.env.example

@@ -24,8 +24,12 @@ GEN_SMALL_SERVED_NAME_THINK=gen-small-thinking
 # the boot check (Free at start must be >= util x total). Lowering it freed GPU 0 slack for parakeet-nemo
 # at zero KV cost (still 670,142 tokens / 2.56x). Before ANY restart, check the arithmetic:
 # util x 95.6 GiB <= measured Free + this seat's own resident memory. Do not trial-boot.
+# 2026-10-05 (Prime): KV 8 -> 5.5 GiB (~461k tokens, ~1.76x) so parakeet-nemo can stay on GPU 0.
+# The "Free ~1.05 GB at rest" on 10-01 was a just-after-restart read: per Beszel the card plateaus
+# at ~60 MiB free after 1-2 days of creep. 2.5 GiB back covers parakeet's hard cap (MEM_CAP_MIB
+# 3840, i.e. +1,732 MiB over its 2,108 rest) with ~0.9 GB spare at the plateau.
 GEN_SMALL_GPU_MEM_UTIL=0.33
-GEN_SMALL_KV_CACHE_MEMORY=8589934592
+GEN_SMALL_KV_CACHE_MEMORY=5905580032
 GEN_SMALL_MAX_MODEL_LEN=262144
 GEN_SMALL_MAX_NUM_SEQS=16
 GEN_SMALL_MAX_NUM_BATCHED_TOKENS=4096

stacks/gen-small-seat/compose.yaml

@@ -58,7 +58,7 @@ services:
       # avoids the util-fraction fight between co-resident seats. fp8 KV + an A3B's small
       # attention footprint make this cheap; tuned to a sane concurrency band, not max.
       - --kv-cache-memory
-      - "${GEN_SMALL_KV_CACHE_MEMORY:-8589934592}"
+      - "${GEN_SMALL_KV_CACHE_MEMORY:-5905580032}"
       - --max-model-len
       - "${GEN_SMALL_MAX_MODEL_LEN:-262144}"
       - --max-num-seqs

stacks/intern-decision/README.md

@@ -87,6 +87,12 @@ endpoint to hit 32k tokens if possible"). With scriberr gone, GPU 1 holds only t
 service. This container may therefore use its rest (8,812 MiB) plus GPU 1's nvidia-smi `Free` (6,625 MiB), which is
 **15,437 MiB**. The 14.4 GiB cap plus the ~660 MiB outside the allocator puts the card ceiling at ~15,408 MiB.
 
+> ⚠ **2026-10-05:** the 0234 Docker-move restart broke this budget. `vllm-meromero-rp` sized its KV from
+> `--gpu-memory-utilization`, re-profiled during a co-boot, and came back 8.7 GiB larger, so `Free` fell to
+> 3,981 MiB, about 2.4 GB short of a 32k call. Fixed 2131–2135 the same night: meromero and coder now pin KV in
+> bytes (`playbooks/fv-ml1-kv-pin-20261005.yaml`), and `Free` reads **11,801 MiB**, a 20,613 MiB budget, so
+> ~5.4 GB is spare at the 15,220 peak. The static seats can no longer drift across restarts.
+
 Measured on the live service (per-process nvidia-smi every 0.1 s; single `noul` question unless noted; n=3, deterministic):
 
 | call tokens | card peak MiB | wall (warm) |

stacks/meromero-charrp/.env.example

@@ -26,6 +26,15 @@
 # stopped fitting on the 24th": nothing about this seat changed, gen crept up
 # underneath it. Headroom arithmetic done against a long-running gen is
 # measuring a moving number.
+#
+# 2026-10-05: the seat lives on fv-ml1 GPU 1 (MEROMERO_GPU_ID=1 on the host),
+# shared with erp-seat, coder and intern-decision. KV is now PINNED in bytes
+# (Prime: 22 GiB, ~346k tokens, ~1.32x at 262k). Util-sized KV was a co-boot
+# lottery (20.37 / 26.74 / 29.04 GiB on three boots) and the 29.04 boot ate
+# intern-decision's 32k headroom. Util now only gates the boot check, so it
+# sits just above the real footprint (~43.6 GiB). Before ANY restart:
+# util x 95.0 GiB <= measured Free + this seat's own resident memory.
 MEROMERO_GPU_MEM_UTIL=0.47
+MEROMERO_KV_CACHE_MEMORY=23622320128
 MEROMERO_MAX_MODEL_LEN=262144
 MEROMERO_GPU_ID=0

stacks/meromero-charrp/compose.yaml

@@ -64,7 +64,14 @@ services:
       - --max-num-seqs
       - "${MEROMERO_MAX_NUM_SEQS:-32}"
       - --gpu-memory-utilization
-      - "${MEROMERO_GPU_MEM_UTIL:-0.52}"
+      - "${MEROMERO_GPU_MEM_UTIL:-0.47}"
+      # KV pinned in bytes (2026-10-05, Prime: 22 GiB). Util-sized KV was a lottery on
+      # this shared card: vLLM's profile counts co-booting neighbours as its own non-torch
+      # memory, so three boots gave 20.37 / 26.74 / 29.04 GiB of KV. With this set, vLLM
+      # skips profiling and util only gates the boot check (Free at start >= util x total),
+      # so util stays just above the real footprint (~43.6 GiB).
+      - --kv-cache-memory
+      - "${MEROMERO_KV_CACHE_MEMORY:-23622320128}"
       - --kv-cache-dtype
       - fp8
       - --trust-remote-code

stacks/vllm/.env.example

@@ -88,7 +88,9 @@ CODER_MODEL=Qwen/Qwen2.5-Coder-1.5B
 CODER_SERVED_NAME=qwen2.5-coder-1.5b
 CODER_MAX_MODEL_LEN=8192
 CODER_KV_CACHE_DTYPE=fp8
-CODER_GPU_MEM_UTIL=0.06
+CODER_GPU_MEM_UTIL=0.055
+# KV pinned 2026-10-05 (77,056 tokens; see compose.yaml). Util now only gates the boot check.
+CODER_KV_CACHE_MEMORY=1104674816
 CODER_MAX_NUM_SEQS=32
 
 # === LFM2.5-2.6B — RETIRED PERMANENTLY 2026-08-20 (operator directive) ===

stacks/vllm/compose.yaml

@@ -79,6 +79,11 @@ services:
       - "8000"
       - --gpu-memory-utilization
       - ${CODER_GPU_MEM_UTIL}
+      # KV pinned in bytes (2026-10-05): GPU 1 is shared, and util-sized profiling read
+      # negative KV on every co-boot (crash, then a retry). 1,104,674,816 B = 4,816 blocks
+      # = 77,056 tokens, the size every successful boot had landed on.
+      - --kv-cache-memory
+      - ${CODER_KV_CACHE_MEMORY:-1104674816}
       - --max-model-len
       - ${CODER_MAX_MODEL_LEN}
       - --max-num-seqs