fix(fv-ml1): pin vLLM KV in bytes on meromero (22 GiB) and coder; gen-small KV 8 -> 5.5 GiB

meromero-rp sized its KV from --gpu-memory-utilization, and vLLM's profile
counts co-booting neighbours as its own non-torch memory. Three boots gave
20.37 / 26.74 / 29.04 GiB of KV, and the 2026-10-05 Docker-move restart cut
GPU 1 free memory to 3,981 MiB, below intern-decision's 32k-token budget.
Coder read negative KV on every co-boot. Both are now pinned, so util only
gates the boot check (meromero util 0.52 -> 0.47, just above its footprint).

GPU 0 plateaus at ~60 MiB free 1-2 days after a boot (Beszel), so
gen-small's KV drops to 5.5 GiB (461,215 tokens, 1.76x) and parakeet-nemo
stays on GPU 0 with room for its 3,840 MiB cap. GPU 3 stays clear.

Applied with playbooks/fv-ml1-kv-pin-20261005.yaml: GPU 1 free 3,981 ->
11,801 MiB, GPU 0 495 -> 5,039 MiB right after boot; all nine seats healthy.
This commit is contained in:
vh
2026-10-05 21:40:28 -07:00
parent 269c80f3ca
commit 6851ecbd0b
9 changed files with 229 additions and 9 deletions
+5
View File
@@ -79,6 +79,11 @@ services:
- "8000"
- --gpu-memory-utilization
- ${CODER_GPU_MEM_UTIL}
# KV pinned in bytes (2026-10-05): GPU 1 is shared, and util-sized profiling read
# negative KV on every co-boot (crash, then a retry). 1,104,674,816 B = 4,816 blocks
# = 77,056 tokens, the size every successful boot had landed on.
- --kv-cache-memory
- ${CODER_KV_CACHE_MEMORY:-1104674816}
- --max-model-len
- ${CODER_MAX_MODEL_LEN}
- --max-num-seqs