fix(fv-ml1): pin vLLM KV in bytes on meromero (22 GiB) and coder; gen-small KV 8 -> 5.5 GiB
meromero-rp sized its KV from --gpu-memory-utilization, and vLLM's profile counts co-booting neighbours as its own non-torch memory. Three boots gave 20.37 / 26.74 / 29.04 GiB of KV, and the 2026-10-05 Docker-move restart cut GPU 1 free memory to 3,981 MiB, below intern-decision's 32k-token budget. Coder read negative KV on every co-boot. Both are now pinned, so util only gates the boot check (meromero util 0.52 -> 0.47, just above its footprint). GPU 0 plateaus at ~60 MiB free 1-2 days after a boot (Beszel), so gen-small's KV drops to 5.5 GiB (461,215 tokens, 1.76x) and parakeet-nemo stays on GPU 0 with room for its 3,840 MiB cap. GPU 3 stays clear. Applied with playbooks/fv-ml1-kv-pin-20261005.yaml: GPU 1 free 3,981 -> 11,801 MiB, GPU 0 495 -> 5,039 MiB right after boot; all nine seats healthy.
This commit is contained in:
@@ -87,6 +87,12 @@ endpoint to hit 32k tokens if possible"). With scriberr gone, GPU 1 holds only t
|
||||
service. This container may therefore use its rest (8,812 MiB) plus GPU 1's nvidia-smi `Free` (6,625 MiB), which is
|
||||
**15,437 MiB**. The 14.4 GiB cap plus the ~660 MiB outside the allocator puts the card ceiling at ~15,408 MiB.
|
||||
|
||||
> ⚠ **2026-10-05:** the 0234 Docker-move restart broke this budget. `vllm-meromero-rp` sized its KV from
|
||||
> `--gpu-memory-utilization`, re-profiled during a co-boot, and came back 8.7 GiB larger, so `Free` fell to
|
||||
> 3,981 MiB, about 2.4 GB short of a 32k call. Fixed 2131–2135 the same night: meromero and coder now pin KV in
|
||||
> bytes (`playbooks/fv-ml1-kv-pin-20261005.yaml`), and `Free` reads **11,801 MiB**, a 20,613 MiB budget, so
|
||||
> ~5.4 GB is spare at the 15,220 peak. The static seats can no longer drift across restarts.
|
||||
|
||||
Measured on the live service (per-process nvidia-smi every 0.1 s; single `noul` question unless noted; n=3, deterministic):
|
||||
|
||||
| call tokens | card peak MiB | wall (warm) |
|
||||
|
||||
Reference in New Issue
Block a user