The seat documentation must stay current, and a hand-written document cannot.
The LiteLLM config described char-rp as a 31B model on a host and GPU it had not
been on since 2026-08-24 -- three weeks of silent drift in a file that read as
authoritative, and the reason a seat spent that period serving a model nobody
intended. Anything typed here drifts the same way; anything read off the running
containers cannot.
scripts/seat-inventory.py derives the whole document from the host:
- placement and VRAM from nvidia-smi compute-apps, mapped to containers through
/proc/<pid>/cgroup -- nvidia-smi reports the vLLM engine child while docker
reports the container pid, so matching them directly silently yields nothing
- weights and KV tokens parsed from each engine's own startup log, not derived
arithmetically, with concurrency computed as KV tokens over context
- architecture, layer and expert counts, and the exact quantization group scheme
(W4A4 vs W4A16 distinguished) from each model's config.json
- speculative-decoding method and k from the container argv, which is how the
three incompatible methods on this box became visible
- lineage from the .PROVENANCE.txt SIBLING files -- they sit beside the model
directory, not inside it, which is why an earlier pass wrongly reported two
fully-documented seats as having no provenance
- gateway aliases resolved from the LiteLLM config on ana-docker
--check compares the committed document against the live box and exits non-zero
when they diverge, ignoring only the generation timestamp. Suitable for CI or a
scheduled drift alarm; read-only throughout, safe against production.
Also commits the KV_CACHE_BYTES override added to the MTP campaign runner, which
asserts the flag exists in the derived argv and aborts rather than running a
campaign that silently ignored it.
Read from docker inspect, nvidia-smi and each model's own config.json rather
than from compose files or the gateway config, so the document records what is
deployed rather than what was intended.
Findings worth naming:
- char-rp drift. The LiteLLM config documents char-rp as G4-MeroMero-v2-31B on
ana-ml2 GPU 0; :8016 actually serves gemma4-26b-a4b-it-nvfp4, a 26B 30-layer
128-expert stock Gemma-4, on fv-ml1 GPU 1. Host renaming explains part of it,
but 31B MeroMero-v2 and 26B stock gemma4 are different models and the comment
block carries sampler defaults keyed to the model it names.
- Two gateway aliases are dead: erp-tune-v2 (:8098) and gemma4-26b-a4b-it-base
(:8099). Nothing is listening on either; callers get a connection failure
rather than a clear retirement error.
- Three different speculative-decoding methods coexist -- dflash k=7 with a
separate drafter on mog-sec, qwen3_5_mtp k=3 on gen, mtp k=3 on flash-next.
The method string is architecture-specific; copying a --speculative-config
between seats without changing it will not work.
- Three seats run the floating :latest tag, so what they would pull on recreate
is not what is running and was never recorded.
- Seven aliases resolve to vllm-gen on :8015, which is why GPU 0 shows draw
during ordinary OpenWebUI use -- its auto-titling fires summarizer per turn.
Also commits the flash-next-seat MTP k=3 change made today: compose gains the
speculative-config with its measured rationale, and .env.example documents the
KV-cache reduction MTP requires (14 GiB OOMs, 10 GiB verified in production).
vh