Read from docker inspect, nvidia-smi and each model's own config.json rather
than from compose files or the gateway config, so the document records what is
deployed rather than what was intended.
Findings worth naming:
- char-rp drift. The LiteLLM config documents char-rp as G4-MeroMero-v2-31B on
ana-ml2 GPU 0; :8016 actually serves gemma4-26b-a4b-it-nvfp4, a 26B 30-layer
128-expert stock Gemma-4, on fv-ml1 GPU 1. Host renaming explains part of it,
but 31B MeroMero-v2 and 26B stock gemma4 are different models and the comment
block carries sampler defaults keyed to the model it names.
- Two gateway aliases are dead: erp-tune-v2 (:8098) and gemma4-26b-a4b-it-base
(:8099). Nothing is listening on either; callers get a connection failure
rather than a clear retirement error.
- Three different speculative-decoding methods coexist -- dflash k=7 with a
separate drafter on mog-sec, qwen3_5_mtp k=3 on gen, mtp k=3 on flash-next.
The method string is architecture-specific; copying a --speculative-config
between seats without changing it will not work.
- Three seats run the floating :latest tag, so what they would pull on recreate
is not what is running and was never recorded.
- Seven aliases resolve to vllm-gen on :8015, which is why GPU 0 shows draw
during ordinary OpenWebUI use -- its auto-titling fires summarizer per turn.
Also commits the flash-next-seat MTP k=3 change made today: compose gains the
speculative-config with its measured rationale, and .env.example documents the
KV-cache reduction MTP requires (14 GiB OOMs, 10 GiB verified in production).
vh
2d83a895c1