Files

T

vh ff8dfed498 docs(fv-ml1): regenerate seat inventory — cyberprev in the sec seat, mog-sec gone

Live-read regeneration after promoting cyberprev into sec/sec-reasoning and
retiring mog-sec. GPU0 now carries cyberprev alone; sec/sec-reasoning aliases
resolve to :8025.

2026-09-14 09:11:07 -07:00

5.1 KiB

Raw Blame History

fv-ml1 — GPU seat inventory and model lineage

Generated 2026-09-14 16:10 UTC by scripts/seat-inventory.py, read from the running containers on 100.64.0.7 — docker inspect, nvidia-smi, each model's own config.json, and the .PROVENANCE.txt siblings on /tank.

⚠ .PROVENANCE.txt lives beside the model directory, not inside it: /tank/aimodels/<model>.PROVENANCE.txt. ls <model>/ will not show it.

Placement, KV cache and concurrency

GPU	seat	VRAM	weights	KV tokens	ctx	concurrency	util
0	`vllm-cyberprev`	38.0 GiB	25.11 GiB	166,264	163840	1.01×	0.40
1	`vllm-meromero-rp`	37.9 GiB	19.51 GiB	266,334	262144	1.02×	0.40
1	`vllm-erp-seat`	26.3 GiB	15.9 GiB	534,649	262144	2.04×	0.30
1	`vllm-reward`	9.0 GiB	4.41 GiB	26,224	16384	1.60×	0.10
1	`vllm-coder`	6.0 GiB	2.98 GiB	112,624	8192	13.75×	0.06
1	`vllm-embed`	3.4 GiB	1.12 GiB	10,272	8192	1.25×	0.03
1	`vllm-rerank-a3`	2.1 GiB	1.06 GiB	—	8192	—	0.03
2	`vllm-flash-next`	93.1 GiB	76.82 GiB	344,155	262144	1.31×	0.96

Concurrency = KV tokens ÷ context: how many full-length requests fit at once. Below ~1.0× the seat cannot hold even one conversation at its declared context.

Lineage and quantization

`vllm-cyberprev` — GPU 0

serves: cyberprev-27b, cyberprev-27b-thinking
model: /tank/aimodels/cyberprev-nvfp4-mixed-v2
architecture: Qwen3_5ForConditionalGeneration (qwen3_5), 64 layers
quantization: compressed-tensors / mixed-precision — W8A8 (float-quantized), W4A4 (nvfp4-pack-quantized)
speculative decoding: {"method": "dflash", "model": "/drafter", "num_speculative_tokens": 7}
image: vllm/vllm-openai:nightly-e9d1398d9edfd90fcc1cf783805240e3effec013

`vllm-coder` — GPU 1

serves: qwen2.5-coder-1.5b
model: ?
image: vllm/vllm-openai:latest ⚠ floating tag

`vllm-embed` — GPU 1

serves: Qwen/Qwen3-Embedding-0.6B
model: ?
image: vllm/vllm-openai:latest ⚠ floating tag

`vllm-erp-seat` — GPU 1

serves: G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16
model: /tank/aimodels/G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16
architecture: Gemma4ForConditionalGeneration (gemma4), 30 layers, 128 experts
quantization: compressed-tensors / nvfp4-pack-quantized — W4A16 (nvfp4-pack-quantized)
image: vllm/vllm-openai:nightly-311b3513af33bc29b4acb2fde2e9313e5e9966a0

`vllm-meromero-rp` — GPU 1

serves: char-rp, char-rp-thinking
model: /tank/aimodels/meromero-v2-nvfp4-work/G4-MeroMero-v2-31B-NVFP4A16
architecture: Gemma4ForConditionalGeneration (gemma4), 60 layers
quantization: compressed-tensors / nvfp4-pack-quantized — W4A16 (nvfp4-pack-quantized)
image: vllm/vllm-openai:v0.26.0

`vllm-rerank-a3` — GPU 1

serves: BAAI/bge-reranker-v2-m3
model: ?
image: vllm/vllm-openai:v0.24.0

`vllm-reward` — GPU 1

serves: Skywork/Skywork-Reward-V2-Llama-3.1-8B-AWQ
model: /tank/aimodels/llm/Skywork-Reward-V2-Llama-3.1-8B-AWQ
architecture: LlamaForSequenceClassification (llama), 32 layers
quantization: compressed-tensors / pack-quantized — W4A16 (pack-quantized)
image: vllm/vllm-openai:latest ⚠ floating tag

`vllm-flash-next` — GPU 2

serves: qwen3.8-flash-next-uncensored, qwen3.8-flash-next-uncensored-thinking
model: /tank/aimodels/qwen38-flash-next-orcarouter-nvfp4-plefp8
architecture: Qwen4ExpForConditionalGeneration (qwen4_exp), 48 layers, 512 experts
quantization: compressed-tensors / mixed-precision — W8A16 (naive-quantized), W4A16 (nvfp4-pack-quantized)
speculative decoding: {"method": "mtp", "num_speculative_tokens": 3}
image: vllm/vllm-openai:nightly-eed1f3d0c6043bd494424a22443ee198dd56f657

Gateway aliases resolving to this host

19 aliases. Ports with no listening seat are marked dead.

alias	port
`char-rp`	8016
`char-rp-fast`	8021
`char-rp-reasoning`	8016
`chat-judge`	8022
`classifier`	8022
`coder-fast`	8020
`erp-tune-v2`	8098
`gemma4-26b-a4b-it-base`	8099
`gen`	8022
`gen-large`	8022
`gen-reasoning`	8022
`image-judge`	8022
`qwen-image-bench`	8022
`qwen3-embedding`	8001
`reranker`	8013
`sec`	8025
`sec-reasoning`	8025
`summarizer`	8022
`summarizer-large`	8022

Regenerate with scripts/seat-inventory.py after ANY seat change — model swap, quant change, context or utilization edit, or speculative-decoding change. Run --check in CI to catch a stale document.

5.1 KiB Raw Blame History Unescape Escape

fv-ml1 — GPU seat inventory and model lineage

Placement, KV cache and concurrency

Lineage and quantization

vllm-cyberprev — GPU 0

vllm-coder — GPU 1

vllm-embed — GPU 1

vllm-erp-seat — GPU 1

vllm-meromero-rp — GPU 1

vllm-rerank-a3 — GPU 1

vllm-reward — GPU 1

vllm-flash-next — GPU 2