vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(fv-ml1): add curated LLM seat catalog (lineage/provenance/cards/speed) + bench script

Browse Source 
Adds docs/pfi/llm-seat-catalog.md, the durable hand-curated record of what each
seat IS -- lineage, provenance, model-card facts, quantization, speculative
decoding, licenses, and measured warm tok/s + deep-prefill depth results with
their harness and date. It complements the auto-generated
fv-ml1-gpu-seat-inventory.md (live placement/KV/concurrency): the two split by
volatility, and the catalog defers to the inventory for any live number.

Adds scripts/seat-bench.py so the catalog's speed/depth numbers are reproducible
(warm decode tok/s, n=3, conc=1, median; deep prefill at ~0.97x max-model-len
with an allocator-log OOM scan). Serial by design -- concurrent deep prefills
would confound both OOM and tok/s.

Captures the 2026-09-14 measurements: all six generative seats prefill to ~255K
(coder ~16K) with zero OOM/CUBLAS/illegal-memory; warm decode 62.7-337.3 tok/s;
per-seat VRAM. seat-inventory.py now cross-links the catalog in its footer.
This commit is contained in:
vh
2026-09-14 10:45:31 -07:00
parent 05f3d3b2a3
commit dfa91a8eaf
4 changed files with 266 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/pfi/fv-ml1-gpu-seat-inventory.md
+7 -5
View File
@@ -6,7 +6,7 @@
Hand-edits are overwritten and, worse, drift silently while looking
authoritative — which is exactly the failure this file replaced. -->
**Generated 2026-09-14 17:24 UTC** by `scripts/seat-inventory.py`, read from the running
**Generated 2026-09-14 17:45 UTC** by `scripts/seat-inventory.py`, read from the running
containers on `100.64.0.7` — `docker inspect`, `nvidia-smi`, each model's own
`config.json`, and the `.PROVENANCE.txt` siblings on `/tank`.
@@ -18,11 +18,11 @@ containers on `100.64.0.7` — `docker inspect`, `nvidia-smi`, each model's own
| GPU | seat | VRAM | weights | KV tokens | ctx | concurrency | util |
|---|---|---|---|---|---|---|---|
| 0 | `vllm-cyberprev` | 46.0 GiB | 25.11 GiB | 359,372 | 262144 | **1.37×** | 0.40 |
| 0 | `vllm-gen-small` | 35.3 GiB | 23.98 GiB | 670,142 | 262144 | **2.56×** | 0.48 |
| 1 | `vllm-meromero-rp` | 41.3 GiB | 19.51 GiB | 320,774 | 262144 | **1.22×** | 0.52 |
| 1 | `vllm-erp-seat` | 26.4 GiB | 15.9 GiB | 534,649 | 262144 | **2.04×** | 0.24 |
| 0 | `vllm-gen-small` | 35.8 GiB | 23.98 GiB | 670,142 | 262144 | **2.56×** | 0.48 |
| 1 | `vllm-meromero-rp` | 43.0 GiB | 19.51 GiB | 320,774 | 262144 | **1.22×** | 0.52 |
| 1 | `vllm-erp-seat` | 27.1 GiB | 15.9 GiB | 534,649 | 262144 | **2.04×** | 0.24 |
| 1 | `vllm-reward` | 9.0 GiB | 4.41 GiB | 26,224 | 16384 | **1.60×** | 0.10 |
| 1 | `vllm-coder` | 5.5 GiB | 2.98 GiB | 77,056 | 16384 | **4.70×** | 0.055 |
| 1 | `vllm-coder` | 6.1 GiB | 2.98 GiB | 77,056 | 16384 | **4.70×** | 0.055 |
| 1 | `vllm-embed` | 3.4 GiB | 1.12 GiB | 10,272 | 8192 | **1.25×** | 0.03 |
| 1 | `vllm-rerank-a3` | 2.1 GiB | 1.06 GiB | — | 8192 | — | 0.03 |
| 2 | `vllm-flash-next` | 93.1 GiB | 76.82 GiB | 344,155 | 262144 | **1.31×** | 0.96 |
@@ -133,6 +133,8 @@ context.
---
*Lineage, provenance, model cards, measured tok/s and depth results live in the hand-curated companion [`llm-seat-catalog.md`](llm-seat-catalog.md).*
*Regenerate with `scripts/seat-inventory.py` after ANY seat change —
model swap, quant change, context or utilization edit, or speculative-decoding
change. Run `--check` in CI to catch a stale document.*