feat(erp-seat): Pfish-6 replaces run 7 — run-6 NVFP4 quant on both boxes, max-num-seqs 8->32

Operator ruling: run 6 is declared Pfish-6 and is the standing seat; run 7 is
retired (its gate failure was the detector bug fixed in cc42d76, but the run was
independently poor).

- served under its TRUE name Pfish-6; LiteLLM alias trial -> Pfish-6
- ana-ml2 :8021 and pfi-gx10 :8098, both at 262,144 ctx, same artifact
- SPEED: moe_backend=flashinfer_cutedsl PROVEN unusable here (engine init fails,
  'kernel does not support current device'); Marlin is correct for a weight-only
  scheme, and vLLM's 'no native FP4' warning is about the scheme not the card
- the real lever was max-num-seqs 8 -> 32: n=8 1269 tok/s and n=16 2170 tok/s,
  3.2x the old ceiling, with single-stream latency unchanged
- head-to-head measured: ana-ml2 is 4.1x FASTER than the GX10, not slower
This commit is contained in:
vh
2026-09-09 19:16:36 -07:00
parent 202e7f60e1
commit b8956b58fa
3 changed files with 114 additions and 31 deletions
+46
View File
@@ -230,3 +230,49 @@ result; it makes a bad one moot.**
Re-testing the opening-split idea is a fresh run on a clean base, not a re-read of
this one — and it is the operator's call, not a default.
## Postscript — run 7 retired, Pfish-6 is the standing seat (2026-09-09)
Operator ruling: *"declare run 6 as Pfish-6 … we're gonna stay on 6 for now."*
Run 7's gate failure was a **detector bug** (the adjective "minor" in a HARD rule,
fixed `cc42d76`), but run 7 was independently a poor run and is not returning.
`Pfish-6` = `erp-tune-v6-nvfp4a16`, served under that name on **both** boxes at
262,144 context for a head-to-head.
### Head-to-head, same artifact, same context, 3 rounds each
| | ana-ml2 (RTX PRO 6000 Blackwell) | pfi-gx10 (GB10) |
|---|---|---|
| n=1 | **219 tok/s** | 53.6 tok/s |
| n=4 aggregate | **676 tok/s** | 175 tok/s |
| KV pool | 534,649 tok (2.04× ctx) | 4,608,942 tok (17.6× ctx) |
⚠ **ana-ml2 is ~4.1× FASTER, not slower.** The expectation going in was the
reverse. Decode is memory-bandwidth-bound and the discrete card's GDDR7 has
several times the bandwidth of the GX10's unified LPDDR5X; the GX10's advantage is
capacity (a 4.6M-token KV pool) and that is not what single-stream decode uses.
### Kernel: Marlin is correct here, and this was PROVEN not assumed
vLLM warns *"your GPU does not have native support for FP4 computation"* on
ana-ml2, which is **misleading — sm_120 does have native FP4**. NVFP4A16 is
weight-only: activations stay 16-bit, so no FP4 GEMM is issued. Forcing
`moe_backend=flashinfer_cutedsl` fails at engine init with *"does not support the
deployment configuration since kernel does not support current device cuda"*.
`auto` → MARLIN is already optimal for this scheme. A W4A4 build would use the
FP4 units, at the fidelity cost already settled against for this seat.
### The real lever was concurrency
`max-num-seqs` was **8**, capping the seat far below its KV pool. Raised to 32
(CUDA-graph capture sizes followed to 64). Measured on ana-ml2:
| streams | aggregate tok/s |
|---|---|
| 1 | 219 |
| 4 | 676 |
| 8 | **1,269** — unreachable at the old cap |
| 16 | **2,170** — 3.2× the old ceiling |
Single-stream latency is **unchanged**, so the raise costs nothing.