Commit Graph

1 Commits

Author SHA1 Message Date
vh 5f049cb4ad feat(sglang): stage vLLM-vs-SGLang bench stack on ana-ml2
SGLang 0.5.13 confirmed to support our formats on Blackwell sm_120
(compressed-tensors NVFP4 W4A4, fp8, modelopt_fp4, petit_nvfp4, fp4_e2m1 KV),
so the bench can be a real NVFP4 head-to-head. Parameterized compose (model/
quant/GPU via .env) + a common streaming load generator (bench.py: agg tok/s,
TTFT p50/p99, TPOT) so both engines are driven identically on an exclusive GPU.
Bench-oriented; promote to a real stack only if SGLang wins. Launch deferred
until the NVFP4 eval frees a GPU.
2026-06-12 22:40:26 -07:00