perf(gen-seat): record prefill measurements — roughly doubled

Closes the one axis of the original premise left unverified. Measured
cold (cache-busted) on both builds under matching serve configs:

  ~6.7k-token prompt   3,206 -> 6,334 tok/s prefill   (+98%)
  ~27k-token prompt    2,862 -> 5,085 tok/s prefill   (+78%)
  TTFT on a ~27k doc    9.43 -> 5.31 s                (-44%)

Prefill gains far exceed the +18% decode gain, and that ordering is the
expected one: decode at bs=1 is memory-bandwidth-bound and the weights
are 4-bit under either scheme, so little changes; prefill is
compute-bound, which is where native Blackwell FP4 tensor cores replace
the Marlin dequant-to-BF16 path. The summarizer aliases are the
consumers that feel this.

Adds bench/prefill_bench.py plus the raw JSON. The harness deliberately
uses SystemRandom: a seeded nonce regenerates the previous run's prompts
verbatim, prefix caching then serves them, and the first attempt read
~41k tok/s of cache-hit rather than ~5k of actual prefill.

services/gen-seat-mixed-quant/README.md

@@ -90,13 +90,25 @@ Speed alone does not justify cutting over a seat backing 7 LiteLLM aliases.
 | metric | W4A16 (old) | mixed (new) | delta |
 |---|---|---|---|
 | decode tok/s, bs=1, cache-busted | 80.12 | **94.53** | **+18.0%** |
+| **prefill tok/s, ~6.7k prompt** | 3,206 | **6,334** | **+98%** |
+| **prefill tok/s, ~27k prompt** | 2,862 | **5,085** | **+78%** |
+| TTFT on a ~27k-token doc | 9.43 s | **5.31 s** | −44% |
 | MTP acceptance | 47.8% | 47.7% | unchanged |
 | perplexity, 6 held-out passages | 6.941 | 7.059 | +1.7% worse |
 | abliteration compliance | 4/4 | 4/4 | preserved |
 | weights on disk | 27.7 GB | 22.5 GB | −19% |
 
+**Prefill roughly doubled** — the bigger practical win, and exactly what theory predicts:
+decode at bs=1 is memory-bandwidth-bound (weights are 4-bit either way, so little changes),
+while prefill is compute-bound and is where Blackwell's native FP4 tensor cores replace the
+Marlin dequant-to-BF16 path. This is what the `summarizer` / `summarizer-large` aliases feel
+on long documents.
+
 - `quickbench.py` — cache-busted bs=1 decode + MTP acceptance. **Bust the cache:** with
   a fixed prompt, prefix caching returns byte-identical timings and you measure nothing.
+- `prefill_bench.py` — TTFT on long prompts. Same trap, worse: a *seeded* nonce reproduces
+  the previous run's prompts verbatim, so prefix caching serves them and you read ~41k tok/s
+  of cache-hit instead of ~5k of real prefill. Uses `SystemRandom`; never seed it.
 - `eval_quality.py` — perplexity, deterministic generations, abliteration survival.
   **PPL must be measured with `--speculative-config` OFF**: under MTP, vLLM's
   `prompt_logprobs` come back ~uniform over the vocab (median rank ~10^5, logprob

services/gen-seat-mixed-quant/bench/bench_w4a16_recheck.json

@@ -0,0 +1,29 @@
+{
+  "tag": "w4a16-recheck",
+  "model": "probe",
+  "tok_s_median": 77.51086756145159,
+  "tok_s_mean": 77.55102759761677,
+  "tok_s_min": 69.14223827875307,
+  "tok_s_max": 88.11959921725081,
+  "mtp_accept_median": 0.4493388052895577,
+  "rates": [
+    82.25341481185131,
+    77.12830062541333,
+    78.45783182678721,
+    69.14223827875307,
+    71.73740809898078,
+    77.89343449748986,
+    88.11959921725081,
+    75.67599342440771
+  ],
+  "accs": [
+    0.494824016563147,
+    0.44573643410852715,
+    0.4536489151873767,
+    0.36300174520069806,
+    0.3894927536231884,
+    0.45294117647058824,
+    0.56,
+    0.43238095238095237
+  ]
+}

services/gen-seat-mixed-quant/bench/prefill_bench.py

@@ -0,0 +1,52 @@
+#!/usr/bin/env python3
+"""Prefill throughput via TTFT on long prompts (prefill-dominated)."""
+import argparse, json, random, statistics, time, urllib.request
+
+def ttft(base, model, prompt):
+    req = urllib.request.Request(base + "/v1/chat/completions",
+        data=json.dumps({"model": model, "messages": [{"role": "user", "content": prompt}],
+                         "max_tokens": 16, "temperature": 0, "stream": True,
+                         "chat_template_kwargs": {"enable_thinking": False}}).encode(),
+        headers={"Content-Type": "application/json"})
+    t0 = time.perf_counter()
+    with urllib.request.urlopen(req, timeout=900) as r:
+        for line in r:
+            if line.startswith(b"data: ") and b"[DONE]" not in line:
+                dl = json.loads(line[6:])["choices"][0].get("delta", {})
+                if dl.get("content") or dl.get("reasoning"):
+                    return time.perf_counter() - t0
+    return None
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--base", required=True)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--tag", default="run")
+    ap.add_argument("--out", default=None)
+    a = ap.parse_args()
+    # NO fixed seed: a seeded nonce reproduces the previous run's prompts verbatim,
+    # so --enable-prefix-caching serves them from cache and you measure cache-hit
+    # TTFT, not prefill. (First attempt read 41k tok/s that way.) Lengths stay
+    # identical across builds; only the nonce differs.
+    rnd = random.SystemRandom()
+    res = {"tag": a.tag}
+    body = ""
+    for reps, label in [(4000, "~6.5k"), (16000, "~26k")]:
+        ts = []
+        for _ in range(3):
+            n = rnd.randint(10**12, 10**13)
+            body = f"[doc {n}] The archived engineering report details subsystem telemetry. " * (reps // 12)
+            v = ttft(a.base, a.model, body + "\n\nSummarize in one sentence.")
+            if v:
+                ts.append(v)
+        tok = len(body) // 4
+        med = statistics.median(ts)
+        rate = tok / med
+        res[label] = {"approx_prompt_tokens": tok, "ttft_median_s": med, "prefill_tok_s": rate}
+        print(f"  {label} prompt (~{tok} tok): TTFT {med:.3f}s -> ~{rate:,.0f} tok/s prefill")
+    if a.out:
+        json.dump(res, open(a.out, "w"), indent=2)
+    return 0
+
+if __name__ == "__main__":
+    raise SystemExit(main())

services/gen-seat-mixed-quant/bench/prefill_mixed.json

@@ -0,0 +1,13 @@
+{
+  "tag": "mixed",
+  "~6.5k": {
+    "approx_prompt_tokens": 6743,
+    "ttft_median_s": 1.0645924881100655,
+    "prefill_tok_s": 6333.878996244484
+  },
+  "~26k": {
+    "approx_prompt_tokens": 26993,
+    "ttft_median_s": 5.307923917658627,
+    "prefill_tok_s": 5085.415770598847
+  }
+}

services/gen-seat-mixed-quant/bench/prefill_w4a16.json

@@ -0,0 +1,13 @@
+{
+  "tag": "w4a16",
+  "~6.5k": {
+    "approx_prompt_tokens": 6743,
+    "ttft_median_s": 2.1035482240840793,
+    "prefill_tok_s": 3205.536209152522
+  },
+  "~26k": {
+    "approx_prompt_tokens": 26993,
+    "ttft_median_s": 9.43084010668099,
+    "prefill_tok_s": 2862.205243080904
+  }
+}