docs(esh-ml1): speed A/B vs fv-ml1 — single queries a wash, bulk 3-10x slower; stay failover-only

servers/esh-ml1/README.md

@@ -42,6 +42,29 @@ The cross-site difference is inside each site's own run-to-run noise; this
 method cannot resolve a cosine gap below ~2×10⁻⁴. Reranker scores differed by
 at most 0.000145 (FV-vs-FV floor 0.000181), with identical ranking.
 
+**Speed vs fv-ml1, measured 2026-09-25 0628–0640 PT** (same vLLM v0.24.0 and
+flags; fv-ml1 GPU 1 is shared with other seats but read 0% util before and
+after; synthetic text; each cell = median of 6 reps = 2 interleaved runs × 3):
+
+| workload | on-box FV | on-box ESH | from ana-docker FV | from ana-docker ESH |
+|---|---|---|---|---|
+| embed 1 query, p50 | ~12 ms | ~9 ms | ~29 ms | ~23 ms |
+| embed 1 × ~512 tok, p50 | ~15 ms | ~23 ms | ~31 ms | ~31 ms |
+| embed 1 × ~2k tok, p50 | ~25 ms | ~86 ms | ~48 ms | ~90 ms |
+| bulk embed, passages/s (64×512 tok, conc 4) | ~403 | ~50 | ~374 | ~51 |
+| rerank 20 docs, p50 | ~51 ms | ~170 ms | ~82 ms | ~175 ms |
+| rerank 20 docs, req/s at conc 8 | ~55 | ~5.7 | ~52 | ~5.9 |
+
+Read it as: **single queries are a wash** (ESH is ~6 ms faster from the gateway
+because it is closer — 6 ms RTT vs 17.6 ms), **anything bulk is 3–10× slower**
+on the 50 W Ada. Noise floor (run vs run, same site): ~3–5% on throughput, ±3 ms
+on small-query latency on fv-ml1, so the small-query rows cannot rank the sites.
+Positive control: 2k-token inputs were slower than 512-token ones on both
+sites. One fv-ml1 rep of the gateway-vantage rerank burst stalled to 1.0 req/s
+(64 requests in ~64 s); it did not recur in 11 other reps — unexplained, n=1.
+**Consequence: keep esh-ml1 as failover, don't load-share** — sharing would
+roughly double rerank latency for half the fleet's calls.
+
 **VRAM:** 0.20 × 16,380 MiB each; 4,823 MiB in use with both loaded, ~11 GB
 free.