vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(esh-ml1): speed A/B vs fv-ml1 — single queries a wash, bulk 3-10x slower; stay failover-only

Browse Source
This commit is contained in:
vh
2026-09-25 06:40:12 -07:00
parent e128f5da63
commit 3e73374c6e
2 changed files with 30 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.d/2026-09-24-esh-ml1-embed-rerank.md
+7
View File
@@ -60,3 +60,10 @@ an FV-outage failover for Worldtree + nevermore, not local service for ESH.
ESH consumers, which are idle — recommended parked. esh-ml1 not yet in
Homepage docker.yaml or Beszel. The ~18.7 s host-down failover penalty is
untuned (router retries/cooldown).
**Speed A/B vs fv-ml1 (2026-09-25 0628–0640 PT, table in `servers/esh-ml1/README.md`).**
Single queries: a wash (on-box ESH ~9 ms vs FV ~12 ms; from ana-docker ESH is
faster because it is network-closer). Bulk: ESH 3–10× slower (bulk embed ~50 vs
~403 passages/s; rerank-20 ~170 vs ~51 ms p50, ~5.7 vs ~55 req/s at conc 8).
Noise floor 3–5% on throughput. ⇒ failover role confirmed; load-sharing would
roughly double rerank latency for half the calls.