feat(litellm): load-share embed/rerank across esh-ml1 + nh3-ml1 with intra-group failover

- qwen3-embedding and reranker: second deployment on nh3-ml1 (config);
  reranker-a3-bge-v2-m3: second DB deployment via /model/new.
- router_settings.enable_weighted_failover: true. Only the three
  multi-deployment groups are affected.
- Measured by stopping nh3-ml1 TEI: without failover 7/40 embeds 500'd;
  with it rerank 80/80, embed 38/40 at onset and 60/60 over a 34 s outage;
  nh3 rejoins rotation after restart. Split 12/28 embed, 22/18 rerank.

servers/esh-ml1/README.md

@@ -5,12 +5,11 @@ GPU LXC for the ESH home lab: **CT 110 on esh-pve**, holding the **NVIDIA RTX
 reranking service** — the only backend behind the gateway's `qwen3-embedding`,
 `reranker` and `reranker-a3-bge-v2-m3` since 2026-09-25. Built 2026-09-24.
 
-⚠ **Still the single gateway backend.** The second RTX 2000 is live as
-[nh3-ml1](../nh3-ml1/README.md) (2026-09-25, parity-verified: the two hosts cannot
-be told apart), but LiteLLM does not route to it yet; that is Prime's call. Until
-it does, if esh-ml1, esh-pve, or ESH's mesh route (esh-scale, CT 108) is down,
-fleet embeddings and reranking are down: Worldtree recall, nevermore clustering,
-Open WebUI RAG.
+**Load-shared with [nh3-ml1](../nh3-ml1/README.md) since 2026-09-26 (Prime).** The
+gateway splits `qwen3-embedding`, `reranker` and `reranker-a3-bge-v2-m3` across both
+TEI hosts, which are parity-verified as indistinguishable. Losing either host no
+longer takes fleet embeddings down; the failover test results are in the nh3-ml1
+README. The reward seat (`/scalar-judge`) is still esh-ml1 only.
 
 | | |
 |---|---|

servers/nh3-ml1/README.md

@@ -6,9 +6,24 @@ reranking backend**, the twin of [esh-ml1](../esh-ml1/README.md): same card, sam
 driver, same TEI image and models. Built 2026-09-25, after the NH3 site visit
 turned Secure Boot off on nh3-pve.
 
-⚠ **Not behind the gateway yet.** It serves on its own ports and is monitored, but
-LiteLLM still routes `qwen3-embedding` and `reranker` to esh-ml1 alone. Routing is
-Prime's call (recommendation: load-share; see below).
+**Load-shared with esh-ml1 since 2026-09-26 (Prime).** The gateway splits
+`qwen3-embedding`, `reranker` and `reranker-a3-bge-v2-m3` across both hosts at
+random, one `simple-shuffle` deployment each. Measured with 40 + 40 requests: embed
+12 esh / 28 nh3, rerank 22 / 18.
+
+**Failover, measured by stopping this host's TEI** (2026-09-26 0747–0753):
+- Without failover settings: **7/40 embeds returned 500**. The router's default 2
+  retries re-picked at random and hit the dead backend again.
+- With `router_settings.enable_weighted_failover: true`, which only affects
+  multi-deployment groups:
+  - rerank **80/80** OK.
+  - embeds **38/40** in the first test (2 failures at outage onset) and **60/60**
+    across a 34 s sustained outage.
+  - Once TEI restarted, nh3 was back in rotation (15/30).
+- Residual: in LiteLLM 1.97 the embedding path does not stamp the failed
+  deployment, so embeds rely on the router's cooldown rather than per-request
+  re-pick. Expect a couple of failed embeds when a backend first dies. n = 2
+  outages.
 
 | | |
 |---|---|