feat(esh-ml1): RTX 2000E Ada on esh-pve serves embed + rerank as a LiteLLM failover

esh-pve: NVIDIA 580.178.04 (open modules, DKMS) installed on the host from
NVIDIA's .run and loaded live, no reboot. nvidia-persistenced unit creates the
device nodes before pve-guests; the T400's vfio-pci ids and `blacklist nvidia`
retired. playbooks/esh-pve-nvidia-host.yaml.

esh-ml1: CT 110, unprivileged Debian 12, 10.0.50.80, GPU nodes via devN,
NVIDIA userspace from the same .run (--no-kernel-modules), docker-ce +
nvidia-container-toolkit (no-cgroups). playbooks/esh-ml1-lxc.yaml. Not in the
vzdump job on purpose. DNS esh-ml1.esh.internal.

stacks/embed-rerank: Qwen3-Embedding-0.6B :8001 + bge-reranker-v2-m3 :8013 on
the same vLLM v0.24.0 digest and flags as fv-ml1. Measured parity: embed
cosine FV-vs-ESH median 0.999908 (min 0.999772), inside both self-noise
floors; rerank max |delta| 0.000145 vs floor 0.000181, identical ranking.

litellm: qwen3-embedding and reranker gain an esh-ml1 deployment at order 2
behind fv-ml1 (order 1). Order fallback proven with throwaway groups:
refused primary +0.15 s, host-down primary ~18.7 s per call, dead-only 500.

Also: repaired the DB-only alias reranker-a3-bge-v2-m3, dead since the
fv-ml1 relocation (still named 10.250.50.54); documented the third
unkillable homepage wedge on esh-docker-vm.
This commit is contained in:
vh
2026-09-24 22:38:55 -07:00
parent eb46973051
commit 5402568b76
16 changed files with 1053 additions and 30 deletions
+15
View File
@@ -228,3 +228,18 @@ on fv-ml1; `/model/delete` the two aliases (DB-persisted); weights retained in H
(recreate = brief blip), (c) run a 2nd A3 replica load-balanced behind `reranker` (~2× tput,
identical replicas so no split-measurement issue now the bake-off is closed). Brokkr will
re-run the grid against A4 if it bites — no intuition swaps.
## 2026-09-24 — second site + a dead alias repaired (infra-ops)
- **`reranker-a3-bge-v2-m3` was dead from the fv-ml1 relocation (2026-09-12) until
2026-09-24.** It is a DB-only alias (added via /model/new, so not in `config.yaml`) and
still named ana-ml2's old address `10.250.50.54:8013`; every call returned 500 after ~23 s.
Spend logs show **no callers in the 7-day retention window** besides the probe that found
it. Repointed with `PATCH /model/{id}/update` to `10.251.50.54:8013` (3/3 × 200). Lesson:
a DB-only alias is invisible to a grep of the config, so an IP migration misses it —
sweep `/model/info` too.
- **esh-ml1 added as an order-2 failover behind `reranker`** (and `qwen3-embedding`) — same
model and vLLM version on an RTX 2000E Ada at ESH. This is lever (c) above in failover
form, not load-balancing: fv-ml1 serves every request while it is up. Parity: |score|
FV-vs-ESH max 0.000145 vs FV-vs-FV floor 0.000181, identical ranking over 11 documents.
See `servers/esh-ml1/README.md`.