feat(esh-ml1): RTX 2000E Ada on esh-pve serves embed + rerank as a LiteLLM failover
esh-pve: NVIDIA 580.178.04 (open modules, DKMS) installed on the host from NVIDIA's .run and loaded live, no reboot. nvidia-persistenced unit creates the device nodes before pve-guests; the T400's vfio-pci ids and `blacklist nvidia` retired. playbooks/esh-pve-nvidia-host.yaml. esh-ml1: CT 110, unprivileged Debian 12, 10.0.50.80, GPU nodes via devN, NVIDIA userspace from the same .run (--no-kernel-modules), docker-ce + nvidia-container-toolkit (no-cgroups). playbooks/esh-ml1-lxc.yaml. Not in the vzdump job on purpose. DNS esh-ml1.esh.internal. stacks/embed-rerank: Qwen3-Embedding-0.6B :8001 + bge-reranker-v2-m3 :8013 on the same vLLM v0.24.0 digest and flags as fv-ml1. Measured parity: embed cosine FV-vs-ESH median 0.999908 (min 0.999772), inside both self-noise floors; rerank max |delta| 0.000145 vs floor 0.000181, identical ranking. litellm: qwen3-embedding and reranker gain an esh-ml1 deployment at order 2 behind fv-ml1 (order 1). Order fallback proven with throwaway groups: refused primary +0.15 s, host-down primary ~18.7 s per call, dead-only 500. Also: repaired the DB-only alias reranker-a3-bge-v2-m3, dead since the fv-ml1 relocation (still named 10.250.50.54); documented the third unkillable homepage wedge on esh-docker-vm.
This commit is contained in:
@@ -228,3 +228,18 @@ on fv-ml1; `/model/delete` the two aliases (DB-persisted); weights retained in H
|
||||
(recreate = brief blip), (c) run a 2nd A3 replica load-balanced behind `reranker` (~2× tput,
|
||||
identical replicas so no split-measurement issue now the bake-off is closed). Brokkr will
|
||||
re-run the grid against A4 if it bites — no intuition swaps.
|
||||
|
||||
## 2026-09-24 — second site + a dead alias repaired (infra-ops)
|
||||
|
||||
- **`reranker-a3-bge-v2-m3` was dead from the fv-ml1 relocation (2026-09-12) until
|
||||
2026-09-24.** It is a DB-only alias (added via /model/new, so not in `config.yaml`) and
|
||||
still named ana-ml2's old address `10.250.50.54:8013`; every call returned 500 after ~23 s.
|
||||
Spend logs show **no callers in the 7-day retention window** besides the probe that found
|
||||
it. Repointed with `PATCH /model/{id}/update` to `10.251.50.54:8013` (3/3 × 200). Lesson:
|
||||
a DB-only alias is invisible to a grep of the config, so an IP migration misses it —
|
||||
sweep `/model/info` too.
|
||||
- **esh-ml1 added as an order-2 failover behind `reranker`** (and `qwen3-embedding`) — same
|
||||
model and vLLM version on an RTX 2000E Ada at ESH. This is lever (c) above in failover
|
||||
form, not load-balancing: fv-ml1 serves every request while it is up. Parity: |score|
|
||||
FV-vs-ESH max 0.000145 vs FV-vs-FV floor 0.000181, identical ranking over 11 documents.
|
||||
See `servers/esh-ml1/README.md`.
|
||||
|
||||
Reference in New Issue
Block a user