vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(esh-ml1): RTX 2000E Ada on esh-pve serves embed + rerank as a LiteLLM failover

Browse Source 
esh-pve: NVIDIA 580.178.04 (open modules, DKMS) installed on the host from
NVIDIA's .run and loaded live, no reboot. nvidia-persistenced unit creates the
device nodes before pve-guests; the T400's vfio-pci ids and `blacklist nvidia`
retired. playbooks/esh-pve-nvidia-host.yaml.

esh-ml1: CT 110, unprivileged Debian 12, 10.0.50.80, GPU nodes via devN,
NVIDIA userspace from the same .run (--no-kernel-modules), docker-ce +
nvidia-container-toolkit (no-cgroups). playbooks/esh-ml1-lxc.yaml. Not in the
vzdump job on purpose. DNS esh-ml1.esh.internal.

stacks/embed-rerank: Qwen3-Embedding-0.6B :8001 + bge-reranker-v2-m3 :8013 on
the same vLLM v0.24.0 digest and flags as fv-ml1. Measured parity: embed
cosine FV-vs-ESH median 0.999908 (min 0.999772), inside both self-noise
floors; rerank max |delta| 0.000145 vs floor 0.000181, identical ranking.

litellm: qwen3-embedding and reranker gain an esh-ml1 deployment at order 2
behind fv-ml1 (order 1). Order fallback proven with throwaway groups:
refused primary +0.15 s, host-down primary ~18.7 s per call, dead-only 500.

Also: repaired the DB-only alias reranker-a3-bge-v2-m3, dead since the
fv-ml1 relocation (still named 10.250.50.54); documented the third
unkillable homepage wedge on esh-docker-vm.
This commit is contained in:
vh
2026-09-24 22:38:55 -07:00
parent eb46973051
commit 5402568b76
16 changed files with 1053 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dns/internal.yaml
+1
View File
@@ -99,6 +99,7 @@ hosts:
- {name: esh-pve, site: esh, v4: 10.0.250.35, note: Proxmox hypervisor}
- {name: esh-pve-nas, site: esh, v4: 10.0.50.55, note: Proxmox hypervisor — storage/media}
- {name: esh-vm-db, site: esh, v4: 10.0.50.60, note: PostgreSQL + MongoDB}
- {name: esh-ml1, site: esh, v4: 10.0.50.80, note: CT110 on esh-pve — RTX 2000E Ada, embedding + reranking (vLLM)}
- {name: vm-esh-nas, site: esh, v4: 10.0.50.154, note: NAS-adjacent docker host}
- {name: esh-filebot, site: esh, v4: 10.0.50.70, note: restic / file-sync VM}
- {name: esh-gw, site: esh, v4: 10.0.250.1, note: esh-gw}