vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(nh3-ml1): second TEI embed/rerank backend live on nh3-pve; parity-verified vs esh-ml1

Browse Source 
NH3 site visit done: Secure Boot off, iGPU restored as boot VGA, AMT port cabled.

- nh3-pve: NVIDIA 580.178.04 (DKMS, open modules) via pve-nvidia-host.yaml.
- nh3-ml1 = CT 109 @ 10.100.50.80 via gpu-lxc.yaml; embed-rerank (TEI 1.9.4)
  deployed with HOST_NAME/HOST_IP labels.
- Parity vs esh-ml1 (1,126 texts, 2 runs/host, controls): embed cosine min
  0.999993 = own noise floor; overlap@10 1.000 vs MRL-256 positive control
  0.684; rerank top-1 1.00, max diff 0.0014 vs floor 0.0020. On-box speed
  identical within rep spread.
- gpu-lxc.yaml: first step upgrades lxc-pve to >= 6.0.0-2 (Proxmox fix #7006).
  With 6.0.0-1 every docker run in a nesting CT failed on runc 1.5's sysctl
  reopen; applied on nh3-pve (one package).
- pve-nvidia-host.yaml: document that the headers meta drags in the newest
  kernel (nh3-pve went 6.8.12-11 -> -43 at the next reboot).
- Monitoring: Beszel NVIDIA agent + 5 alerts, Kuma #29/#30, Homepage
  nh3-ml1-docker, Dozzle agent (hub 8 clients). DNS nh3-ml1.nh3.internal.
- nh3-pve README: SB/IGFX/driver/kernel state, btmtk oops on -4x kernels,
  AMT cabled but unreachable on the network.

Gateway routing to nh3-ml1 is not changed.
This commit is contained in:
vh
2026-09-25 15:55:33 -07:00
parent 6fa8213c20
commit 5960526c3f
17 changed files with 815 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dns/internal.yaml
+2 -1
View File
@@ -99,7 +99,7 @@ hosts:
- {name: esh-pve, site: esh, v4: 10.0.250.35, note: Proxmox hypervisor}
- {name: esh-pve-nas, site: esh, v4: 10.0.50.55, note: Proxmox hypervisor — storage/media}
- {name: esh-vm-db, site: esh, v4: 10.0.50.60, note: PostgreSQL + MongoDB}
- {name: esh-ml1, site: esh, v4: 10.0.50.80, note: CT110 on esh-pve — RTX 2000E Ada, embedding + reranking (vLLM)}
- {name: esh-ml1, site: esh, v4: 10.0.50.80, note: CT110 on esh-pve — RTX 2000E Ada, embedding + reranking (TEI) + reward seat}
- {name: vm-esh-nas, site: esh, v4: 10.0.50.154, note: NAS-adjacent docker host}
- {name: esh-filebot, site: esh, v4: 10.0.50.70, note: restic / file-sync VM}
- {name: esh-gw, site: esh, v4: 10.0.250.1, note: esh-gw}
@@ -111,6 +111,7 @@ hosts:
- {name: ana-scale, site: ana, v4: 10.250.50.45, note: CT114 on pfi-pve — headscale/tailscale subnet router for 10.250.0.0/16}
- {name: nh3-headscale, site: nh3, v4: 10.100.50.45, note: CT106 on nh3-pve — headscale control plane (public headscale.phasefinal.com:443)}
- {name: nh3-scale, site: nh3, v4: 10.100.50.46, note: CT107 on nh3-pve — subnet router for 10.100.0.0/16}
- {name: nh3-ml1, site: nh3, v4: 10.100.50.80, note: CT109 on nh3-pve — RTX 2000E Ada, second embedding + reranking backend (TEI)}
- {name: esh-scale, site: esh, v4: 10.0.50.65, note: CT108 on esh-pve — subnet router for 10.0.0.0/16}
# Service aliases — a name that points at whatever host currently runs it, so