vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(embed-rerank): TEI is the fleet embed/rerank engine; esh-ml1 sole backend; retire fv-ml1 seats

Browse Source 
Prime, 2026-09-25: TEI serves embedding + reranking for esh-ml1 and the fleet
from now on; fv-ml1 retires both once esh-ml1 is up.

- stacks/embed-rerank: vLLM -> TEI 1.9.4 (89- Ada build), same ports
  8001/8013, fail-closed truncation (--auto-truncate false; embed
  --max-batch-tokens 32768).
- litellm: qwen3-embedding -> esh-ml1 only (hosted_vllm/, unchanged address);
  reranker -> huggingface/ provider at :8013 (hosted_vllm/ 422s on TEI's
  `texts` body). DB alias reranker-a3-bge-v2-m3 patched to the same target.
- Verified via the gateway against the retiring fv-ml1 seats: embed cosine
  median 0.999927 (n=203); rerank top-1/top-3 29/30.
- stacks/vllm: vllm-embed and vllm-rerank-a3 removed (containers retired on
  fv-ml1, GPU 1 freed ~6.1 GB); reward + coder unchanged.
- Bake-off record moved to docs/pfi/embed-rerank-tei-vs-vllm-bakeoff.md;
  CLAUDE.md gains the TEI convention.
This commit is contained in:
vh
2026-09-25 08:30:53 -07:00
parent 49b4bf0177
commit 7bdac80878
17 changed files with 271 additions and 418 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+5 -1
View File
@@ -331,6 +331,10 @@ Observed and standardized across servers:
the new content serves immediately — it is the route table that is frozen at
start, not the file contents. Reference it as `icon: /images/<file>`.
- **Healthchecks** on services that expose HTTP
- **Embedding + reranking models are served with Hugging Face TEI, not vLLM** (Prime,
2026-09-25). The fleet service is `stacks/embed-rerank` on esh-ml1. Why, plus the
parity bar any engine or model change must clear:
`docs/pfi/embed-rerank-tei-vs-vllm-bakeoff.md`.
## Servers
@@ -362,7 +366,7 @@ Observed and standardized across servers:
| esh-pve | 10.0.250.35 | ESH home lab | Proxmox VE hypervisor | `servers/esh-pve/README.md` |
| esh-pve-nas | 10.0.50.55 | ESH home lab | Proxmox VE hypervisor (storage / media) | `servers/esh-pve-nas/README.md` |
| esh-vm-db | 10.0.50.60 | ESH home lab | DB VM — PostgreSQL (paperless-ng) + MongoDB; bare-metal VM, no Docker | `servers/esh-vm-db/README.md` |
| esh-ml1 | 10.0.50.80 | ESH home lab | GPU LXC (CT 110 on esh-pve, RTX 2000E Ada 16 GB, host-side NVIDIA driver) — embedding + reranking, LiteLLM order-2 failover behind fv-ml1 | `servers/esh-ml1/README.md` |
| esh-ml1 | 10.0.50.80 | ESH home lab | GPU LXC (CT 110 on esh-pve, RTX 2000E Ada 16 GB, host-side NVIDIA driver) — **the fleet's embedding + reranking service** (TEI), sole gateway backend since 2026-09-25 | `servers/esh-ml1/README.md` |
**Placement rules:**
- GPU-required stacks → `fv-ml1` (primary, Fountain Valley) or `irv-ml1`