feat(embed-rerank): TEI is the fleet embed/rerank engine; esh-ml1 sole backend; retire fv-ml1 seats

Prime, 2026-09-25: TEI serves embedding + reranking for esh-ml1 and the fleet
from now on; fv-ml1 retires both once esh-ml1 is up.

- stacks/embed-rerank: vLLM -> TEI 1.9.4 (89- Ada build), same ports
  8001/8013, fail-closed truncation (--auto-truncate false; embed
  --max-batch-tokens 32768).
- litellm: qwen3-embedding -> esh-ml1 only (hosted_vllm/, unchanged address);
  reranker -> huggingface/ provider at :8013 (hosted_vllm/ 422s on TEI's
  `texts` body). DB alias reranker-a3-bge-v2-m3 patched to the same target.
- Verified via the gateway against the retiring fv-ml1 seats: embed cosine
  median 0.999927 (n=203); rerank top-1/top-3 29/30.
- stacks/vllm: vllm-embed and vllm-rerank-a3 removed (containers retired on
  fv-ml1, GPU 1 freed ~6.1 GB); reward + coder unchanged.
- Bake-off record moved to docs/pfi/embed-rerank-tei-vs-vllm-bakeoff.md;
  CLAUDE.md gains the TEI convention.
This commit is contained in:
vh
2026-09-25 08:30:53 -07:00
parent 49b4bf0177
commit 7bdac80878
17 changed files with 271 additions and 418 deletions
+7 -7
View File
@@ -117,14 +117,13 @@ no longer deployed sidecars here. See Recent decisions.)
_As of 2026-09-24 ~2245 PT._
### ⭐ esh-ml1: TEI vs vLLM decision pending (Prime), then "what else moves"
### ⭐ TEI is the fleet embed/rerank engine; esh-ml1 is the SOLE backend (2026-09-25)
Prime's direction 2026-09-25: move the utility seats off fv-ml1 to esh-ml1
(accepting slower); **parakeet STAYS on fv-ml1** (real-time chat path). TEI
bake-off done — results `stacks/tei-bakeoff/README.md`; tei-embed :8081 +
tei-rerank :8083 running beside vLLM, NOT gateway-wired. Next: Prime picks the
engine, then the rest of the move (reward model is the open candidate; it needs
vLLM — TEI can't serve a Llama classifier).
Cut-over done 0830 PT: gateway `qwen3-embedding` + `reranker` (+ DB alias a3) →
TEI on esh-ml1; fv-ml1's two vLLM seats retired. ⚠ **Single backend until the
second RTX 2000 arrives**, and **esh-ml1 has no monitoring yet**: wire Beszel +
Kuma next. Next conversation with Prime: "what else moves" (vllm-reward is the
candidate; **parakeet stays**). → Recent decisions.
### ✅ esh-ml1 built (2026-09-24) — two follow-ups
@@ -170,6 +169,7 @@ a shutdown (stays off by design). Outcomes and revert in
## Recent decisions
- `[2026-09-25]` **TEI adopted as the fleet embed/rerank engine (Prime); esh-ml1 is the sole gateway backend for `qwen3-embedding` + `reranker`; fv-ml1's vLLM embed/rerank seats retired (~6.1 GB freed); parakeet stays on fv-ml1.** → `persistent-memory.d/2026-09-25-tei-fleet-embed-rerank.md`
- `[2026-09-24]` **esh-ml1 built — RTX 2000E Ada as CT 110 on esh-pve (host driver 580.178.04 DKMS, loaded live, no reboot), vLLM embed+rerank at measured parity with fv-ml1, wired as LiteLLM `order: 2` failover; dead DB alias `reranker-a3-bge-v2-m3` (pre-relocation IP) repaired.** → `persistent-memory.d/2026-09-24-esh-ml1-embed-rerank.md`
- `[2026-09-24]` **esh-pve: VM 102 retired, a 14-day hung VFIO process found and cleared, T400 → RTX 2000 Ada; Prime chose LXC + host driver for embed/rerank** (implementation deferred to next session, tracked here + `servers/esh-pve/README.md`). → `persistent-memory.d/2026-09-24-esh-pve-vm102-retired-gpu-swap.md`
- `[2026-09-24]` **pfi-gx10 AC-restore UEFI patch applied but NOT validated; box OFF until Prime's AC pull 2026-09-25** (tracking: `servers/pfi-gx10/README.md`, `e5197a3`). → `persistent-memory.d/2026-09-24-gx10-ac-restore-patch.md`