feat(embed-rerank): TEI is the fleet embed/rerank engine; esh-ml1 sole backend; retire fv-ml1 seats
Prime, 2026-09-25: TEI serves embedding + reranking for esh-ml1 and the fleet from now on; fv-ml1 retires both once esh-ml1 is up. - stacks/embed-rerank: vLLM -> TEI 1.9.4 (89- Ada build), same ports 8001/8013, fail-closed truncation (--auto-truncate false; embed --max-batch-tokens 32768). - litellm: qwen3-embedding -> esh-ml1 only (hosted_vllm/, unchanged address); reranker -> huggingface/ provider at :8013 (hosted_vllm/ 422s on TEI's `texts` body). DB alias reranker-a3-bge-v2-m3 patched to the same target. - Verified via the gateway against the retiring fv-ml1 seats: embed cosine median 0.999927 (n=203); rerank top-1/top-3 29/30. - stacks/vllm: vllm-embed and vllm-rerank-a3 removed (containers retired on fv-ml1, GPU 1 freed ~6.1 GB); reward + coder unchanged. - Bake-off record moved to docs/pfi/embed-rerank-tei-vs-vllm-bakeoff.md; CLAUDE.md gains the TEI convention.
This commit is contained in:
@@ -124,8 +124,6 @@ context.
|
||||
| `gen-small-reasoning` | 8026 |
|
||||
| `image-judge` | 8022 |
|
||||
| `qwen-image-bench` | 8022 |
|
||||
| `qwen3-embedding` | 8001 |
|
||||
| `reranker` | 8013 |
|
||||
| `sec` | 8025 |
|
||||
| `sec-reasoning` | 8025 |
|
||||
| `summarizer` | 8026 |
|
||||
@@ -138,3 +136,7 @@ context.
|
||||
*Regenerate with `scripts/seat-inventory.py` after ANY seat change —
|
||||
model swap, quant change, context or utilization edit, or speculative-decoding
|
||||
change. Run `--check` in CI to catch a stale document.*
|
||||
|
||||
> **2026-09-25:** `qwen3-embedding` (:8001) and `reranker` (:8013) LEFT fv-ml1 —
|
||||
> their vLLM seats were retired and the names now route to TEI on esh-ml1
|
||||
> (`stacks/embed-rerank`). ~6.1 GB freed on GPU 1.
|
||||
|
||||
Reference in New Issue
Block a user