vh
7bdac80878
Prime, 2026-09-25: TEI serves embedding + reranking for esh-ml1 and the fleet from now on; fv-ml1 retires both once esh-ml1 is up. - stacks/embed-rerank: vLLM -> TEI 1.9.4 (89- Ada build), same ports 8001/8013, fail-closed truncation (--auto-truncate false; embed --max-batch-tokens 32768). - litellm: qwen3-embedding -> esh-ml1 only (hosted_vllm/, unchanged address); reranker -> huggingface/ provider at :8013 (hosted_vllm/ 422s on TEI's `texts` body). DB alias reranker-a3-bge-v2-m3 patched to the same target. - Verified via the gateway against the retiring fv-ml1 seats: embed cosine median 0.999927 (n=203); rerank top-1/top-3 29/30. - stacks/vllm: vllm-embed and vllm-rerank-a3 removed (containers retired on fv-ml1, GPU 1 freed ~6.1 GB); reward + coder unchanged. - Bake-off record moved to docs/pfi/embed-rerank-tei-vs-vllm-bakeoff.md; CLAUDE.md gains the TEI convention.
20 lines
600 B
Bash
20 lines
600 B
Bash
# embed-rerank tunables (esh-ml1, TEI). Copy to `.env` on the server.
|
|
|
|
# TEI image tag. `89-` = the Ada Lovelace (sm_89) build; a different GPU
|
|
# generation needs a different prefix (see the TEI README's image table).
|
|
# PINNED to an exact release.
|
|
TEI_TAG=89-1.9.4
|
|
|
|
# Kept from the vLLM era so the embedding gateway entry did not have to change.
|
|
EMBED_PORT=8001
|
|
RERANK_PORT=8013
|
|
|
|
# ⚠ Changing EMBED_MODEL invalidates every index built on it.
|
|
EMBED_MODEL=Qwen/Qwen3-Embedding-0.6B
|
|
RERANK_MODEL=BAAI/bge-reranker-v2-m3
|
|
|
|
MAX_CLIENT_BATCH_SIZE=128
|
|
|
|
# Both models are public; no token needed.
|
|
HF_TOKEN=
|