fix(nevermore): repoint onto live aliases — its LLM pass had been dead 8 days
nevermore pinned LLAMA_SWAP_MODEL=granite-4.1-8b, an alias retired with the
granite seat on 2026-08-12. Every summarization call since then failed: 67
consecutive status=failure rows, 0 tokens, twice daily, entirely silently. The
briefing had been rendering with no LLM pass at all. Nothing alerts on
status=failure in the spend logs, so it took an unrelated question about
reranker VRAM to surface it.
It was also pinned to NEVERMORE_RERANK_MODEL=qwen3-reranker -- the incumbent
Brokkr R43 measured harming 80/90 fleet queries -- and was its ONLY caller,
while the production `reranker` alias sat at 0 calls for 4 days. The R43
cutover repointed the alias but never moved the consumer.
nevermore/.env LLAMA_SWAP_MODEL granite-4.1-8b -> summarizer
NEVERMORE_RERANK_MODEL qwen3-reranker -> reranker
(server-only; .env is excluded from the mirror both ways)
Verified against nevermore's exact call shape: summarizer returns clean content
with 0 reasoning chars at temperature 0.2 / max_tokens 4000; reranker scores
0.95 on-topic vs ~1e-5 off-topic; embedding returns dim-1024.
Retired alongside it:
vllm-rerank :8002 Qwen3-Reranker-0.6B + the qwen3-reranker alias
vllm-rerank-a4 :8014 gte-reranker-modernbert + its alias
vllm-granite :8004 Exited 8 days, dead service block
and vllm-rerank-a3 was promoted from a throwaway `docker run` into this stack
(the selection ledger's own open follow-up). Healthy in 55s. It keeps the
bake-off arm name so the ledger, memory and R43 record stay valid.
VLLM_VERSION is pinned latest -> v0.24.0. Every service in the stack shares that
one variable, so a bare `compose up -d` could have silently upgraded all of
them at once; both tags resolved to the same local image (4091d5593f77), so the
pin changed nothing at runtime.
GPU1 is down to 81,448 of 97,887 MiB -- 13.9 GB reclaimed tonight.
Correction: an earlier claim that A4 had no gateway alias was wrong. It did.
LiteLLM serves both config-defined and DB-defined models -- live showed 32
against config.yaml's 26 -- and grepping the file cannot see the difference.
/v1/models and /model/info (which flags db_model) are the ground truth. DB
models delete hot via POST /model/delete with no restart.
Left alone: reranker-a3-bge-v2-m3, a zero-call duplicate of `reranker` on the
same backend. It is Brokkr's cutover-verification handle -- redundant rather
than broken, and another agent's tooling is not mine to delete unilaterally.
This commit is contained in:
@@ -8,7 +8,8 @@
|
||||
# Model-name → upstream mapping:
|
||||
# phi4-mini → vLLM :8004 (generative chat)
|
||||
# qwen3-embedding → vLLM :8001 (/v1/embeddings)
|
||||
# qwen3-reranker → vLLM :8002 (/rerank)
|
||||
# reranker → vLLM :8013 (/rerank; bge-v2-m3. The old qwen3-reranker
|
||||
# alias on :8002 was retired 2026-08-20.)
|
||||
# * (wildcard) → llama-swap :9292 (the swappable generative zoo)
|
||||
#
|
||||
# The wildcard fronts llama-swap so its whole model zoo logs through the
|
||||
@@ -313,14 +314,14 @@ model_list:
|
||||
model_info:
|
||||
mode: embedding
|
||||
|
||||
# --- Qwen3 reranker (proxy /rerank route) ---
|
||||
- model_name: qwen3-reranker
|
||||
litellm_params:
|
||||
model: hosted_vllm/Qwen/Qwen3-Reranker-0.6B
|
||||
api_base: http://10.250.50.54:8002/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
model_info:
|
||||
mode: rerank
|
||||
# --- qwen3-reranker RETIRED 2026-08-20. It named Qwen3-Reranker-0.6B on :8002,
|
||||
# the incumbent the R43 bake-off replaced on 2026-08-06 after measuring it
|
||||
# HARMING 80/90 fleet queries (no-reranker beat it 89/90 vs 56/90). The alias
|
||||
# was kept as the rollback path and, for 13 days, was the ONLY reranker
|
||||
# actually receiving traffic: nevermore was pinned to it by name, so the
|
||||
# cutover moved `reranker` but never moved nevermore. Fixed at the consumer
|
||||
# (nevermore now pins `reranker`), then the seat and this alias were retired.
|
||||
# Use `reranker` -> bge-reranker-v2-m3 :8013. ---
|
||||
|
||||
# --- Worldtree capability aliases (role→capability gateway swaps, ADR-0012).
|
||||
# Stable role-named aliases so consumers bind the CAPABILITY, not a concrete
|
||||
@@ -337,7 +338,8 @@ model_list:
|
||||
top_p: 0.9
|
||||
model_info:
|
||||
mode: chat
|
||||
# reranker → generic capability name for rerank (currently qwen3-reranker).
|
||||
# reranker → generic capability name for rerank. THE reranker alias — the only
|
||||
# one left as of 2026-08-20. Every consumer pins this name, never a model name.
|
||||
- model_name: reranker
|
||||
litellm_params:
|
||||
model: hosted_vllm/BAAI/bge-reranker-v2-m3
|
||||
|
||||
Reference in New Issue
Block a user