fix(litellm): remove the * → llama-swap wildcard (decommissioned backend)
llama-swap (ana-ml2:9292) is decommissioned (:9292 confirmed down), so the catch-all wildcard routed every unmatched / typo'd / stale model name to a DEAD backend, surfacing a misleading "Connection error" instead of a clean "model not found". This is the exact footgun that silently swallowed Worldtree's defunct model names (mistral-small-4 etc.) instead of erroring. Removed (operator call) so unknown models now 404 loudly. Verified: gateway healthy post-restart, a bogus model name now returns a clean not-found error, real aliases (gen) still serve. Re-add explicit per-model entries if a swappable zoo ever returns.
This commit is contained in:
@@ -273,22 +273,13 @@ model_list:
|
|||||||
api_base: https://api.z.ai/api/coding/paas/v4
|
api_base: https://api.z.ai/api/coding/paas/v4
|
||||||
api_key: os.environ/Z_AI_API_KEY
|
api_key: os.environ/Z_AI_API_KEY
|
||||||
|
|
||||||
# --- llama-swap passthrough (the swappable generative LLM zoo on
|
# --- (removed 2026-06-20, operator call) the `*` wildcard → llama-swap
|
||||||
# ana-ml2:9292) ---
|
# (ana-ml2:9292). llama-swap is decommissioned (:9292 confirmed down), so
|
||||||
# Wildcard: any model name NOT matched by an exact entry above routes to
|
# the wildcard routed every unmatched / typo'd / stale model name to a DEAD
|
||||||
# llama-swap, which swaps the requested model into GPU on demand. This lets
|
# backend → a misleading "Connection error" instead of a clean "model not
|
||||||
# the gateway front the WHOLE swappable zoo (artemis / selene / qwen3.x / …)
|
# found". This is the footgun that silently swallowed Worldtree's defunct
|
||||||
# for logging + auth WITHOUT registering each model here — keep adding and
|
# model names. Removed so unknown models now fail loudly (404). Re-add an
|
||||||
# swapping models in llama-swap freely; litellm logs them all. litellm does
|
# explicit per-model entry if a swappable zoo ever returns. ---
|
||||||
# no inference; llama-swap still does all the model loading + serving.
|
|
||||||
# Exact matches above (phi4-mini / qwen3-embedding / qwen3-reranker) win;
|
|
||||||
# this only catches everything else. `openai/*` forwards the requested model
|
|
||||||
# name verbatim to llama-swap's OpenAI-compatible endpoint.
|
|
||||||
- model_name: "*"
|
|
||||||
litellm_params:
|
|
||||||
model: openai/*
|
|
||||||
api_base: http://10.250.50.54:9292/v1
|
|
||||||
api_key: "noauth" # llama-swap takes no auth; placeholder bearer
|
|
||||||
|
|
||||||
general_settings:
|
general_settings:
|
||||||
master_key: os.environ/LITELLM_MASTER_KEY
|
master_key: os.environ/LITELLM_MASTER_KEY
|
||||||
|
|||||||
Reference in New Issue
Block a user