fix(litellm): remove the * → llama-swap wildcard (decommissioned backend)

llama-swap (ana-ml2:9292) is decommissioned (:9292 confirmed down), so the
catch-all wildcard routed every unmatched / typo'd / stale model name to a DEAD
backend, surfacing a misleading "Connection error" instead of a clean
"model not found". This is the exact footgun that silently swallowed Worldtree's
defunct model names (mistral-small-4 etc.) instead of erroring. Removed (operator
call) so unknown models now 404 loudly. Verified: gateway healthy post-restart,
a bogus model name now returns a clean not-found error, real aliases (gen) still
serve. Re-add explicit per-model entries if a swappable zoo ever returns.
This commit is contained in:
2026-06-20 10:08:20 -07:00
parent d3721034c1
commit d0eb09cac1
+7 -16
View File
@@ -273,22 +273,13 @@ model_list:
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
# --- llama-swap passthrough (the swappable generative LLM zoo on
# ana-ml2:9292) ---
# Wildcard: any model name NOT matched by an exact entry above routes to
# llama-swap, which swaps the requested model into GPU on demand. This lets
# the gateway front the WHOLE swappable zoo (artemis / selene / qwen3.x / …)
# for logging + auth WITHOUT registering each model here — keep adding and
# swapping models in llama-swap freely; litellm logs them all. litellm does
# no inference; llama-swap still does all the model loading + serving.
# Exact matches above (phi4-mini / qwen3-embedding / qwen3-reranker) win;
# this only catches everything else. `openai/*` forwards the requested model
# name verbatim to llama-swap's OpenAI-compatible endpoint.
- model_name: "*"
litellm_params:
model: openai/*
api_base: http://10.250.50.54:9292/v1
api_key: "noauth" # llama-swap takes no auth; placeholder bearer
# --- (removed 2026-06-20, operator call) the `*` wildcard → llama-swap
# (ana-ml2:9292). llama-swap is decommissioned (:9292 confirmed down), so
# the wildcard routed every unmatched / typo'd / stale model name to a DEAD
# backend → a misleading "Connection error" instead of a clean "model not
# found". This is the footgun that silently swallowed Worldtree's defunct
# model names. Removed so unknown models now fail loudly (404). Re-add an
# explicit per-model entry if a swappable zoo ever returns. ---
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY