fix(litellm): strip empty tools:[] before forwarding to vLLM
vLLM's OpenAI server 400s on an empty tools array ("tools must not be an
empty array"), which broke every gateway call carrying tools:[] (clients
that send it to mean "no tools" -- OpenAI tolerates it, vLLM does not).
drop_params doesn't help: it drops unsupported PARAMS, not empty VALUES.
Add a CustomLogger async_pre_call_hook (conf/strip_empty_tools.py) that
pops an empty/None tools field (+ orphaned tool_choice) before forwarding,
registered globally via litellm_settings.callbacks so it covers every
vLLM-backed model, not just mistral-small-4. Mounted at
/app/strip_empty_tools.py beside config.yaml (LiteLLM resolves callbacks
relative to the config dir). Surgical: only fires when tools is present
and empty; real tools pass through untouched.
Verified on live gateway (1.87.0): mistral-small-4 and granite-4.1-8b
with tools:[] now 200 (were 400); no-tools baseline unchanged; a real
tool still passes through.
This commit is contained in:
@@ -32,6 +32,10 @@ services:
|
||||
- "${LITELLM_BIND:-0.0.0.0}:${LITELLM_PORT:-4000}:4000"
|
||||
volumes:
|
||||
- /opt/docker/conf/litellm/config.yaml:/app/config.yaml:ro
|
||||
# Custom pre-call hook (strip empty `tools: []` before forwarding to vLLM).
|
||||
# Must sit beside config.yaml — LiteLLM loads callbacks relative to the
|
||||
# config file's directory, so this lands at /app/strip_empty_tools.py.
|
||||
- /opt/docker/conf/litellm/strip_empty_tools.py:/app/strip_empty_tools.py:ro
|
||||
environment:
|
||||
# master_key gates the proxy + admin UI login. Must start with sk-.
|
||||
- LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
|
||||
|
||||
Reference in New Issue
Block a user