fix(litellm): strip empty tools:[] before forwarding to vLLM

vLLM's OpenAI server 400s on an empty tools array ("tools must not be an
empty array"), which broke every gateway call carrying tools:[] (clients
that send it to mean "no tools" -- OpenAI tolerates it, vLLM does not).
drop_params doesn't help: it drops unsupported PARAMS, not empty VALUES.

Add a CustomLogger async_pre_call_hook (conf/strip_empty_tools.py) that
pops an empty/None tools field (+ orphaned tool_choice) before forwarding,
registered globally via litellm_settings.callbacks so it covers every
vLLM-backed model, not just mistral-small-4. Mounted at
/app/strip_empty_tools.py beside config.yaml (LiteLLM resolves callbacks
relative to the config dir). Surgical: only fires when tools is present
and empty; real tools pass through untouched.

Verified on live gateway (1.87.0): mistral-small-4 and granite-4.1-8b
with tools:[] now 200 (were 400); no-tools baseline unchanged; a real
tool still passes through.
This commit is contained in:
vh
2026-06-16 00:43:57 -07:00
parent f9277f5440
commit d1bea13994
4 changed files with 65 additions and 0 deletions
+4
View File
@@ -32,6 +32,10 @@ services:
- "${LITELLM_BIND:-0.0.0.0}:${LITELLM_PORT:-4000}:4000"
volumes:
- /opt/docker/conf/litellm/config.yaml:/app/config.yaml:ro
# Custom pre-call hook (strip empty `tools: []` before forwarding to vLLM).
# Must sit beside config.yaml — LiteLLM loads callbacks relative to the
# config file's directory, so this lands at /app/strip_empty_tools.py.
- /opt/docker/conf/litellm/strip_empty_tools.py:/app/strip_empty_tools.py:ro
environment:
# master_key gates the proxy + admin UI login. Must start with sk-.
- LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}