The gen-reasoning seat accepts only xhigh/medium/low and 400s on anything
else — including `high`, which is the default of several clients, so the
seat presented as broken rather than as one enum value out of step. The
DeepSeek Harness failed every request on its default setting, and the only
working client value was `low`: the seat's WEAKEST reasoning tier, while
its own default is xhigh.
conf/reasoning_effort_map.py is a pre-call hook in the same shape as the
existing strip_empty_tools hook. It is scoped to one model group, measured
rather than assumed: gen-reasoning rejects `high`; gen, sec,
char-rp-reasoning and summarizer all accept it and are left alone. Paid
passthroughs were not probed, because probing them spends vendor credits,
and are not mapped.
Verified after deploy: high and max now succeed on gen-reasoning, low and
xhigh still work, a request with no effort param still works, `gen` with
`high` still passes through unmapped, and the harness completes a real
file-edit task at full reasoning.
Needed a compose change as well as a conf push — callbacks are bind-mounted
per file, so the volume only attaches on container create. Recreated the
litellm service by name so the DB was not bounced with it.
vLLM's OpenAI server 400s on an empty tools array ("tools must not be an
empty array"), which broke every gateway call carrying tools:[] (clients
that send it to mean "no tools" -- OpenAI tolerates it, vLLM does not).
drop_params doesn't help: it drops unsupported PARAMS, not empty VALUES.
Add a CustomLogger async_pre_call_hook (conf/strip_empty_tools.py) that
pops an empty/None tools field (+ orphaned tool_choice) before forwarding,
registered globally via litellm_settings.callbacks so it covers every
vLLM-backed model, not just mistral-small-4. Mounted at
/app/strip_empty_tools.py beside config.yaml (LiteLLM resolves callbacks
relative to the config dir). Surgical: only fires when tools is present
and empty; real tools pass through untouched.
Verified on live gateway (1.87.0): mistral-small-4 and granite-4.1-8b
with tools:[] now 200 (were 400); no-tools baseline unchanged; a real
tool still passes through.
LiteLLM proxy fronting the vLLM services on ana-ml2 so every request +
response is captured and inspectable in a browser Logs UI — the
visibility vLLM itself lacks (Dozzle shows only connection metadata).
- compose: litellm (proxy + /ui Logs) + litellm-db (Postgres store)
- conf/config.yaml: routes phi4-mini (chat, :8004), qwen3-embedding
(:8001), qwen3-reranker (:8002); store_prompts_in_spend_logs persists
full prompt/completion text. reward classifier (:8003) stays direct
(no first-class LiteLLM route).
- Langfuse-ready: lean first cut intentionally skips Langfuse's heavy v3
stack; graduating is one env-var + callback step, no re-architecture.
- roadmap: mark the vLLM-observability item's first cut as shipped.
Lean first cut of docs/roadmap.md "Observability for the vLLM stack".
vh