feat(litellm): map reasoning_effort high/max -> xhigh for gen-reasoning

The gen-reasoning seat accepts only xhigh/medium/low and 400s on anything
else — including `high`, which is the default of several clients, so the
seat presented as broken rather than as one enum value out of step. The
DeepSeek Harness failed every request on its default setting, and the only
working client value was `low`: the seat's WEAKEST reasoning tier, while
its own default is xhigh.

conf/reasoning_effort_map.py is a pre-call hook in the same shape as the
existing strip_empty_tools hook. It is scoped to one model group, measured
rather than assumed: gen-reasoning rejects `high`; gen, sec,
char-rp-reasoning and summarizer all accept it and are left alone. Paid
passthroughs were not probed, because probing them spends vendor credits,
and are not mapped.

Verified after deploy: high and max now succeed on gen-reasoning, low and
xhigh still work, a request with no effort param still works, `gen` with
`high` still passes through unmapped, and the harness completes a real
file-edit task at full reasoning.

Needed a compose change as well as a conf push — callbacks are bind-mounted
per file, so the volume only attaches on container create. Recreated the
litellm service by name so the DB was not bounced with it.
This commit is contained in:
vh
2026-09-02 15:43:22 -07:00
parent e9605df6ff
commit 926fc2fb7a
4 changed files with 104 additions and 1 deletions
+7 -1
View File
@@ -894,7 +894,13 @@ litellm_settings:
# unsupported params. Runs on every request → fixes it for all vLLM models.
# File mounted at /app/strip_empty_tools.py; reference is module.instance,
# resolved relative to this config's directory.
callbacks: ["strip_empty_tools.strip_empty_tools_instance"]
# Second pre-call hook: translate `reasoning_effort` values a backend does not
# accept (gen-reasoning takes only xhigh/medium/low and 400s on `high`, which
# is the DEFAULT of several clients). Scoped per model group inside the file;
# measured, not assumed. Added 2026-09-02.
callbacks:
- "strip_empty_tools.strip_empty_tools_instance"
- "reasoning_effort_map.reasoning_effort_map_instance"
# Langfuse trace export RETIRED 2026-06-20 (operator). Its ClickHouse member spewed
# ~94 GB of unrotated logs and filled ana-docker's disk; the trace UI was redundant
# with LiteLLM's native spend_logs (store_prompts_in_spend_logs: true → full