feat(litellm): add gen-nt / gen-reasoning-nt (noop-tool + tool_choice:none compat variants)

Same Qwopus gen model as gen / gen-reasoning (served-name qwen3.5-122-a10b @
:8013, thinking off/on respectively), but each bakes a dummy 'noop' function tool
+ tool_choice:none into litellm_params so a NON-EMPTY tools array always reaches
vLLM — for consumers where the global strip_empty_tools hook isn't the right fix
(they need a valid tools structure present, not stripped). tool_choice:none means
the noop is never called. api_base = the real LAN endpoint http://10.250.50.54:8013
(the requested http://vllm:8000 template wouldn't resolve from the ana-docker
litellm container). Verified: gen-nt + gen-reasoning-nt both survive a client
tools:[] send; noop never invoked; reasoning split intact.
This commit is contained in:
2026-06-19 11:35:48 -07:00
parent 378261763c
commit 2e5ab72e2c
+49
View File
@@ -134,6 +134,55 @@ model_list:
enable_thinking: true
model_info:
mode: chat
# --- gen-nt / gen-reasoning-nt: same Qwopus `gen` model, but with a dummy `noop`
# tool + tool_choice:"none" baked into the record so a NON-EMPTY tools array
# always reaches vLLM (which 400s on an empty `tools:[]`). For consumers where
# the global strip_empty_tools hook isn't the right fix — i.e. they need a valid
# tools structure PRESENT rather than stripped. tool_choice:none disables actual
# calling, so the noop is never invoked. api_base = the real LAN endpoint (the
# operator's `http://vllm:8000` template wouldn't resolve from ana-docker). ---
- model_name: gen-nt
litellm_params:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
extra_body:
chat_template_kwargs:
enable_thinking: false
tools:
- type: function
function:
name: noop
description: "No-op tool. Do not call this tool; it exists only for compatibility."
parameters:
type: object
properties: {}
required: []
additionalProperties: false
tool_choice: "none"
model_info:
mode: chat
- model_name: gen-reasoning-nt
litellm_params:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
extra_body:
chat_template_kwargs:
enable_thinking: true
tools:
- type: function
function:
name: noop
description: "No-op tool. Do not call this tool; it exists only for compatibility."
parameters:
type: object
properties: {}
required: []
additionalProperties: false
tool_choice: "none"
model_info:
mode: chat
# --- Selene 1 Mini 8B (AtlaAI judge, FP8) — restored on GPU1 after the
# llama-swap teardown (was the Q6_K GGUF in the swap zoo). vLLM dynamic fp8,