remove gen-nt / gen-reasoning-nt litellm records (operator)
Source + deployed config cleaned without bouncing the gateway. NOTE: these were config-loaded models, which the /model/delete API can't remove (DB-only -> 'not found in db'), so the LIVE gateway still serves them until its next restart, at which point the cleaned config drops them. No bounce performed.
This commit is contained in:
@@ -134,55 +134,6 @@ model_list:
|
||||
enable_thinking: true
|
||||
model_info:
|
||||
mode: chat
|
||||
# --- gen-nt / gen-reasoning-nt: same Qwopus `gen` model, but with a dummy `noop`
|
||||
# tool + tool_choice:"none" baked into the record so a NON-EMPTY tools array
|
||||
# always reaches vLLM (which 400s on an empty `tools:[]`). For consumers where
|
||||
# the global strip_empty_tools hook isn't the right fix — i.e. they need a valid
|
||||
# tools structure PRESENT rather than stripped. tool_choice:none disables actual
|
||||
# calling, so the noop is never invoked. api_base = the real LAN endpoint (the
|
||||
# operator's `http://vllm:8000` template wouldn't resolve from ana-docker). ---
|
||||
- model_name: gen-nt
|
||||
litellm_params:
|
||||
model: hosted_vllm/qwen3.5-122-a10b
|
||||
api_base: http://10.250.50.54:8013/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
tools:
|
||||
- type: function
|
||||
function:
|
||||
name: noop
|
||||
description: "No-op tool. Do not call this tool; it exists only for compatibility."
|
||||
parameters:
|
||||
type: object
|
||||
properties: {}
|
||||
required: []
|
||||
additionalProperties: false
|
||||
tool_choice: "none"
|
||||
model_info:
|
||||
mode: chat
|
||||
- model_name: gen-reasoning-nt
|
||||
litellm_params:
|
||||
model: hosted_vllm/qwen3.5-122-a10b
|
||||
api_base: http://10.250.50.54:8013/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: true
|
||||
tools:
|
||||
- type: function
|
||||
function:
|
||||
name: noop
|
||||
description: "No-op tool. Do not call this tool; it exists only for compatibility."
|
||||
parameters:
|
||||
type: object
|
||||
properties: {}
|
||||
required: []
|
||||
additionalProperties: false
|
||||
tool_choice: "none"
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# --- Selene 1 Mini 8B (AtlaAI judge, FP8) — restored on GPU1 after the
|
||||
# llama-swap teardown (was the Q6_K GGUF in the swap zoo). vLLM dynamic fp8,
|
||||
|
||||
Reference in New Issue
Block a user