feat(litellm): add model aliases summarizer / gen / gen-reasoning
Duplicate-entry aliases (NOT router_settings.model_group_alias — that's hidden from /v1/models and can be silently ignored in config per litellm #15020/#5524): - summarizer -> granite-4.1-8b - gen -> mistral-small-4 - gen-reasoning -> mistral-small-4-reasoning (reasoning_effort:high preserved) Each alias is a real model_name co-located with its target (keep api_base in sync). Verified live: all 3 in /v1/models + route end-to-end; gen-reasoning returns reasoning_content.
This commit is contained in:
@@ -30,6 +30,17 @@ model_list:
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# alias: summarizer -> granite-4.1-8b (operator 2026-06-19). Duplicate-entry alias
|
||||
# (not router_settings.model_group_alias — that's hidden from /v1/models and can be
|
||||
# silently ignored in config per litellm #15020/#5524). Keep api_base in sync above.
|
||||
- model_name: summarizer
|
||||
litellm_params:
|
||||
model: hosted_vllm/granite-4.1-8b
|
||||
api_base: http://10.250.50.54:8004/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# --- Qwen3.6-35B-A3B vision-language MoE (official FP8) — vision + chat. vLLM
|
||||
# on ana-ml2 GPU 1, :8007. Explicit entry shadows the "*" wildcard llama-swap
|
||||
# route. REPLACED qwen3.5-9b-fp8 2026-06-14 (the 9B is retired; this is a
|
||||
@@ -95,6 +106,26 @@ model_list:
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# aliases: gen -> mistral-small-4, gen-reasoning -> mistral-small-4-reasoning
|
||||
# (operator 2026-06-19). Duplicate-entry aliases — keep api_base in sync with the
|
||||
# two mistral-small-4 entries above.
|
||||
- model_name: gen
|
||||
litellm_params:
|
||||
model: hosted_vllm/mistral-small-4
|
||||
api_base: http://10.250.50.54:8010/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
model_info:
|
||||
mode: chat
|
||||
- model_name: gen-reasoning
|
||||
litellm_params:
|
||||
model: hosted_vllm/mistral-small-4
|
||||
api_base: http://10.250.50.54:8010/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
extra_body:
|
||||
reasoning_effort: high
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# --- Selene 1 Mini 8B (AtlaAI judge, FP8) — restored on GPU1 after the
|
||||
# llama-swap teardown (was the Q6_K GGUF in the swap zoo). vLLM dynamic fp8,
|
||||
# :8011. Explicit entry shadows the "*" wildcard (which used to reach it via
|
||||
|
||||
Reference in New Issue
Block a user