feat(litellm): add classifier -> granite + summarizer-large -> gen aliases (operator)
Duplicate-entry aliases. classifier -> granite-4.1-8b (:8004, same backend as the existing summarizer alias). summarizer-large -> gen/qwen3.5-122-a10b (:8013, thinking off) for heavier summarization on the 122B Qwopus. summarizer -> granite already existed (no-op). Config-staged + deployed without bouncing the gateway; like any config-add these activate on the next restart (no live-add performed).
This commit is contained in:
@@ -41,6 +41,30 @@ model_list:
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# alias: classifier -> granite-4.1-8b (operator 2026-06-19). Light/fast classification
|
||||
# + triage endpoint; same backend as summarizer. Keep api_base in sync above.
|
||||
- model_name: classifier
|
||||
litellm_params:
|
||||
model: hosted_vllm/granite-4.1-8b
|
||||
api_base: http://10.250.50.54:8004/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# alias: summarizer-large -> gen / qwen3.5-122-a10b (operator 2026-06-19). For heavier
|
||||
# summarization that wants the 122B Qwopus instead of granite-8b. Thinking OFF (matches
|
||||
# gen). Keep api_base (:8013) + enable_thinking in sync with the gen record below.
|
||||
- model_name: summarizer-large
|
||||
litellm_params:
|
||||
model: hosted_vllm/qwen3.5-122-a10b
|
||||
api_base: http://10.250.50.54:8013/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
# --- Qwen-Image-Bench (NVFP4) — text-to-image quality JUDGE, vision-intact. vLLM
|
||||
# on ana-ml2 GPU 1, :8014. REPLACED qwen3.6-35b-a3b (operator 2026-06-19; the
|
||||
# qwen36 VL judge was downed + its FP8 weights freed ~33GB). Multimodal — image
|
||||
|
||||
Reference in New Issue
Block a user