litellm: canonical defaults for granite + GLM (completes fleet sweep)

- granite-4.1-8b (+ summarizer/classifier): temperature 0 (IBM vendor-canonical
  "temp 0 for inferencing"; top_p/top_k no-ops at temp 0, omitted). Deterministic
  baseline for summ/classify; creative callers override.
- GLM family (z.ai cloud): temperature + top_p 0.95 only (the ONLY params z.ai
  chat API accepts per its OpenAPI schema; top_k/min_p/penalties absent -> not set).
  temp 1.0 for glm-5.1/5.2/5-turbo/4.7 + gen-frontier; temp 0.6 for glm-4.5-air.
  Matches z.ai API defaults -> explicit-over-implicit, future-proofs vs vendor drift.
Round-2 dvalin-researched (provenance-labeled), verified live, granite+glm smoked 200.
Embeddings/rerankers excluded (no sampling). Fleet-wide canonical-defaults sweep complete.
This commit is contained in:
2026-06-27 08:33:43 -07:00
parent 4f094fa653
commit 95d0b38d0a
+21
View File
@@ -27,6 +27,7 @@ model_list:
model: hosted_vllm/granite-4.1-8b
api_base: http://10.250.50.54:8004/v1
api_key: os.environ/VLLM_API_KEY
temperature: 0
model_info:
mode: chat
@@ -38,6 +39,7 @@ model_list:
model: hosted_vllm/granite-4.1-8b
api_base: http://10.250.50.54:8004/v1
api_key: os.environ/VLLM_API_KEY
temperature: 0
model_info:
mode: chat
@@ -48,6 +50,7 @@ model_list:
model: hosted_vllm/granite-4.1-8b
api_base: http://10.250.50.54:8004/v1
api_key: os.environ/VLLM_API_KEY
temperature: 0
model_info:
mode: chat
@@ -272,6 +275,8 @@ model_list:
model: openai/glm-5.1
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: disabled
@@ -281,6 +286,8 @@ model_list:
model: openai/glm-5.1
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: enabled
@@ -293,6 +300,8 @@ model_list:
model: openai/glm-5.2
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: disabled
@@ -301,6 +310,8 @@ model_list:
model: openai/glm-5.2
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: enabled
@@ -315,6 +326,8 @@ model_list:
model: openai/glm-5.2
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: disabled
@@ -323,6 +336,8 @@ model_list:
model: openai/glm-5.2
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: enabled
@@ -331,16 +346,22 @@ model_list:
model: openai/glm-5-turbo
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
- model_name: glm-4.7
litellm_params:
model: openai/glm-4.7
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
- model_name: glm-4.5-air
litellm_params:
model: openai/glm-4.5-air
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 0.6
top_p: 0.95
# --- (removed 2026-06-20, operator call) the `*` wildcard → llama-swap
# (ana-ml2:9292). llama-swap is decommissioned (:9292 confirmed down), so