vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(litellm): add glm-5.3, move gen-frontier pair to glm-5.3 (effort low/high), advertise 1M/128K on GLM 5.2/5.3 with a 128K default max_tokens

Browse Source
This commit is contained in:
vh
2026-10-07 17:30:39 -07:00
parent d99f3a1a8d
commit 8a6d2f9433
2 changed files with 72 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/litellm/conf/config.yaml
+70 -17
View File
@@ -38,8 +38,8 @@ model_list:
# is not corroboration, it is an echo. Flagged by brokkr-smithy-dev
# 2026-08-23 while wiring provenance into a probe harness.
#
# Other current collisions: gen-frontier / gen-frontier-reasoning / glm-5.2 /
# glm-5.2-reasoning -> glm-5.2; ext-tts / gpt-4o-mini-tts / tts-1 / tts-1-hd ->
# Other current collisions: glm-5.2 / glm-5.2-reasoning -> glm-5.2;
# gen-frontier / gen-frontier-reasoning / glm-5.3 -> glm-5.3; ext-tts / gpt-4o-mini-tts / tts-1 / tts-1-hd ->
# the fleet TTS gateway; reranker / reranker-a3-bge-v2-m3 -> bge-reranker-v2-m3.
#
# TO CHECK BEFORE RELYING ON TWO ALIASES BEING DIFFERENT MODELS:
@@ -599,6 +599,11 @@ model_list:
# strips a top-level `thinking` param (drop_params), but forwards `extra_body`
# verbatim to z.ai, where the native thinking:{type:disabled} control lands —
# verified reasoning_tokens→0. Reasoning is opt-in via glm-5.1-reasoning below.
# ⚠ 2026-10-07 re-measure: "disabled" is NOT zero thinking on a hard prompt. Direct to
# z.ai, n=2–3 each: glm-5.1 / glm-5.2 / glm-5.3 with thinking:disabled spent 220–340
# reasoning tokens (enabled: ~1,400–1,600). On an easy prompt it spent 0. So read
# "disabled" as LIGHT thinking, about effort `low`. Unknown whether z.ai changed this or
# the 06-11 check used an easy prompt.
- model_name: glm-5.1
litellm_params:
model: openai/glm-5.1
@@ -626,7 +631,9 @@ model_list:
# glm-5.2-reasoning. extra_body.thinking is forwarded verbatim to z.ai.
# CANONICAL LIMITS (probed live vs z.ai 2026-07-05): 1,048,576-token (1M, 2^20)
# INPUT context; 131,072 (128K) MAX OUTPUT (z.ai max_tokens range [1,131072]).
# NO gateway-side cap — pure z.ai passthrough, so these are the effective limits.
# GATEWAY LIMITS (Prime 2026-10-07, every GLM 5.2/5.3 entry): model_info advertises
# 1,048,576 in / 131,072 out, and max_tokens 131,072 is the DEFAULT when a caller
# sends none (a caller's own max_tokens wins). Reasoning tokens count against it.
- model_name: glm-5.2
litellm_params:
model: openai/glm-5.2
@@ -634,9 +641,14 @@ model_list:
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
max_tokens: 131072
extra_body:
thinking:
type: disabled
model_info:
mode: chat
max_input_tokens: 1048576
max_output_tokens: 131072
- model_name: glm-5.2-reasoning
litellm_params:
model: openai/glm-5.2
@@ -644,35 +656,76 @@ model_list:
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
max_tokens: 131072
extra_body:
thinking:
type: enabled
model_info:
mode: chat
max_input_tokens: 1048576
max_output_tokens: 131072
# glm-5.3 (added 2026-10-07, Prime; z.ai release 2026-08-14, same base as 5.2). Text only;
# 1M in / 128K out per docs.z.ai/guides/llm/glm-5.3. ⚠ REASONING CANNOT BE TURNED OFF on
# 5.3 (z.ai docs; measured 2026-10-07: `thinking: disabled` still spent ~250 reasoning
# tokens on a hard prompt, the same as effort `low`). Efforts: low / high / max
# (`medium` is accepted, not rejected, so reasoning_effort_map.py does not map it).
# This entry leaves the effort to z.ai's default (high-like on our probes) or the caller.
- model_name: glm-5.3
litellm_params:
model: openai/glm-5.3
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
max_tokens: 131072
extra_body:
thinking:
type: enabled
model_info:
mode: chat
max_input_tokens: 1048576
max_output_tokens: 131072
# gen-frontier / gen-frontier-reasoning: capability aliases for the PAID
# frontier tier (GLM 5.2 @ z.ai), mirroring glm-5.2 / glm-5.2-reasoning
# (thinking off / on). Worldtree binds these for frontier-grade generation
# / reasoning; swap the backing frontier model here, callers unaffected.
# frontier tier, GLM 5.3 @ z.ai since 2026-10-07 (Prime; was GLM 5.2). Worldtree
# binds these for frontier-grade generation / reasoning; swap the backing frontier
# model here, callers unaffected. 5.3 cannot disable reasoning, so the old
# "thinking off / on" pair is now effort LOW / HIGH: gen-frontier is the fast
# variant, gen-frontier-reasoning the deliberate one (measured on one hard prompt,
# n=3: low ~250–350 reasoning tokens, high ~1,400–1,500).
# PAID — only all-proxy-models / explicitly-scoped keys reach them; the free
# all-agents-local key is fenced off z.ai spend and cannot.
- model_name: gen-frontier
litellm_params:
model: openai/glm-5.2
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
thinking:
type: disabled
- model_name: gen-frontier-reasoning
litellm_params:
model: openai/glm-5.2
model: openai/glm-5.3
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
max_tokens: 131072
extra_body:
thinking:
type: enabled
reasoning_effort: low
model_info:
mode: chat
max_input_tokens: 1048576
max_output_tokens: 131072
- model_name: gen-frontier-reasoning
litellm_params:
model: openai/glm-5.3
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
temperature: 1.0
top_p: 0.95
max_tokens: 131072
extra_body:
thinking:
type: enabled
reasoning_effort: high
model_info:
mode: chat
max_input_tokens: 1048576
max_output_tokens: 131072
- model_name: glm-5-turbo
litellm_params:
model: openai/glm-5-turbo