feat(litellm): add glm-5.3, move gen-frontier pair to glm-5.3 (effort low/high), advertise 1M/128K on GLM 5.2/5.3 with a 128K default max_tokens
This commit is contained in:
@@ -186,6 +186,8 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache-
|
||||
|
||||
- `[2026-10-06]` **Scriberr diarization fixed: gated pyannote, a torchcodec/torch mismatch, and WhisperX's CPU defaults; model caches on /tank, HF_TOKEN a placeholder (Prime), GPU+speaker profiles.** Patch 0003 waits for the next rebuild (tracked at `stacks/scriberr/patches/README.md`). → `persistent-memory.d/2026-10-06-scriberr-diarization-fixed.md`
|
||||
- `[2026-10-06]` **Backup checker: destroyed guests reported as residue, an ssh stdin swallow fixed, snapshot age from the name; residue pruned (Prime); pbs-nh3 mirror keeps pbs-ana's history (Prime "match history"), first prune 10-07 00:00.** → `persistent-memory.d/2026-10-06-backup-checker-fixes-and-mirror-retention.md`
|
||||
- `[2026-10-07]` **Gateway: glm-5.3 added; gen-frontier / gen-frontier-reasoning moved 5.2 → 5.3 (Prime); every GLM 5.2/5.3 entry advertises 1,048,576 in / 131,072 out with max_tokens 131,072 as the default.** 5.3 cannot turn reasoning off, so the gen-frontier pair is now effort low / high (measured ~260 vs ~1,490 reasoning tokens). ⚠ z.ai's `thinking: disabled` is light thinking, not zero, on 5.1/5.2/5.3 alike (220–340 reasoning tokens on a hard prompt). Restarted 1725 (61 s), held 2 h for albok-dev's load A/B. The gate script's first two runs could never pass: `grep -c` exits 1 on a zero count. Images: glm-5.x is text only (z.ai rejects image parts); glm-4.6v and glm-5.3-flash read images on our plan, but neither is on the gateway. → `stacks/litellm/conf/config.yaml`
|
||||
- `[2026-10-07]` **Hermes launchers repaired (Miranda relay 8462):** svos-dev's scratch probe had rewritten `~/.hermes/hermes-agent/.hermes/bin/{hermes,hermes-acp}` to exec a /tmp scratch Python. Repointed to `~/.hermes/tools/python-3.14.7…`; the gateway was not restarted. ⚠ Don't run `git` in `~/.hermes/hermes-agent`: a `status` there looped on auto-gc.
|
||||
- `[2026-10-07]` **albok-service 0.1.13 (unit 4.2, the sensitivity gate) deployed on nh3-albok at 1135 at albok-dev's ask 8413.** Rollback is NOT free: the first-start scan writes `sensitivity` sidecars, which 0.1.12 cannot load, so rolling back needs the strip snippet first. The census found `leaks` 8: six hold the shared all-agents key, two are false positives on a doc placeholder, nothing to rotate. → `stacks/albok-service/README.md`
|
||||
- `[2026-10-06]` **albok-service 0.1.11 (0202) then 0.1.12 (1254) deployed on nh3-albok at albok-dev's asks 7991/8199**, built from clean archives, pinned by digest, single-file upload (deploy-stack's `--delete` would wipe the on-host rollback copies). Every wing is now migrated to generation collections, so a 0.1.10 rollback needs a full reindex. albok's 2.01×-vs-2× load check awaits Prime. → `stacks/albok-service/README.md`
|
||||
- `[2026-10-06]` **The LiteLLM gateway keeps no request/response content, and `store_prompts_in_spend_logs: false` is now a marked consumer guarantee (comfy-dev, arbo incognito).** Proved with a canary plus positive controls; per-request `no-log` does NOT cover the proxy's own spend-log callback (v1.97 source). irv-ml1 restic also excludes `input/incognito-*` (dry-run proved with a canary and a control). `847424c`, `0f6c7a0`.
|
||||
|
||||
@@ -38,8 +38,8 @@ model_list:
|
||||
# is not corroboration, it is an echo. Flagged by brokkr-smithy-dev
|
||||
# 2026-08-23 while wiring provenance into a probe harness.
|
||||
#
|
||||
# Other current collisions: gen-frontier / gen-frontier-reasoning / glm-5.2 /
|
||||
# glm-5.2-reasoning -> glm-5.2; ext-tts / gpt-4o-mini-tts / tts-1 / tts-1-hd ->
|
||||
# Other current collisions: glm-5.2 / glm-5.2-reasoning -> glm-5.2;
|
||||
# gen-frontier / gen-frontier-reasoning / glm-5.3 -> glm-5.3; ext-tts / gpt-4o-mini-tts / tts-1 / tts-1-hd ->
|
||||
# the fleet TTS gateway; reranker / reranker-a3-bge-v2-m3 -> bge-reranker-v2-m3.
|
||||
#
|
||||
# TO CHECK BEFORE RELYING ON TWO ALIASES BEING DIFFERENT MODELS:
|
||||
@@ -599,6 +599,11 @@ model_list:
|
||||
# strips a top-level `thinking` param (drop_params), but forwards `extra_body`
|
||||
# verbatim to z.ai, where the native thinking:{type:disabled} control lands —
|
||||
# verified reasoning_tokens→0. Reasoning is opt-in via glm-5.1-reasoning below.
|
||||
# ⚠ 2026-10-07 re-measure: "disabled" is NOT zero thinking on a hard prompt. Direct to
|
||||
# z.ai, n=2–3 each: glm-5.1 / glm-5.2 / glm-5.3 with thinking:disabled spent 220–340
|
||||
# reasoning tokens (enabled: ~1,400–1,600). On an easy prompt it spent 0. So read
|
||||
# "disabled" as LIGHT thinking, about effort `low`. Unknown whether z.ai changed this or
|
||||
# the 06-11 check used an easy prompt.
|
||||
- model_name: glm-5.1
|
||||
litellm_params:
|
||||
model: openai/glm-5.1
|
||||
@@ -626,7 +631,9 @@ model_list:
|
||||
# glm-5.2-reasoning. extra_body.thinking is forwarded verbatim to z.ai.
|
||||
# CANONICAL LIMITS (probed live vs z.ai 2026-07-05): 1,048,576-token (1M, 2^20)
|
||||
# INPUT context; 131,072 (128K) MAX OUTPUT (z.ai max_tokens range [1,131072]).
|
||||
# NO gateway-side cap — pure z.ai passthrough, so these are the effective limits.
|
||||
# GATEWAY LIMITS (Prime 2026-10-07, every GLM 5.2/5.3 entry): model_info advertises
|
||||
# 1,048,576 in / 131,072 out, and max_tokens 131,072 is the DEFAULT when a caller
|
||||
# sends none (a caller's own max_tokens wins). Reasoning tokens count against it.
|
||||
- model_name: glm-5.2
|
||||
litellm_params:
|
||||
model: openai/glm-5.2
|
||||
@@ -634,9 +641,14 @@ model_list:
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
max_tokens: 131072
|
||||
extra_body:
|
||||
thinking:
|
||||
type: disabled
|
||||
model_info:
|
||||
mode: chat
|
||||
max_input_tokens: 1048576
|
||||
max_output_tokens: 131072
|
||||
- model_name: glm-5.2-reasoning
|
||||
litellm_params:
|
||||
model: openai/glm-5.2
|
||||
@@ -644,35 +656,76 @@ model_list:
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
max_tokens: 131072
|
||||
extra_body:
|
||||
thinking:
|
||||
type: enabled
|
||||
model_info:
|
||||
mode: chat
|
||||
max_input_tokens: 1048576
|
||||
max_output_tokens: 131072
|
||||
# glm-5.3 (added 2026-10-07, Prime; z.ai release 2026-08-14, same base as 5.2). Text only;
|
||||
# 1M in / 128K out per docs.z.ai/guides/llm/glm-5.3. ⚠ REASONING CANNOT BE TURNED OFF on
|
||||
# 5.3 (z.ai docs; measured 2026-10-07: `thinking: disabled` still spent ~250 reasoning
|
||||
# tokens on a hard prompt, the same as effort `low`). Efforts: low / high / max
|
||||
# (`medium` is accepted, not rejected, so reasoning_effort_map.py does not map it).
|
||||
# This entry leaves the effort to z.ai's default (high-like on our probes) or the caller.
|
||||
- model_name: glm-5.3
|
||||
litellm_params:
|
||||
model: openai/glm-5.3
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
max_tokens: 131072
|
||||
extra_body:
|
||||
thinking:
|
||||
type: enabled
|
||||
model_info:
|
||||
mode: chat
|
||||
max_input_tokens: 1048576
|
||||
max_output_tokens: 131072
|
||||
# gen-frontier / gen-frontier-reasoning: capability aliases for the PAID
|
||||
# frontier tier (GLM 5.2 @ z.ai), mirroring glm-5.2 / glm-5.2-reasoning
|
||||
# (thinking off / on). Worldtree binds these for frontier-grade generation
|
||||
# / reasoning; swap the backing frontier model here, callers unaffected.
|
||||
# frontier tier, GLM 5.3 @ z.ai since 2026-10-07 (Prime; was GLM 5.2). Worldtree
|
||||
# binds these for frontier-grade generation / reasoning; swap the backing frontier
|
||||
# model here, callers unaffected. 5.3 cannot disable reasoning, so the old
|
||||
# "thinking off / on" pair is now effort LOW / HIGH: gen-frontier is the fast
|
||||
# variant, gen-frontier-reasoning the deliberate one (measured on one hard prompt,
|
||||
# n=3: low ~250–350 reasoning tokens, high ~1,400–1,500).
|
||||
# PAID — only all-proxy-models / explicitly-scoped keys reach them; the free
|
||||
# all-agents-local key is fenced off z.ai spend and cannot.
|
||||
- model_name: gen-frontier
|
||||
litellm_params:
|
||||
model: openai/glm-5.2
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
extra_body:
|
||||
thinking:
|
||||
type: disabled
|
||||
- model_name: gen-frontier-reasoning
|
||||
litellm_params:
|
||||
model: openai/glm-5.2
|
||||
model: openai/glm-5.3
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
max_tokens: 131072
|
||||
extra_body:
|
||||
thinking:
|
||||
type: enabled
|
||||
reasoning_effort: low
|
||||
model_info:
|
||||
mode: chat
|
||||
max_input_tokens: 1048576
|
||||
max_output_tokens: 131072
|
||||
- model_name: gen-frontier-reasoning
|
||||
litellm_params:
|
||||
model: openai/glm-5.3
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
max_tokens: 131072
|
||||
extra_body:
|
||||
thinking:
|
||||
type: enabled
|
||||
reasoning_effort: high
|
||||
model_info:
|
||||
mode: chat
|
||||
max_input_tokens: 1048576
|
||||
max_output_tokens: 131072
|
||||
- model_name: glm-5-turbo
|
||||
litellm_params:
|
||||
model: openai/glm-5-turbo
|
||||
|
||||
Reference in New Issue
Block a user