feat(litellm): front z.ai GLM cloud models through the gateway

Adds explicit gateway entries for the four z.ai GLM models (glm-5.1,
glm-5-turbo, glm-4.7, glm-4.5-air) routed to api.z.ai with Z_AI_API_KEY,
plus the compose env passthrough + .env.example doc. Explicit entries
win over the llama-swap wildcard (distinct IDs, no collision). Extends
the gateway's unified logging to cloud inference, not just local
vLLM/llama-swap.

Cost note: paid API — only gateway-keyed callers reach these, but calls
spend z.ai credits (documented in config + compose comments).
This commit is contained in:
vh
2026-06-04 11:13:32 -07:00
parent 349b3614db
commit c7ee8884c6
3 changed files with 34 additions and 0 deletions
+5
View File
@@ -30,6 +30,11 @@ POSTGRES_PASSWORD=
# vllm stack's .env.
VLLM_API_KEY=
# Cloud-provider API keys fronted by the gateway (PAID — spend on use; only
# gateway-keyed callers can reach them). z.ai GLM models
# (glm-5.1 / glm-5-turbo / glm-4.7 / glm-4.5-air).
Z_AI_API_KEY=
# --- Langfuse-ready (leave blank for the lean first cut) ---
# Fill these AND uncomment success/failure_callback in conf/config.yaml
# to ship full traces to a Langfuse instance. That is the whole upgrade.
+3
View File
@@ -42,6 +42,9 @@ services:
# Upstream vLLM API key (empty in the vllm stack's .env by default →
# leave blank; LiteLLM still needs the var to exist).
- VLLM_API_KEY=${VLLM_API_KEY:-}
# Cloud API keys fronted by the gateway for unified logging (z.ai GLM,
# etc.). Paid — only gateway-keyed callers reach them, but they spend.
- Z_AI_API_KEY=${Z_AI_API_KEY:-}
# Langfuse-ready: blank until you bolt Langfuse on. Filling these +
# uncommenting the callback in config.yaml is the entire upgrade.
- LANGFUSE_PUBLIC_KEY=${LANGFUSE_PUBLIC_KEY:-}
+26
View File
@@ -45,6 +45,32 @@ model_list:
model_info:
mode: rerank
# --- z.ai GLM (cloud API) — fronted for unified logging across local
# + cloud inference. Explicit entries, so they win over the "*"
# wildcard below (no collision with llama-swap's glm4.7-flash etc.
# — different model IDs). NOTE: paid API; only gateway-keyed callers
# can reach these, but they DO spend z.ai credits. Key in .env. ---
- model_name: glm-5.1
litellm_params:
model: openai/glm-5.1
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
- model_name: glm-5-turbo
litellm_params:
model: openai/glm-5-turbo
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
- model_name: glm-4.7
litellm_params:
model: openai/glm-4.7
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
- model_name: glm-4.5-air
litellm_params:
model: openai/glm-4.5-air
api_base: https://api.z.ai/api/coding/paas/v4
api_key: os.environ/Z_AI_API_KEY
# --- llama-swap passthrough (the swappable generative LLM zoo on
# ana-ml2:9292) ---
# Wildcard: any model name NOT matched by an exact entry above routes to