feat(litellm): front z.ai GLM cloud models through the gateway
Adds explicit gateway entries for the four z.ai GLM models (glm-5.1, glm-5-turbo, glm-4.7, glm-4.5-air) routed to api.z.ai with Z_AI_API_KEY, plus the compose env passthrough + .env.example doc. Explicit entries win over the llama-swap wildcard (distinct IDs, no collision). Extends the gateway's unified logging to cloud inference, not just local vLLM/llama-swap. Cost note: paid API — only gateway-keyed callers reach these, but calls spend z.ai credits (documented in config + compose comments).
This commit is contained in:
@@ -45,6 +45,32 @@ model_list:
|
||||
model_info:
|
||||
mode: rerank
|
||||
|
||||
# --- z.ai GLM (cloud API) — fronted for unified logging across local
|
||||
# + cloud inference. Explicit entries, so they win over the "*"
|
||||
# wildcard below (no collision with llama-swap's glm4.7-flash etc.
|
||||
# — different model IDs). NOTE: paid API; only gateway-keyed callers
|
||||
# can reach these, but they DO spend z.ai credits. Key in .env. ---
|
||||
- model_name: glm-5.1
|
||||
litellm_params:
|
||||
model: openai/glm-5.1
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
- model_name: glm-5-turbo
|
||||
litellm_params:
|
||||
model: openai/glm-5-turbo
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
- model_name: glm-4.7
|
||||
litellm_params:
|
||||
model: openai/glm-4.7
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
- model_name: glm-4.5-air
|
||||
litellm_params:
|
||||
model: openai/glm-4.5-air
|
||||
api_base: https://api.z.ai/api/coding/paas/v4
|
||||
api_key: os.environ/Z_AI_API_KEY
|
||||
|
||||
# --- llama-swap passthrough (the swappable generative LLM zoo on
|
||||
# ana-ml2:9292) ---
|
||||
# Wildcard: any model name NOT matched by an exact entry above routes to
|
||||
|
||||
Reference in New Issue
Block a user