vh
c7ee8884c6
Adds explicit gateway entries for the four z.ai GLM models (glm-5.1, glm-5-turbo, glm-4.7, glm-4.5-air) routed to api.z.ai with Z_AI_API_KEY, plus the compose env passthrough + .env.example doc. Explicit entries win over the llama-swap wildcard (distinct IDs, no collision). Extends the gateway's unified logging to cloud inference, not just local vLLM/llama-swap. Cost note: paid API — only gateway-keyed callers reach these, but calls spend z.ai credits (documented in config + compose comments).
44 lines
1.6 KiB
Bash
44 lines
1.6 KiB
Bash
# LiteLLM gateway — copy to .env on ana-docker and fill the secrets.
|
|
# Real .env lives on the server only (gitignored); never commit it.
|
|
|
|
# Image tag. main-stable is the rolling stable; pin to a dated/SHA tag
|
|
# (e.g. main-v1.74.0-stable) once a known-good build is confirmed.
|
|
LITELLM_TAG=main-stable
|
|
|
|
# Publish. Bind to all interfaces on the LAN; 4000 is the LiteLLM default
|
|
# (proxy API + admin/Logs UI at /ui).
|
|
LITELLM_BIND=0.0.0.0
|
|
LITELLM_PORT=4000
|
|
|
|
# Proxy + admin-UI master key. MUST start with "sk-". Generate one:
|
|
# openssl rand -hex 24 | sed 's/^/sk-/'
|
|
LITELLM_MASTER_KEY=
|
|
|
|
# Salt for encrypting any virtual/model keys stored in Postgres. Generate:
|
|
# openssl rand -hex 32
|
|
# Changing this after keys are stored makes them undecryptable — set once.
|
|
LITELLM_SALT_KEY=
|
|
|
|
# Postgres (backing store for spend logs + keys). Password is required;
|
|
# user/db default to "litellm".
|
|
POSTGRES_USER=litellm
|
|
POSTGRES_DB=litellm
|
|
POSTGRES_PASSWORD=
|
|
|
|
# Upstream vLLM API key. The vllm stack on ana-ml2 ships API_KEY= (empty)
|
|
# by default → leave this blank. Set it ONLY if you set API_KEY in the
|
|
# vllm stack's .env.
|
|
VLLM_API_KEY=
|
|
|
|
# Cloud-provider API keys fronted by the gateway (PAID — spend on use; only
|
|
# gateway-keyed callers can reach them). z.ai GLM models
|
|
# (glm-5.1 / glm-5-turbo / glm-4.7 / glm-4.5-air).
|
|
Z_AI_API_KEY=
|
|
|
|
# --- Langfuse-ready (leave blank for the lean first cut) ---
|
|
# Fill these AND uncomment success/failure_callback in conf/config.yaml
|
|
# to ship full traces to a Langfuse instance. That is the whole upgrade.
|
|
LANGFUSE_PUBLIC_KEY=
|
|
LANGFUSE_SECRET_KEY=
|
|
LANGFUSE_HOST=
|