vh
01b5ad93ed
Operator: update LiteLLM to latest and repull; get rid of the spend-log DB and cap its growth. Upgrade: pinned v1.97.0 (latest stable point release; v1.98.0-rc.1 skipped as a pre-release on the fleet gateway, v1.97.0-stable not yet cut). Image pre-pulled, DB pg_dump'd (1.8GB gz, keys+config+schema) and .env backed up before the Prisma migration, which applied cleanly. DB was 6.08 GB, 6.02 GB of it LiteLLM_SpendLogs storing full prompt+ completion bodies (store_prompts_in_spend_logs: true). Purged via TRUNCATE on the running 1.91 BEFORE the upgrade so the schema migration ran against an empty table -- 6081 MB -> 16 MB, keys (32) and models (3) intact. 'Get rid of the db' read as the spend-log DATA, not the database: dropping it would have destroyed every virtual key (incl. the Lobe key) and the model config in the same DB. Cap: store_prompts_in_spend_logs -> false (bodies no longer persisted; lightweight cost/usage rows and cross-project spend tracking survive) plus maximum_spend_logs_retention_period 7d / interval 1d as a hard age bound. Verified post-upgrade: v1.97.0 running, liveliness 200, 31-model roster, chat round-trip on master + scoped Lobe key, key scoping still enforced (glm-5.2 blocked), ext-tts 200, and store_prompts confirmed off (a marked prompt persisted 0 bodies; 4 lightweight rows). Rollback: .env LITELLM_TAG=v1.91.0 + the 1.8GB dump, both on the host.
52 lines
1.9 KiB
Bash
52 lines
1.9 KiB
Bash
# LiteLLM gateway — copy to .env on ana-docker and fill the secrets.
|
|
# Real .env lives on the server only (gitignored); never commit it.
|
|
|
|
# Image tag. main-stable is the rolling stable; pin to a dated/SHA tag
|
|
# (e.g. main-v1.74.0-stable) once a known-good build is confirmed.
|
|
LITELLM_TAG=v1.97.0
|
|
|
|
# Publish. Bind to all interfaces on the LAN; 4000 is the LiteLLM default
|
|
# (proxy API + admin/Logs UI at /ui).
|
|
LITELLM_BIND=0.0.0.0
|
|
LITELLM_PORT=4000
|
|
|
|
# Proxy + admin-UI master key. MUST start with "sk-". Generate one:
|
|
# openssl rand -hex 24 | sed 's/^/sk-/'
|
|
LITELLM_MASTER_KEY=
|
|
|
|
# Salt for encrypting any virtual/model keys stored in Postgres. Generate:
|
|
# openssl rand -hex 32
|
|
# Changing this after keys are stored makes them undecryptable — set once.
|
|
LITELLM_SALT_KEY=
|
|
|
|
# Postgres (backing store for spend logs + keys). Password is required;
|
|
# user/db default to "litellm".
|
|
POSTGRES_USER=litellm
|
|
POSTGRES_DB=litellm
|
|
POSTGRES_PASSWORD=
|
|
|
|
# Upstream vLLM API key. The vllm stack on ana-ml2 ships API_KEY= (empty)
|
|
# by default → leave this blank. Set it ONLY if you set API_KEY in the
|
|
# vllm stack's .env.
|
|
VLLM_API_KEY=
|
|
|
|
# Cloud-provider API keys fronted by the gateway (PAID — spend on use; only
|
|
# gateway-keyed callers can reach them). z.ai GLM models
|
|
# (glm-5.1 / glm-5-turbo / glm-4.7 / glm-4.5-air).
|
|
Z_AI_API_KEY=
|
|
|
|
# Kimi (Moonshot) keys — PAID.
|
|
# KIMI_CODE_API_KEY — the CODING endpoint (https://api.kimi.com/coding/v1),
|
|
# Vivace membership; fronts the primary `kimi-k3` arm (upstream model `k3`).
|
|
# MOONSHOT_API_KEY — the general endpoint (https://api.moonshot.ai/v1);
|
|
# fronts the `kimi-k3-gen-api` variant.
|
|
KIMI_CODE_API_KEY=
|
|
MOONSHOT_API_KEY=
|
|
|
|
# --- Langfuse-ready (leave blank for the lean first cut) ---
|
|
# Fill these AND uncomment success/failure_callback in conf/config.yaml
|
|
# to ship full traces to a Langfuse instance. That is the whole upgrade.
|
|
LANGFUSE_PUBLIC_KEY=
|
|
LANGFUSE_SECRET_KEY=
|
|
LANGFUSE_HOST=
|