From 01b5ad93ed4370317f5883c372023026735d40d9 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Sun, 16 Aug 2026 21:23:07 -0700 Subject: [PATCH] chore(litellm): upgrade v1.91.0 -> v1.97.0; purge + cap the 6GB spend-log DB Operator: update LiteLLM to latest and repull; get rid of the spend-log DB and cap its growth. Upgrade: pinned v1.97.0 (latest stable point release; v1.98.0-rc.1 skipped as a pre-release on the fleet gateway, v1.97.0-stable not yet cut). Image pre-pulled, DB pg_dump'd (1.8GB gz, keys+config+schema) and .env backed up before the Prisma migration, which applied cleanly. DB was 6.08 GB, 6.02 GB of it LiteLLM_SpendLogs storing full prompt+ completion bodies (store_prompts_in_spend_logs: true). Purged via TRUNCATE on the running 1.91 BEFORE the upgrade so the schema migration ran against an empty table -- 6081 MB -> 16 MB, keys (32) and models (3) intact. 'Get rid of the db' read as the spend-log DATA, not the database: dropping it would have destroyed every virtual key (incl. the Lobe key) and the model config in the same DB. Cap: store_prompts_in_spend_logs -> false (bodies no longer persisted; lightweight cost/usage rows and cross-project spend tracking survive) plus maximum_spend_logs_retention_period 7d / interval 1d as a hard age bound. Verified post-upgrade: v1.97.0 running, liveliness 200, 31-model roster, chat round-trip on master + scoped Lobe key, key scoping still enforced (glm-5.2 blocked), ext-tts 200, and store_prompts confirmed off (a marked prompt persisted 0 bodies; 4 lightweight rows). Rollback: .env LITELLM_TAG=v1.91.0 + the 1.8GB dump, both on the host. --- stacks/litellm/.env.example | 2 +- stacks/litellm/conf/config.yaml | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/stacks/litellm/.env.example b/stacks/litellm/.env.example index 0c2fde8..b26f72e 100644 --- a/stacks/litellm/.env.example +++ b/stacks/litellm/.env.example @@ -3,7 +3,7 @@ # Image tag. main-stable is the rolling stable; pin to a dated/SHA tag # (e.g. main-v1.74.0-stable) once a known-good build is confirmed. -LITELLM_TAG=main-stable +LITELLM_TAG=v1.97.0 # Publish. Bind to all interfaces on the LAN; 4000 is the LiteLLM default # (proxy API + admin/Logs UI at /ui). diff --git a/stacks/litellm/conf/config.yaml b/stacks/litellm/conf/config.yaml index 3ef7fe5..0b5f8b7 100644 --- a/stacks/litellm/conf/config.yaml +++ b/stacks/litellm/conf/config.yaml @@ -537,7 +537,20 @@ general_settings: # THE log switch: persists full request messages + response bodies into # SpendLogs so they render in the Logs UI. Without this you get metadata # (tokens, latency, model) but not the prompt/completion text. - store_prompts_in_spend_logs: true + # + # ⚠️ TURNED OFF 2026-08-16 (operator: "I don't need any of that information"). + # With this TRUE the SpendLogs table stored every prompt+completion body and + # grew to 6.0 GB (of a 6.08 GB DB). Off = lightweight cost/usage rows only + # (tokens, latency, model, cost) — the cross-project spend tracking survives, + # the bulky bodies do not. Re-enable ONLY for a bounded debugging window, not + # standing. + store_prompts_in_spend_logs: false + # HARD CAP on SpendLogs growth (operator: "if there's a way to cap it, CAP + # it"). The retention job deletes rows older than the period on the interval + # cadence, so the table is bounded by ~7 days of lightweight rows rather than + # unbounded. Names verified against LiteLLM docs (proxy/spend_logs_deletion). + maximum_spend_logs_retention_period: "7d" + maximum_spend_logs_retention_interval: "1d" # scalar-judge → Skywork-Reward-V2 (scalar reward model; vLLM pooling on # ana-ml2:8003). LiteLLM has no reward/pooling MODE, so this is a passthrough, # not a model_list alias. Gateway-key-gated. Consumers POST the reward body to