From 847424c3cfc28fc221655a3a71ce5acfb25a325d Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 6 Oct 2026 15:36:09 -0700 Subject: [PATCH] docs(litellm): mark store_prompts_in_spend_logs=false as a consumer guarantee (arbo incognito) Verified 2026-10-06 for comfy-dev: the gateway keeps no request or response content. A canary sent through 'gen' appears in none of SpendLogs' text columns, LiteLLM's container log, any ana-docker container log, or flash-next's log (positive controls hit on the same stores). In 24h, 0 of 281,432 spend rows carried messages, response or proxy_server_request. Langfuse is retired, there are no DB-stored callbacks, and LiteLLM_ErrorLogs is empty. The per-request no-log flag skips third-party callbacks but not the proxy's own spend-log callback (v1.97 source), so the guarantee is this setting. Comment-only change; the live file was updated in place with no restart. --- stacks/litellm/conf/config.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/stacks/litellm/conf/config.yaml b/stacks/litellm/conf/config.yaml index fa5602b..a8d7c1d 100644 --- a/stacks/litellm/conf/config.yaml +++ b/stacks/litellm/conf/config.yaml @@ -1100,6 +1100,15 @@ general_settings: # (tokens, latency, model, cost) — the cross-project spend tracking survives, # the bulky bodies do not. Re-enable ONLY for a bounded debugging window, not # standing. + # ⚠ CONSUMERS DEPEND ON THIS BEING FALSE (2026-10-06): arbo's "incognito" runs + # (comfy-dev, Prime's ask) promise that no prompt or image is kept on our side, and + # this flag is the guarantee. The per-request `"no-log": true` flag does NOT cover + # it: in v1.97 it skips third-party logging callbacks but still runs the proxy's own + # `_PROXY_*` spend-log callback (litellm_logging.py should_run_callback). Before ANY + # debugging window with this set to true, tell comfy-dev first and turn it back + # off after. Verified 2026-10-06 with a canary through `gen`: the canary was in 0 of + # SpendLogs' text columns, LiteLLM's container log, every ana-docker container log + # and flash-next's log. Positive controls hit on the same stores. store_prompts_in_spend_logs: false # HARD CAP on SpendLogs growth (operator: "if there's a way to cap it, CAP # it"). The retention job deletes rows older than the period on the interval