esh-pfi-infrastructure/stacks/open-webui/README.md

open-webui

Env-declarative chat frontend over the LiteLLM gateway, on esh-docker-vm (10.0.50.45:3211). Stood up 2026-08-21 as a parallel bake-off against lobe-chat (:3210) — operator-approved candidate replacement. Nothing about lobe-chat changes unless/until the operator retires it.

• URL: http://10.0.50.45:3211
• Image: ghcr.io/open-webui/open-webui:main (deployed v0.11.0)
• Compose (server): /opt/docker/compose/open-webui/
• Admin: lkraven / vh@phasefinal.com (first account = admin). Signup is disabled in steady state; add users via the Admin UI.

Why this exists

Lobe's config does not survive a fleet that changes daily: its model list is a static env string that never auto-discovers (rotted both directions), and half its config (TTS/model/voice) lives in each browser's local store, so "configured" is a property of whoever's laptop. Open WebUI is env-declarative (the deploy is the single source of config truth) and auto-tracks the gateway roster with no pins.

The deal-sealer — ENABLE_PERSISTENT_CONFIG=False

Default is True, under which env vars are read once on first boot, written to the DB, then ignored forever (edit-env-and-bounce silently does nothing). We set it False: every knob is re-read from env on each boot, and Admin-UI changes do not survive a restart. Verified both directions at deploy (see gates).

⚠️ A var only reaches the container if compose.yaml references it in its environment: block. Adding a line to .env alone does nothing — Compose uses .env for ${VAR} substitution, not as an env_file. This bit us once: ENABLE_API_KEYS (see below) was set in .env but not wired in compose, so API-key minting stayed 403 until the compose var was added.

Acceptance gate results (verified on the box, 2026-08-21)

Gate	Result	Notes
G1 declarative config works	✅ PASS	env change takes on bounce (signup 200→403); UI/runtime change reverts on restart (first-admin runtime signup-disable + a runtime ENABLE_API_KEYS=true both reverted to env on restart). No persistent-config bug bit this deploy (no Redis; flags honored once named right).
G2 model list auto-tracks LiteLLM, no pins	✅ PASS	Picker = the 31 live gateway models 1:1, nothing hand-listed; a retired gateway model disappears with no OWUI edit. arena-model (+1) is OWUI's own eval-arena feature, not a pin. Tradeoff: the picker also shows non-chat seats (embeddings/rerankers/tts) — the flip side of "no hand-listing"; can't be pruned to chat-only without reintroducing exactly the pinning G2 forbids.
G3 reconciliation API is real	✅ PASS	GET /api/v1/models/export round-trips JSON; POST /api/v1/models/sync genuinely reconciles — create and delete to match desired state (verified: syncing set B removed set A). Sync requires full records (user_id/created_at/updated_at) — it round-trips the export shape. API keys enabled via ENABLE_API_KEYS=True (plural — see warning above).
G5 task model pinned	✅ PASS	TASK_MODEL_EXTERNAL=summarizer (gen seat @ temp 0).
G4 TTS by env, second-browser zero-setup	⏳ tts-dev	Env knobs set (direct at http://10.100.79.3:8198/v1, no LiteLLM); tts-dev runs the gate.

End-to-end sanity: a chat completion to summarizer through OWUI → LiteLLM → seat returns correctly.

Credential posture (G6)

• Fresh, capped LiteLLM key key_alias: open-webui-esh — all-proxy-models (reaches paid passthroughs per the operator's standing posture) but capped: max_budget: $50, budget_duration: 1mo (unlike lobe-chat-esh, which is uncapped — deliberately not inherited). A paid call spends real vendor credits until the cap trips.
• Signup off after admin creation (no first-registrant LAN race).
• Secrets in the vault, never in git:

  secret get esh-docker-vm/open-webui-litellm-key   # -> OPENAI_API_KEY
  secret get esh-docker-vm/open-webui-secret-key      # -> WEBUI_SECRET_KEY
  secret get esh-docker-vm/open-webui-admin           # -> lkraven admin cred state
  

TTS (G4 — tts-dev owns)

Routed direct at the fleet dots gateway http://10.100.79.3:8198/v1, not through LiteLLM (:8198 is itself the swap-proof switchboard; LiteLLM would add a hop + a wrong audio/mpeg stamp; direct needs no gateway key). Five knobs: AUDIO_TTS_ENGINE=openai, AUDIO_TTS_OPENAI_API_BASE_URL, AUDIO_TTS_OPENAI_API_KEY (placeholder — seat is LAN/WG-only, no auth), AUDIO_TTS_MODEL, AUDIO_TTS_VOICE (model/voice are tts-dev's to tune). Reversible to ext-tts via the base URL if audio spend/log visibility is wanted.

Deploy / operate

# Canonical lives here; push to the host:
scripts/deploy-stack.sh esh-docker-vm open-webui --compose
# Real .env lives on the host (secrets from the vault), never committed.

# On the host:
cd /opt/docker/compose/open-webui
$EDITOR .env          # change any knob HERE (env is the source of truth)
docker compose up -d  # bounce to apply — UI changes do NOT persist

# RAG embeddings route through the gateway (qwen3-embedding) to avoid a
# boot-time HuggingFace download and stay fleet-consistent.