From e9362de065c12df3ce6256a3308511241c5b5233 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Sun, 16 Aug 2026 16:42:05 -0700 Subject: [PATCH] feat(lobe-chat): stand up Lobe Chat on esh-docker-vm over the LiteLLM gateway Replacement candidate for the hand-rolled gateway-chat HTML surface, which the operator does not want to keep improving -- it has already produced two defects tonight. Chosen over Open WebUI on weight, measured from the registries rather than recalled: Lobe 143 MB compressed / 1 layer vs Open WebUI 1,825 MB / 19 layers, a 12.8x difference. Open WebUI was declined in June 2026 on weight and that still holds; its secondary recorded objection (empty-tools 400 against vLLM) is now moot since strip_empty_tools covers the normal API path and only missed LiteLLM's built-in playground. CREDENTIAL POSTURE: deliberately NOT the shared all-agents key, which reaches the paid GLM/Kimi passthroughs -- a LAN-exposed chat UI holding it would let anyone reaching the port spend vendor credits from a pool shared across every project. Minted a scoped LiteLLM virtual key (key_alias lobe-chat-esh) limited to the 20 free local models, and verified the scoping BOTH ways: gen answers, glm-5.2 / kimi-k3 / gen-frontier all return 'key not allowed to access model'. Secrets vaulted, host .env 0600. Verified from INSIDE the container, not just from the host: /v1/models returns the fleet seats and a gen round-trip returns 'ok', so the app's own network path and key both work. Container healthy, / -> 307 -> /chat -> 200. Documents the open question this deploy exists to answer: whether Lobe's TTS is ENV-configurable or UI-only. That is the operator's deciding criterion and is NOT yet established -- Open WebUI has dedicated AUDIO_TTS_* vars, Lobe documents a shared OPENAI_PROXY_URL which should carry TTS since LiteLLM serves audio/speech on the same base, but that is inference. Also records the ext-tts voice foot-gun: unknown voices 404 and can trip the router cooldown, so the voice must be pinned rather than left at a UI default. --- .gitignore | 1 + stacks/lobe-chat/.env.example | 10 +++++ stacks/lobe-chat/README.md | 84 +++++++++++++++++++++++++++++++++++ stacks/lobe-chat/compose.yaml | 75 +++++++++++++++++++++++++++++++ 4 files changed, 170 insertions(+) create mode 100644 stacks/lobe-chat/.env.example create mode 100644 stacks/lobe-chat/README.md create mode 100644 stacks/lobe-chat/compose.yaml diff --git a/.gitignore b/.gitignore index 7e108f0..3ef7e3b 100644 --- a/.gitignore +++ b/.gitignore @@ -39,3 +39,4 @@ graphify-out/* # Python bytecode (e.g. from local py_compile of stack wrappers) __pycache__/ *.pyc +stacks/lobe-chat/.env diff --git a/stacks/lobe-chat/.env.example b/stacks/lobe-chat/.env.example new file mode 100644 index 0000000..0bc2e9c --- /dev/null +++ b/stacks/lobe-chat/.env.example @@ -0,0 +1,10 @@ +# lobe-chat on esh-docker-vm. Real .env lives on the host and is NEVER committed. +# Populate from the vault: +# secret get esh-docker-vm/lobe-chat-litellm-key -> OPENAI_API_KEY +# secret get esh-docker-vm/lobe-chat-access-code -> ACCESS_CODE +# secret get esh-docker-vm/lobe-chat-key-vaults-secret -> KEY_VAULTS_SECRET +LOBE_PORT=3210 +OPENAI_PROXY_URL=http://10.250.50.70:4000/v1 +OPENAI_API_KEY= +ACCESS_CODE= +KEY_VAULTS_SECRET= diff --git a/stacks/lobe-chat/README.md b/stacks/lobe-chat/README.md new file mode 100644 index 0000000..c3ae4cd --- /dev/null +++ b/stacks/lobe-chat/README.md @@ -0,0 +1,84 @@ +# lobe-chat — chat frontend over the LiteLLM gateway (esh-docker-vm) + +Evaluation replacement for the hand-rolled `gateway-chat` single-file HTML +surface, which the operator does not want to keep improving — it has already +produced two defects (a 1024 `max_tokens` default that read as model degeneracy, +and a `NaN`→`null` `max_tokens` bug). + +- **Host:** esh-docker-vm (10.0.50.45) · **Port:** 3210 · **URL:** http://10.0.50.45:3210 +- **Backend:** LiteLLM gateway at `10.250.50.70:4000/v1` (reachable from ESH, ~30 ms) + +## Why Lobe over Open WebUI + +Weight, measured from the registries rather than from marketing: + +| | compressed | layers | +|---|---|---| +| Lobe Chat | **143 MB** | 1 | +| Open WebUI | 1,825 MB | 19 | + +12.8×. Open WebUI was declined by the operator in June 2026 on weight grounds and +that objection still holds. (Its recorded *secondary* objection — the empty-`tools` +400 against vLLM — is now moot: the `strip_empty_tools` callback covers the normal +API path and only failed to protect LiteLLM's own built-in playground.) + +## ⚠️ The open question this deploy exists to answer + +**Is Lobe's TTS configurable by ENV, or only through the settings UI?** That is the +operator's deciding criterion — manageable/scriptable by an agent — and it is +unresolved. Open WebUI has dedicated `AUDIO_TTS_ENGINE` / `AUDIO_TTS_OPENAI_API_BASE_URL` +/ `AUDIO_TTS_MODEL` / `AUDIO_TTS_VOICE`. Lobe documents a **shared** `OPENAI_PROXY_URL`, +which *should* carry TTS because LiteLLM serves `/v1/chat/completions` and +`/v1/audio/speech` on the same base — but that is inference, not verification. + +If it turns out UI-only, the real choice is: 1.8 GB with genuine scriptability, or +143 MB with click-ops. That is an operator call, not an agent one. + +## ⚠️ ext-tts voice foot-gun + +`ext-tts` accepts `donut` / `emmie` / `glados` / `miranda` (+ emotion variants) and +the OpenAI aliases `nova` / `alloy`. **Any other OpenAI voice name (`echo`, `fable`, +`onyx`, `shimmer`) 404s and can trip the LiteLLM router cooldown.** Pin the voice +explicitly rather than accepting whatever the UI defaults to. + +## Credential posture + +Deliberately **not** the shared all-agents key — that reaches the paid passthroughs +(GLM, Kimi), and a LAN-exposed chat UI holding it would let anyone who can reach the +port spend vendor credits from a pool shared across every project. + +This stack uses a purpose-minted LiteLLM virtual key, `key_alias: lobe-chat-esh`, +scoped to the 20 free **local** models. Scoping was verified at mint time, both +directions: + +- `gen` → answers +- `glm-5.2`, `kimi-k3`, `gen-frontier` → `key not allowed to access model` + +Secrets are in the vault, never in git. `.env` on the host is `0600`: + +```bash +secret get esh-docker-vm/lobe-chat-litellm-key # -> OPENAI_API_KEY +secret get esh-docker-vm/lobe-chat-access-code # -> ACCESS_CODE (UI gate) +secret get esh-docker-vm/lobe-chat-key-vaults-secret # -> KEY_VAULTS_SECRET +``` + +`ACCESS_CODE` matters: this is a home-lab LAN segment with nothing in front of it. + +## Verified on deploy (2026-08-16) + +- container healthy; `http://10.0.50.45:3210/` → 307 → `/chat` → 200 +- from **inside** the container: `GET /v1/models` returns the fleet seats, and a + `gen` chat round-trip returns `"ok"` — so the app's own network path and key work, + not merely the host's +- image on disk 617 MB (143 MB compressed) + +## Deploy + +```bash +scripts/deploy-stack.sh esh-docker-vm lobe-chat --compose +# on host: populate .env from the vault (see above), chmod 600, then +ssh lkraven@10.0.50.45 'cd /opt/docker/compose/lobe-chat && docker compose up -d' +``` + +`lkraven` owns `/opt/docker` and is in the `docker` group on this host, so no sudo +is needed. Note ESH is outside the infra-ops NOPASSWD grant. diff --git a/stacks/lobe-chat/compose.yaml b/stacks/lobe-chat/compose.yaml new file mode 100644 index 0000000..d59319b --- /dev/null +++ b/stacks/lobe-chat/compose.yaml @@ -0,0 +1,75 @@ +# lobe-chat — evaluation chat frontend over the LiteLLM gateway, on esh-docker-vm. +# +# WHY THIS EXISTS +# `gateway-chat` is a hand-rolled single-file HTML surface the operator does not +# want to keep improving; it has already cost two defects (a 1024 max_tokens +# default that read as model degeneracy, and a NaN->null max_tokens bug). +# Lobe was chosen over Open WebUI on weight, measured from the registries +# rather than from marketing: 143 MB compressed / 1 layer, against Open WebUI's +# 1,825 MB / 19 layers. 12.8x. +# +# OPEN QUESTION THIS DEPLOY EXISTS TO ANSWER +# Whether Lobe's TTS base URL / model / voice are settable by ENV or only via +# the settings UI. Open WebUI has dedicated AUDIO_TTS_* vars; Lobe documents a +# SHARED OPENAI_PROXY_URL, which should work because LiteLLM serves both +# /v1/chat/completions and /v1/audio/speech on one base -- but "should" is not +# "does". If it turns out UI-only it fails the operator's manageable-by-agent +# criterion even while winning on weight. +# +# ⚠️ CREDENTIAL POSTURE — deliberately NOT the shared all-agents key. +# That key reaches the PAID passthroughs (GLM, Kimi) and a LAN-exposed chat UI +# holding it would let anyone who can reach the port spend vendor credits on a +# pool shared across every project. This stack uses a purpose-minted LiteLLM +# virtual key (`key_alias: lobe-chat-esh`) scoped to the 20 free LOCAL models. +# Verified at mint time: `gen` answers; `glm-5.2`, `kimi-k3` and `gen-frontier` +# all return "key not allowed to access model". +# Secrets live in the vault, never in git: +# secret get esh-docker-vm/lobe-chat-litellm-key +# secret get esh-docker-vm/lobe-chat-access-code +# secret get esh-docker-vm/lobe-chat-key-vaults-secret +# +# ⚠️ ext-tts FOOT-GUN: an unknown voice 404s and can trip the LiteLLM router +# cooldown. `ext-tts` accepts donut/emmie/glados/miranda (+ emotion variants) +# and the OpenAI aliases nova/alloy. If Lobe sends any other OpenAI voice name +# (echo, fable, onyx, shimmer) it will 404 -- pin the voice rather than leaving +# it at whatever the UI defaults to. + +name: lobe-chat + +services: + lobe-chat: + image: lobehub/lobe-chat:${LOBE_TAG:-latest} + container_name: ${LOBE_CONTAINER:-lobe-chat} + restart: unless-stopped + ports: + - "${LOBE_PORT:-3210}:3210" + environment: + # Chat + TTS both resolve through the one LiteLLM base. + - OPENAI_API_KEY=${OPENAI_API_KEY:?set in .env, from the vault} + - OPENAI_PROXY_URL=${OPENAI_PROXY_URL:-http://10.250.50.70:4000/v1} + # Pin the visible model list so the picker shows fleet seats rather than + # Lobe's built-in OpenAI catalogue (which would 404 against the gateway). + - OPENAI_MODEL_LIST=${OPENAI_MODEL_LIST:--all,+gen,+gen-reasoning,+char-rp,+char-rp-reasoning,+char-rp-fable,+summarizer,+summarizer-large,+classifier,+image-judge,+coder-fast,+chat-judge} + # LAN-exposed on a home-lab segment with no auth in front: gate it. + - ACCESS_CODE=${ACCESS_CODE:?set in .env, from the vault} + # Encrypts any provider keys a user stores in the UI. + - KEY_VAULTS_SECRET=${KEY_VAULTS_SECRET:?set in .env, from the vault} + healthcheck: + test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:3210/ >/dev/null 2>&1 || exit 1"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 60s + networks: + - tnet + labels: + - homepage.group=Apps + - homepage.name=Lobe Chat + - homepage.icon=mdi-chat-processing + - homepage.description=Chat frontend over the LiteLLM gateway (eval) + - homepage.href=http://10.0.50.45:${LOBE_PORT:-3210} + +networks: + tnet: + name: traefik-net + external: true