esh-pfi-infrastructure/stacks/lobe-chat/README.md

lobe-chat — chat frontend over the LiteLLM gateway (esh-docker-vm)

Evaluation replacement for the hand-rolled gateway-chat single-file HTML surface, which the operator does not want to keep improving — it has already produced two defects (a 1024 max_tokens default that read as model degeneracy, and a NaN→null max_tokens bug).

• Host: esh-docker-vm (10.0.50.45) · Port: 3210 · URL: http://10.0.50.45:3210
• Backend: LiteLLM gateway at 10.250.50.70:4000/v1 (reachable from ESH, ~30 ms)

Why Lobe over Open WebUI

Weight, measured from the registries rather than from marketing:

	compressed	layers
Lobe Chat	143 MB	1
Open WebUI	1,825 MB	19

12.8×. Open WebUI was declined by the operator in June 2026 on weight grounds and that objection still holds. (Its recorded secondary objection — the empty-tools 400 against vLLM — is now moot: the strip_empty_tools callback covers the normal API path and only failed to protect LiteLLM's own built-in playground.)

ANSWERED — Lobe TTS is a SPLIT: endpoint env-driven, voice/model/format UI-only

Resolved 2026-08-16 against the running image (.next server bundle + route probe), not docs:

• There is a server-side route (backend)/webapi/tts/openai/route.js — TTS goes browser → Lobe server → the OpenAI provider, whose server-side base URL is OPENAI_PROXY_URL. So the TTS endpoint inherits the gateway. (Probing the route bare returns 401 — it exists and wants the browser's provider payload — not 404.)
• No process.env.*TTS* / *AUDIO* / *SPEECH* vars exist at all. Voice, model, and the enable-toggle live in a client-side settings store (bundle key TTS_SETTING_KEY = 'tts'), configured in Settings, per browser. There is no AUDIO_TTS_* equivalent to Open WebUI's.

Verdict against the operator's "manageable OR scriptable" criterion: the load-bearing part (endpoint → gateway) IS env-scriptable and already wired; voice and model are a one-time UI setup per browser, not a maintenance surface. That is "manageable", which was acceptable.

⚠ CORRECTION 2026-08-17 (tts-dev) — this section's two footnotes were wrong,

and one of them meant TTS did not work at all

Both claims below were reasoned from the wrong hop. Re-measured end-to-end from esh-docker-vm with this stack's own .env key:

1. "An unknown model routes to the gateway default" — false on this path. True direct-to-:8198, which accepts-and-ignores model. But requests go through LiteLLM, which resolves the model name first. Lobe's default is tts-1, and POST $OPENAI_PROXY_URL/audio/speech {"model":"tts-1",...} returned 403 key not allowed to access model (the scoped key's allow-list held ext-tts alone; on an unscoped key it is a 400 Invalid model name). So TTS was dead on arrival from deploy until 2026-08-17 — the endpoint being wired is necessary, not sufficient. Fixed the same day: infra-ops aliased tts-1, tts-1-hd and gpt-4o-mini-tts to ext-tts's upstream and extended the lobe-chat-esh key allow-list 20 → 23 models, so the stock payload now works with zero client-side settings (verified from this host with this .env: 200, audio/mpeg, real MP3). Setting the UI model field to ext-tts remains valid and harmless.
2. "response_format: mp3 … set it in the UI" — not possible. Lobe's OpenAI TTS client sends exactly three fields (server bundle chunks/29685.js): { input, model: options?.model || 'tts-1', voice }. response_format is never sent and has no UI setting, so format cannot be chosen from this side at all. You get the fleet gateway's default, which today is WAV (~123 KB for three words; 23.5 MB for a 245 s turn), and LiteLLM labels it content-type: audio/mpeg regardless of the bytes. Chrome sniffs and plays it. Fixing the size is tts-dev's side of the fence, not a setting here.

Nothing to do in the browser any more. The alias + allow-list pair above removed the manual step entirely; a fresh browser, a phone, or any stock OpenAI client works out of the box.

⚠️ Coupling worth knowing: tts-1, tts-1-hd and gpt-4o-mini-tts are independent LiteLLM DB rows pointing at ext-tts's upstream, not references to ext-tts. If that upstream ever repoints, all four move together or stock clients quietly land on a dead engine.

Trade recorded for the record: Open WebUI has dedicated AUDIO_TTS_* env vars (fully scriptable) at 1,825 MB; Lobe is endpoint-env + UI-cosmetic at 143 MB. Operator chose to proceed with Lobe.

TTS — verified with tts-dev, 2026-08-16

Route: the ext-tts LiteLLM alias. Do NOT go direct to the seat. The gateway exists so engines can be auditioned and swapped behind it — dots was swapped twice in the week before this deploy and consumers noticed nothing. Direct-to-seat means eating every engine change.

Auth: the gateway virtual key, nothing more. The seat itself has no auth (LAN/WireGuard-internal).

Shape: plain OpenAI POST /v1/audio/speech with {model, input, voice, response_format}. No deviations; a stock OpenAI client works drop-in. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. speed honoured 0.25–4.0.

⚠️ model is the one field that is NOT free-form on this route. That tolerance belongs to :8198, which accepts-and-ignores model; LiteLLM in front of it does not — it resolves the name first, so tts-1 never reaches the gateway (403 on the scoped key, 400 on an unscoped one). Set Lobe's TTS model to ext-tts. See the 2026-08-17 correction above.

⚠️ CORRECTION — my earlier voice foot-gun warning was wrong

This file previously warned that any non-fleet OpenAI voice name 404s and could trip the router cooldown, and told you to pin the voice. That was stale and mostly unfounded. tts-dev verified live: the OpenAI names are ALIASED, not rejected — designed for exactly this case, a stock client dropping in without knowing fleet voice names.

alloy, echo, onyx, ash  -> donut      nova            -> miranda
shimmer, coral          -> emmie      fable, sage     -> glados

Authoritative list (13): computer, computer-soft, computer-urgent, donut, emmie, glados, miranda, sindra, sindra-excited, sindra-sad, sindra-soft, sindra-sultry, sindra-whisper.

Voice surface is now fully safe (tts-dev, 2026-08-16, tts-stack c55bc3c). ballad→emmie and verse→donut were the last two unaliased OpenAI names; they are now aliased and live. Verified: the entire modern OpenAI voice set returns 200, and only a genuinely-unknown string (wharrgarbl) 404s. A stock Lobe voice picker cannot produce a 404 from any voice it would plausibly offer, so it cannot trip the router cooldown. The env-vs-UI voice question is therefore moot for safety — the voice string does not need pinning. (Format is a different matter; see below — and it is not pinnable from here at all.)

The real constraint is SIZE, not length

The 71.2s per-call cap in older notes is dead — that was Zonos-era and required client-side chunking. dots chunks server-side. tts-dev threw a 592-word single call at it: HTTP 200, 66.4 s wall, 245 s of audio, byte-identical ending under ASR diff. Long assistant turns are handled, not clipped.

But: 245 s of WAV is 23.5 MB, and WAV is what this deploy gets — Lobe never sends response_format, so the fleet gateway's default (native lane → engine → wav) is what lands. That is not fixable from this stack: no env var, no UI field, not in the payload. It is a fleet-gateway default, and tts-dev owns it. Until it changes, expect a few MB per spoken turn on the LAN — wasteful, not fatal.

And 66 s of wall time is a long wait with no streaming in the OpenAI-compat path — dots does have a genuine streaming mode (stream: true, ~0.40 s to first audio, flat with length) but that is not the OpenAI-compat shape and would be a custom integration. Lobe sends no speed either, so the gateway's atempo path is unreachable from here.

⚠️ Capacity — tell tts-dev if this ramps

The seat serializes generation — one render at a time, by design, no continuous batching. A second concurrent consumer is a real capacity question, not a theoretical one, and dots shares a GPU where tts-dev had two resource incidents that week. Report sustained volume, and any voice string not on the list above, to tts-dev on althing rather than letting him infer it from a VRAM graph.

Credential posture

⚠️ CHANGED 2026-08-21 — this key is NO LONGER free-local-only

On the operator's explicit instruction, infra-ops replaced the key's explicit model array with the all-proxy-models access group. lobe-chat-esh now reaches everything behind the gateway, paid passthroughs included — glm-*, kimi-k3, gen-frontier*. The scoping described below is history, kept because the reasoning still explains what the guard rail was for.

What this means in practice: a paid call through this key spends real vendor credits, and the key has max_budget: None — no ceiling. The UI in front of it is LAN-exposed with an access code and nothing else. A spend cap is available on request from infra-ops; the operator has not asked for one, so it is uncapped by instruction, not by oversight.

Two side-effects of the access group worth knowing: it auto-includes future models, so no more per-seat key edits, and it only ever resolves live models — which is how five retired names (char-rp-fable, char-rp-reasoning, lfm2.5-2.6b, qwen3-reranker, reranker-a4-gte-modernbert) dropped off it silently.

The picker is now the only gate. OPENAI_MODEL_LIST decides what a user can select; the key no longer restricts anything. As of 2026-08-21 that list deliberately excludes the paid family — so exposing paid models to this UI is one env edit away, and should stay a conscious decision rather than a drift.

The original posture, for the record:

Deliberately not the shared all-agents key — that reaches the paid passthroughs (GLM, Kimi), and a LAN-exposed chat UI holding it would let anyone who can reach the port spend vendor credits from a pool shared across every project.

This stack used a purpose-minted LiteLLM virtual key, key_alias: lobe-chat-esh, scoped to the 20 free local models. Scoping was verified at mint time, both directions:

• gen → answers
• glm-5.2, kimi-k3, gen-frontier → key not allowed to access model

Secrets are in the vault, never in git. .env on the host is 0600:

secret get esh-docker-vm/lobe-chat-litellm-key        # -> OPENAI_API_KEY
secret get esh-docker-vm/lobe-chat-access-code        # -> ACCESS_CODE (UI gate)
secret get esh-docker-vm/lobe-chat-key-vaults-secret  # -> KEY_VAULTS_SECRET

ACCESS_CODE matters: this is a home-lab LAN segment with nothing in front of it.

Verified on deploy (2026-08-16)

• container healthy; http://10.0.50.45:3210/ → 307 → /chat → 200
• from inside the container: GET /v1/models returns the fleet seats, and a gen chat round-trip returns "ok" — so the app's own network path and key work, not merely the host's
• image on disk 617 MB (143 MB compressed)

Deploy

scripts/deploy-stack.sh esh-docker-vm lobe-chat --compose
# on host: populate .env from the vault (see above), chmod 600, then
ssh lkraven@10.0.50.45 'cd /opt/docker/compose/lobe-chat && docker compose up -d'

lkraven owns /opt/docker and is in the docker group on this host, so no sudo is needed. Note ESH is outside the infra-ops NOPASSWD grant.

System Agent — why gpt-5-mini was being called

Lobe has a System Agent: a background model, separate from your chat model, that auto-names conversations, summarizes history, rewrites RAG queries, and generates assistant metadata. Its default is openai/gpt-5-mini. Because our OpenAI provider is the gateway, that model name is forwarded verbatim to LiteLLM, and the scoped key (local models only) 403s it — filling the gateway log with "Tried to access gpt-5-mini" and silently breaking auto-naming.

Fixed by env (SYSTEM_AGENT), so this is scriptable, not a UI setting — a point IN Lobe's favour on the manageable-by-agent axis, unlike the output-token cap:

SYSTEM_AGENT=topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen,queryRewrite=openai/summarizer,historyCompress=openai/summarizer,thread=openai/summarizer

All six documented keys are set explicitly — any omitted key falls back to the gpt-5-mini default. summarizer/classifier are the same seat as gen at temp 0, the right fit for naming/summarizing; gen where output quality matters.