esh-pfi-infrastructure/stacks/lobe-chat/README.md

lobe-chat — chat frontend over the LiteLLM gateway (esh-docker-vm)

Evaluation replacement for the hand-rolled gateway-chat single-file HTML surface, which the operator does not want to keep improving — it has already produced two defects (a 1024 max_tokens default that read as model degeneracy, and a NaN→null max_tokens bug).

• Host: esh-docker-vm (10.0.50.45) · Port: 3210 · URL: http://10.0.50.45:3210
• Backend: LiteLLM gateway at 10.250.50.70:4000/v1 (reachable from ESH, ~30 ms)

Why Lobe over Open WebUI

Weight, measured from the registries rather than from marketing:

	compressed	layers
Lobe Chat	143 MB	1
Open WebUI	1,825 MB	19

12.8×. Open WebUI was declined by the operator in June 2026 on weight grounds and that objection still holds. (Its recorded secondary objection — the empty-tools 400 against vLLM — is now moot: the strip_empty_tools callback covers the normal API path and only failed to protect LiteLLM's own built-in playground.)

ANSWERED — Lobe TTS is a SPLIT: endpoint env-driven, voice/model/format UI-only

Resolved 2026-08-16 against the running image (.next server bundle + route probe), not docs:

• There is a server-side route (backend)/webapi/tts/openai/route.js — TTS goes browser → Lobe server → the OpenAI provider, whose server-side base URL is OPENAI_PROXY_URL. So the TTS endpoint inherits the gateway and reaches ext-tts with no extra config. (Probing the route bare returns 401 — it exists and wants the browser's provider payload — not 404.)
• No process.env.*TTS* / *AUDIO* / *SPEECH* vars exist at all. Voice, model, response_format, and the enable-toggle live in a client-side settings store (bundle key TTS_SETTING_KEY = 'tts'), configured in Settings, per browser. There is no AUDIO_TTS_* equivalent to Open WebUI's.

Verdict against the operator's "manageable OR scriptable" criterion: the load-bearing part (endpoint → gateway → ext-tts) IS env-scriptable and already wired; voice/model/format are a one-time UI setup per browser, not a maintenance surface. That is "manageable", which was acceptable. Harmless-by-now footnotes: a voice mis-click can't 404 (tts-dev aliased the full OpenAI set), and an unknown model routes to the gateway default. The ONE thing not forceable by env is response_format: mp3 — if Lobe defaults to WAV you get 23.5 MB/turn; verify at first use and set it in the UI.

Trade recorded for the record: Open WebUI has dedicated AUDIO_TTS_* env vars (fully scriptable) at 1,825 MB; Lobe is endpoint-env + UI-cosmetic at 143 MB. Operator chose to proceed with Lobe.

TTS — verified with tts-dev, 2026-08-16

Route: the ext-tts LiteLLM alias. Do NOT go direct to the seat. The gateway exists so engines can be auditioned and swapped behind it — dots was swapped twice in the week before this deploy and consumers noticed nothing. Direct-to-seat means eating every engine change.

Auth: the gateway virtual key, nothing more. The seat itself has no auth (LAN/WireGuard-internal).

Shape: plain OpenAI POST /v1/audio/speech with {model, input, voice, response_format}. No deviations; a stock OpenAI client works drop-in. Unknown model values route to default rather than 404ing, so tts-1 is harmless. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. speed honoured 0.25–4.0.

⚠️ CORRECTION — my earlier voice foot-gun warning was wrong

This file previously warned that any non-fleet OpenAI voice name 404s and could trip the router cooldown, and told you to pin the voice. That was stale and mostly unfounded. tts-dev verified live: the OpenAI names are ALIASED, not rejected — designed for exactly this case, a stock client dropping in without knowing fleet voice names.

alloy, echo, onyx, ash  -> donut      nova            -> miranda
shimmer, coral          -> emmie      fable, sage     -> glados

Authoritative list (13): computer, computer-soft, computer-urgent, donut, emmie, glados, miranda, sindra, sindra-excited, sindra-sad, sindra-soft, sindra-sultry, sindra-whisper.

Voice surface is now fully safe (tts-dev, 2026-08-16, tts-stack c55bc3c). ballad→emmie and verse→donut were the last two unaliased OpenAI names; they are now aliased and live. Verified: the entire modern OpenAI voice set returns 200, and only a genuinely-unknown string (wharrgarbl) 404s. A stock Lobe voice picker cannot produce a 404 from any voice it would plausibly offer, so it cannot trip the router cooldown. The env-vs-UI voice question is therefore moot for safety — pin mp3 for size, but the voice string itself no longer needs pinning.

The real constraint is SIZE, not length

The 71.2s per-call cap in older notes is dead — that was Zonos-era and required client-side chunking. dots chunks server-side. tts-dev threw a 592-word single call at it: HTTP 200, 66.4 s wall, 245 s of audio, byte-identical ending under ASR diff. Long assistant turns are handled, not clipped.

But: 245 s of WAV is 23.5 MB. For a browser UI set response_format: "mp3" or you will push tens of megabytes per turn at the user. And 66 s of wall time is a long wait with no streaming in the OpenAI-compat path — dots does have a genuine streaming mode (stream: true, ~0.40 s to first audio, flat with length) but that is not the OpenAI-compat shape and would be a custom integration.

⚠️ Capacity — tell tts-dev if this ramps

The seat serializes generation — one render at a time, by design, no continuous batching. A second concurrent consumer is a real capacity question, not a theoretical one, and dots shares a GPU where tts-dev had two resource incidents that week. Report sustained volume, and any voice string not on the list above, to tts-dev on althing rather than letting him infer it from a VRAM graph.

Credential posture

Deliberately not the shared all-agents key — that reaches the paid passthroughs (GLM, Kimi), and a LAN-exposed chat UI holding it would let anyone who can reach the port spend vendor credits from a pool shared across every project.

This stack uses a purpose-minted LiteLLM virtual key, key_alias: lobe-chat-esh, scoped to the 20 free local models. Scoping was verified at mint time, both directions:

• gen → answers
• glm-5.2, kimi-k3, gen-frontier → key not allowed to access model

Secrets are in the vault, never in git. .env on the host is 0600:

secret get esh-docker-vm/lobe-chat-litellm-key        # -> OPENAI_API_KEY
secret get esh-docker-vm/lobe-chat-access-code        # -> ACCESS_CODE (UI gate)
secret get esh-docker-vm/lobe-chat-key-vaults-secret  # -> KEY_VAULTS_SECRET

ACCESS_CODE matters: this is a home-lab LAN segment with nothing in front of it.

Verified on deploy (2026-08-16)

• container healthy; http://10.0.50.45:3210/ → 307 → /chat → 200
• from inside the container: GET /v1/models returns the fleet seats, and a gen chat round-trip returns "ok" — so the app's own network path and key work, not merely the host's
• image on disk 617 MB (143 MB compressed)

Deploy

scripts/deploy-stack.sh esh-docker-vm lobe-chat --compose
# on host: populate .env from the vault (see above), chmod 600, then
ssh lkraven@10.0.50.45 'cd /opt/docker/compose/lobe-chat && docker compose up -d'

lkraven owns /opt/docker and is in the docker group on this host, so no sudo is needed. Note ESH is outside the infra-ops NOPASSWD grant.

System Agent — why gpt-5-mini was being called

Lobe has a System Agent: a background model, separate from your chat model, that auto-names conversations, summarizes history, rewrites RAG queries, and generates assistant metadata. Its default is openai/gpt-5-mini. Because our OpenAI provider is the gateway, that model name is forwarded verbatim to LiteLLM, and the scoped key (local models only) 403s it — filling the gateway log with "Tried to access gpt-5-mini" and silently breaking auto-naming.

Fixed by env (SYSTEM_AGENT), so this is scriptable, not a UI setting — a point IN Lobe's favour on the manageable-by-agent axis, unlike the output-token cap:

SYSTEM_AGENT=topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen,queryRewrite=openai/summarizer,historyCompress=openai/summarizer,thread=openai/summarizer

All six documented keys are set explicitly — any omitted key falls back to the gpt-5-mini default. summarizer/classifier are the same seat as gen at temp 0, the right fit for naming/summarizing; gen where output quality matters.