# lobe-chat — chat frontend over the LiteLLM gateway (esh-docker-vm)

Evaluation replacement for the hand-rolled `gateway-chat` single-file HTML
surface, which the operator does not want to keep improving — it has already
produced two defects (a 1024 `max_tokens` default that read as model degeneracy,
and a `NaN`→`null` `max_tokens` bug).

- **Host:** esh-docker-vm (10.0.50.45) · **Port:** 3210 · **URL:** http://10.0.50.45:3210
- **Backend:** LiteLLM gateway at `10.250.50.70:4000/v1` (reachable from ESH, ~30 ms)

## Why Lobe over Open WebUI

Weight, measured from the registries rather than from marketing:

| | compressed | layers |
|---|---|---|
| Lobe Chat | **143 MB** | 1 |
| Open WebUI | 1,825 MB | 19 |

12.8×. Open WebUI was declined by the operator in June 2026 on weight grounds and
that objection still holds. (Its recorded *secondary* objection — the empty-`tools`
400 against vLLM — is now moot: the `strip_empty_tools` callback covers the normal
API path and only failed to protect LiteLLM's own built-in playground.)

## ANSWERED — Lobe TTS is a SPLIT: endpoint env-driven, voice/model/format UI-only

Resolved 2026-08-16 against the running image (`.next` server bundle + route probe),
not docs:

- **There is a server-side route** `(backend)/webapi/tts/openai/route.js` — TTS goes
  browser → Lobe server → the **OpenAI provider**, whose server-side base URL is
  `OPENAI_PROXY_URL`. So the TTS *endpoint* inherits the gateway. (Probing the route
  bare returns 401 — it exists and wants the browser's provider payload — not 404.)
- **No `process.env.*TTS*` / `*AUDIO*` / `*SPEECH*` vars exist at all.** Voice, model,
  and the enable-toggle live in a **client-side settings store**
  (bundle key `TTS_SETTING_KEY = 'tts'`), configured in Settings, per browser. There is
  no `AUDIO_TTS_*` equivalent to Open WebUI's.

**Verdict against the operator's "manageable OR scriptable" criterion:** the
load-bearing part (endpoint → gateway) IS env-scriptable and already wired; voice and
model are a **one-time UI setup per browser**, not a maintenance surface. That is
"manageable", which was acceptable.

> ### ⚠ CORRECTION 2026-08-17 (tts-dev) — this section's two footnotes were wrong,
> and one of them meant TTS did not work at all
>
> Both claims below were reasoned from the wrong hop. Re-measured end-to-end from
> esh-docker-vm with this stack's own `.env` key:
>
> 1. **"An unknown `model` routes to the gateway default" — false on this path.**
>    True direct-to-`:8198`, which accepts-and-ignores `model`. But requests go
>    through **LiteLLM**, which resolves the model name *first*. Lobe's default is
>    `tts-1`, and `POST $OPENAI_PROXY_URL/audio/speech {"model":"tts-1",...}` returned
>    **403 `key not allowed to access model`** (the scoped key's allow-list held
>    `ext-tts` alone; on an unscoped key it is a 400 `Invalid model name`). **So TTS
>    was dead on arrival from deploy until 2026-08-17** — the endpoint being wired is
>    necessary, not sufficient. **Fixed the same day**: infra-ops aliased `tts-1`,
>    `tts-1-hd` and `gpt-4o-mini-tts` to ext-tts's upstream and extended the
>    `lobe-chat-esh` key allow-list 20 → 23 models, so the stock payload now works
>    with **zero client-side settings** (verified from this host with this `.env`:
>    200, `audio/mpeg`, real MP3). Setting the UI model field to `ext-tts` remains
>    valid and harmless.
> 2. **"`response_format: mp3` … set it in the UI" — not possible.** Lobe's OpenAI
>    TTS client sends exactly three fields (server bundle `chunks/29685.js`):
>    `{ input, model: options?.model || 'tts-1', voice }`. `response_format` is never
>    sent and has no UI setting, so **format cannot be chosen from this side at all.**
>    You get the fleet gateway's default, which today is WAV (~123 KB for three words;
>    23.5 MB for a 245 s turn), and LiteLLM labels it `content-type: audio/mpeg`
>    regardless of the bytes. Chrome sniffs and plays it. Fixing the size is
>    tts-dev's side of the fence, not a setting here.
>
> **Nothing to do in the browser any more.** The alias + allow-list pair above
> removed the manual step entirely; a fresh browser, a phone, or any stock OpenAI
> client works out of the box.
>
> ⚠️ **Coupling worth knowing:** `tts-1`, `tts-1-hd` and `gpt-4o-mini-tts` are
> independent LiteLLM DB rows pointing at ext-tts's upstream, not references to
> `ext-tts`. If that upstream ever repoints, all four move together or stock
> clients quietly land on a dead engine.

Trade recorded for the record: Open WebUI has dedicated `AUDIO_TTS_*` env vars (fully
scriptable) at 1,825 MB; Lobe is endpoint-env + UI-cosmetic at 143 MB. Operator chose
to proceed with Lobe.

## TTS — verified with tts-dev, 2026-08-16

**Route: the `ext-tts` LiteLLM alias. Do NOT go direct to the seat.** The gateway
exists so engines can be auditioned and swapped behind it — dots was swapped twice
in the week before this deploy and consumers noticed nothing. Direct-to-seat means
eating every engine change.

**Auth:** the gateway virtual key, nothing more. The seat itself has no auth
(LAN/WireGuard-internal).

**Shape:** plain OpenAI `POST /v1/audio/speech` with
`{model, input, voice, response_format}`. No deviations; a stock OpenAI client works
drop-in. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. `speed`
honoured 0.25–4.0.

⚠️ **`model` is the one field that is NOT free-form on this route.** That tolerance
belongs to `:8198`, which accepts-and-ignores `model`; **LiteLLM in front of it does
not** — it resolves the name first, so `tts-1` never reaches the gateway (403 on the
scoped key, 400 on an unscoped one). Set Lobe's TTS model to **`ext-tts`**. See the
2026-08-17 correction above.

### ⚠️ CORRECTION — my earlier voice foot-gun warning was wrong

This file previously warned that any non-fleet OpenAI voice name 404s and could trip
the router cooldown, and told you to pin the voice. **That was stale and mostly
unfounded.** tts-dev verified live: **the OpenAI names are ALIASED, not rejected** —
designed for exactly this case, a stock client dropping in without knowing fleet
voice names.

```
alloy, echo, onyx, ash  -> donut      nova            -> miranda
shimmer, coral          -> emmie      fable, sage     -> glados
```

Authoritative list (13): `computer`, `computer-soft`, `computer-urgent`, `donut`,
`emmie`, `glados`, `miranda`, `sindra`, `sindra-excited`, `sindra-sad`,
`sindra-soft`, `sindra-sultry`, `sindra-whisper`.

**Voice surface is now fully safe (tts-dev, 2026-08-16, tts-stack `c55bc3c`).**
`ballad`→emmie and `verse`→donut were the last two unaliased OpenAI names; they are
now aliased and live. Verified: the entire modern OpenAI voice set returns 200, and
only a genuinely-unknown string (`wharrgarbl`) 404s. **A stock Lobe voice picker
cannot produce a 404 from any voice it would plausibly offer, so it cannot trip the
router cooldown.** The env-vs-UI voice question is therefore moot for safety — the
voice string does not need pinning. (Format is a different matter; see below — and
it is not pinnable from here at all.)

### The real constraint is SIZE, not length

The 71.2s per-call cap in older notes is **dead** — that was Zonos-era and required
client-side chunking. dots chunks server-side. tts-dev threw a 592-word single call
at it: HTTP 200, 66.4 s wall, **245 s of audio**, byte-identical ending under ASR
diff. Long assistant turns are handled, not clipped.

But: **245 s of WAV is 23.5 MB**, and WAV is what this deploy gets — Lobe never sends
`response_format`, so the fleet gateway's default (native lane → engine → wav) is what
lands. **That is not fixable from this stack**: no env var, no UI field, not in the
payload. It is a fleet-gateway default, and tts-dev owns it. Until it changes, expect
a few MB per spoken turn on the LAN — wasteful, not fatal.

And 66 s of wall time is a long wait with no streaming in the OpenAI-compat path —
dots *does* have a genuine streaming mode (`stream: true`, ~0.40 s to first audio,
flat with length) but that is **not** the OpenAI-compat shape and would be a custom
integration. Lobe sends no `speed` either, so the gateway's atempo path is unreachable
from here.

### ⚠️ Capacity — tell tts-dev if this ramps

**The seat serializes generation — one render at a time, by design, no continuous
batching.** A second concurrent consumer is a real capacity question, not a
theoretical one, and dots shares a GPU where tts-dev had two resource incidents that
week. Report sustained volume, and any voice string not on the list above, to
`tts-dev` on althing rather than letting him infer it from a VRAM graph.

## Credential posture

> ### ⚠️ CHANGED 2026-08-21 — this key is NO LONGER free-local-only
>
> On the operator's explicit instruction, infra-ops replaced the key's explicit
> model array with the **`all-proxy-models` access group**. `lobe-chat-esh` now
> reaches **everything behind the gateway, paid passthroughs included** —
> `glm-*`, `kimi-k3`, `gen-frontier*`. The scoping described below is history,
> kept because the reasoning still explains what the guard rail was for.
>
> **What this means in practice:** a paid call through this key spends real
> vendor credits, and the key has **`max_budget: None`** — no ceiling. The UI
> in front of it is LAN-exposed with an access code and nothing else. A spend
> cap is available on request from infra-ops; the operator has not asked for
> one, so it is uncapped by instruction, not by oversight.
>
> Two side-effects of the access group worth knowing: it **auto-includes future
> models**, so no more per-seat key edits, and it only ever resolves **live**
> models — which is how five retired names (`char-rp-fable`,
> `char-rp-reasoning`, `lfm2.5-2.6b`, `qwen3-reranker`,
> `reranker-a4-gte-modernbert`) dropped off it silently.
>
> **The picker is now the only gate.** `OPENAI_MODEL_LIST` decides what a user
> can select; the key no longer restricts anything. As of 2026-08-21 that list
> deliberately excludes the paid family — so exposing paid models to this UI is
> one env edit away, and should stay a conscious decision rather than a drift.

The original posture, for the record:

Deliberately **not** the shared all-agents key — that reaches the paid passthroughs
(GLM, Kimi), and a LAN-exposed chat UI holding it would let anyone who can reach the
port spend vendor credits from a pool shared across every project.

This stack used a purpose-minted LiteLLM virtual key, `key_alias: lobe-chat-esh`,
scoped to the 20 free **local** models. Scoping was verified at mint time, both
directions:

- `gen` → answers
- `glm-5.2`, `kimi-k3`, `gen-frontier` → `key not allowed to access model`

Secrets are in the vault, never in git. `.env` on the host is `0600`:

```bash
secret get esh-docker-vm/lobe-chat-litellm-key        # -> OPENAI_API_KEY
secret get esh-docker-vm/lobe-chat-access-code        # -> ACCESS_CODE (UI gate)
secret get esh-docker-vm/lobe-chat-key-vaults-secret  # -> KEY_VAULTS_SECRET
```

`ACCESS_CODE` matters: this is a home-lab LAN segment with nothing in front of it.

## Verified on deploy (2026-08-16)

- container healthy; `http://10.0.50.45:3210/` → 307 → `/chat` → 200
- from **inside** the container: `GET /v1/models` returns the fleet seats, and a
  `gen` chat round-trip returns `"ok"` — so the app's own network path and key work,
  not merely the host's
- image on disk 617 MB (143 MB compressed)

## Deploy

```bash
scripts/deploy-stack.sh esh-docker-vm lobe-chat --compose
# on host: populate .env from the vault (see above), chmod 600, then
ssh lkraven@10.0.50.45 'cd /opt/docker/compose/lobe-chat && docker compose up -d'
```

`lkraven` owns `/opt/docker` and is in the `docker` group on this host, so no sudo
is needed. Note ESH is outside the infra-ops NOPASSWD grant.

## System Agent — why `gpt-5-mini` was being called

Lobe has a **System Agent**: a background model, separate from your chat model,
that auto-names conversations, summarizes history, rewrites RAG queries, and
generates assistant metadata. **Its default is `openai/gpt-5-mini`.** Because our
OpenAI provider is the gateway, that model name is forwarded verbatim to LiteLLM,
and the scoped key (local models only) 403s it — filling the gateway log with
"Tried to access gpt-5-mini" and silently breaking auto-naming.

Fixed by env (`SYSTEM_AGENT`), so this is scriptable, not a UI setting — a point
IN Lobe's favour on the manageable-by-agent axis, unlike the output-token cap:

```
SYSTEM_AGENT=topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen,queryRewrite=openai/summarizer,historyCompress=openai/summarizer,thread=openai/summarizer
```

All six documented keys are set explicitly — any omitted key falls back to the
`gpt-5-mini` default. `summarizer`/`classifier` are the same seat as `gen` at
temp 0, the right fit for naming/summarizing; `gen` where output quality matters.
