vh
303fb7a5aa
Two independent gates kept the new `sec` family out of Lobe, and only one of
them was visible from the symptom.
The picker never auto-discovers. `OPENAI_MODEL_LIST=-all,+<names>` clears
Lobe's built-in OpenAI catalogue and re-adds one model per `+name`, so anything
added to LiteLLM stays invisible until this list is edited and the container
bounced. That pin is deliberate — an unpinned picker offers models that fail on
click — but it means the list rots in both directions, and it had:
- `sec` / `sec-reasoning` missing (hosted_vllm/mog-sec-27b{,-thinking} on
ana-ml2:8019, added to config.yaml earlier today), and
- `char-rp-reasoning` / `char-rp-fable` still listed after being retired
upstream, i.e. two picker entries that 400 on click. Verified: a call to
char-rp-fable now returns 400 Bad Request.
The list is now curated to live, chat-capable, free-local seats — eleven, each
round-tripped through the container after the bounce. The paid family stays out
deliberately; that is now a picker decision rather than a key one.
Which is the other half of this commit: the `lobe-chat-esh` key is no longer
scoped to free local models. On the operator's instruction infra-ops swapped its
explicit array for the `all-proxy-models` access group, so it now reaches the
paid passthroughs with `max_budget: None`. The README documented the old posture
as current, which made it a security claim that was no longer true; it now
carries the change, what it costs, and the fact that the picker is the only
remaining gate.
115 lines
6.6 KiB
YAML
115 lines
6.6 KiB
YAML
# lobe-chat — evaluation chat frontend over the LiteLLM gateway, on esh-docker-vm.
|
|
#
|
|
# WHY THIS EXISTS
|
|
# `gateway-chat` is a hand-rolled single-file HTML surface the operator does not
|
|
# want to keep improving; it has already cost two defects (a 1024 max_tokens
|
|
# default that read as model degeneracy, and a NaN->null max_tokens bug).
|
|
# Lobe was chosen over Open WebUI on weight, measured from the registries
|
|
# rather than from marketing: 143 MB compressed / 1 layer, against Open WebUI's
|
|
# 1,825 MB / 19 layers. 12.8x.
|
|
#
|
|
# OPEN QUESTION THIS DEPLOY EXISTS TO ANSWER
|
|
# Whether Lobe's TTS base URL / model / voice are settable by ENV or only via
|
|
# the settings UI. Open WebUI has dedicated AUDIO_TTS_* vars; Lobe documents a
|
|
# SHARED OPENAI_PROXY_URL, which should work because LiteLLM serves both
|
|
# /v1/chat/completions and /v1/audio/speech on one base -- but "should" is not
|
|
# "does". If it turns out UI-only it fails the operator's manageable-by-agent
|
|
# criterion even while winning on weight.
|
|
#
|
|
# ⚠️ CREDENTIAL POSTURE — deliberately NOT the shared all-agents key.
|
|
# That key reaches the PAID passthroughs (GLM, Kimi) and a LAN-exposed chat UI
|
|
# holding it would let anyone who can reach the port spend vendor credits on a
|
|
# pool shared across every project. This stack uses a purpose-minted LiteLLM
|
|
# virtual key (`key_alias: lobe-chat-esh`) scoped to the 20 free LOCAL models.
|
|
# Verified at mint time: `gen` answers; `glm-5.2`, `kimi-k3` and `gen-frontier`
|
|
# all return "key not allowed to access model".
|
|
# Secrets live in the vault, never in git:
|
|
# secret get esh-docker-vm/lobe-chat-litellm-key
|
|
# secret get esh-docker-vm/lobe-chat-access-code
|
|
# secret get esh-docker-vm/lobe-chat-key-vaults-secret
|
|
#
|
|
# TTS (verified with tts-dev 2026-08-16): route via the `ext-tts` LiteLLM alias,
|
|
# never direct to the seat -- engines get swapped behind the gateway and
|
|
# direct-to-seat eats every change. Auth is the gateway key alone.
|
|
# CORRECTION to an earlier note in this file: the OpenAI voice names are
|
|
# ALIASED, not rejected (echo/alloy/onyx/ash->donut, nova->miranda,
|
|
# shimmer/coral->emmie, fable/sage->glados), so a UI mis-click is NOT the
|
|
# hazard I first recorded. Only `ballad` and `verse` 404 -- both aliased since.
|
|
# MODEL NAME -- no longer a manual step (2026-08-17). Lobe defaults to `tts-1`,
|
|
# and while :8198 ignores `model`, LITELLM IN FRONT OF IT DOES NOT -- it resolves
|
|
# the name first, so `tts-1` used to 403 (`key not allowed to access model`) and
|
|
# TTS did nothing from deploy until it was found. infra-ops then aliased `tts-1`,
|
|
# `tts-1-hd` and `gpt-4o-mini-tts` to ext-tts's upstream AND added them to the
|
|
# `lobe-chat-esh` key allow-list, so the stock payload works with ZERO settings.
|
|
# Verified from this host with this .env: {"input":...,"model":"tts-1",
|
|
# "voice":"alloy"} -> 200 audio/mpeg. Setting the UI field to `ext-tts` also
|
|
# works and does no harm.
|
|
# SIZE: Lobe sends only {input, model, voice} -- no `response_format`, no
|
|
# `speed`, and no UI field for either -- so you get the gateway's default (WAV,
|
|
# ~23.5 MB for a 245 s turn), relabelled `audio/mpeg` by LiteLLM. Not fixable
|
|
# from this stack: it is a fleet-gateway default and tts-dev owns it.
|
|
# The old 71.2s per-call cap is dead (that was Zonos-era); dots chunks
|
|
# server-side and a 592-word call renders intact.
|
|
# ⚠️ The seat SERIALIZES generation -- one render at a time, no continuous
|
|
# batching -- so sustained volume from here is a real capacity question for a
|
|
# shared GPU. Report ramp to tts-dev on althing.
|
|
|
|
name: lobe-chat
|
|
|
|
services:
|
|
lobe-chat:
|
|
image: lobehub/lobe-chat:${LOBE_TAG:-latest}
|
|
container_name: ${LOBE_CONTAINER:-lobe-chat}
|
|
restart: unless-stopped
|
|
ports:
|
|
- "${LOBE_PORT:-3210}:3210"
|
|
environment:
|
|
# Chat + TTS both resolve through the one LiteLLM base.
|
|
- OPENAI_API_KEY=${OPENAI_API_KEY:?set in .env, from the vault}
|
|
- OPENAI_PROXY_URL=${OPENAI_PROXY_URL:-http://10.250.50.70:4000/v1}
|
|
# Pin the visible model list so the picker shows fleet seats rather than
|
|
# Lobe's built-in OpenAI catalogue (which would 404 against the gateway).
|
|
# ⚠️ THIS NEVER AUTO-DISCOVERS. `-all` clears Lobe's catalogue and each
|
|
# `+name` re-adds ONE model, so a seat added to LiteLLM stays invisible
|
|
# here until this list is edited and the container bounced. That is the
|
|
# price of the pin, paid deliberately: an unpinned picker offers models
|
|
# that fail on click. Two consequences to remember when editing:
|
|
# - a RETIRED model left in this list becomes a visible entry that 400s
|
|
# (char-rp-reasoning + char-rp-fable did exactly that until 2026-08-21)
|
|
# - the list is curated to LIVE, CHAT-CAPABLE seats: no TTS, embedding or
|
|
# reranker models (they cannot chat), and no paid passthroughs.
|
|
- OPENAI_MODEL_LIST=${OPENAI_MODEL_LIST:--all,+gen,+gen-reasoning,+char-rp,+summarizer,+summarizer-large,+classifier,+image-judge,+coder-fast,+chat-judge,+sec,+sec-reasoning}
|
|
# SYSTEM AGENT — the background model Lobe uses for auto-naming
|
|
# conversations (topic), history summarization, translation, query
|
|
# rewrite, thread naming, and assistant-meta generation. ITS DEFAULT IS
|
|
# `openai/gpt-5-mini`, which our OpenAI provider (= the gateway) forwards
|
|
# verbatim; the scoped key does not grant gpt-5-mini, so every background
|
|
# task 403s and the log fills with "Tried to access gpt-5-mini". Repoint
|
|
# ALL keys at fleet models (any key left unset falls back to gpt-5-mini).
|
|
# summarizer/classifier are the same seat as gen at temp 0 -- the right
|
|
# fit for naming/summarizing; gen where output quality matters.
|
|
- SYSTEM_AGENT=${SYSTEM_AGENT:-topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen,queryRewrite=openai/summarizer,historyCompress=openai/summarizer,thread=openai/summarizer}
|
|
# LAN-exposed on a home-lab segment with no auth in front: gate it.
|
|
- ACCESS_CODE=${ACCESS_CODE:?set in .env, from the vault}
|
|
# Encrypts any provider keys a user stores in the UI.
|
|
- KEY_VAULTS_SECRET=${KEY_VAULTS_SECRET:?set in .env, from the vault}
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:3210/ >/dev/null 2>&1 || exit 1"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 60s
|
|
networks:
|
|
- tnet
|
|
labels:
|
|
- homepage.group=Apps
|
|
- homepage.name=Lobe Chat
|
|
- homepage.icon=mdi-chat-processing
|
|
- homepage.description=Chat frontend over the LiteLLM gateway (eval)
|
|
- homepage.href=http://10.0.50.45:${LOBE_PORT:-3210}
|
|
|
|
networks:
|
|
tnet:
|
|
name: traefik-net
|
|
external: true
|