vh
3d30a6530b
Operator: "pass everything through, lobe is a test surface for me." The picker goes from 11 curated seats to all 25 chat-capable models the gateway serves, paid passthroughs included. Membership is derived from LiteLLM's declared `mode` rather than by taste: chat (15) and completion (1) are in, and the nine GLM/frontier entries whose mode is unset are chat models in practice. Out: audio_speech (ext-tts, tts-1, tts-1-hd, gpt-4o-mini-tts), embedding (qwen3-embedding) and rerank (reranker, reranker-a3-bge-v2-m3) — seven models that cannot answer a chat request at all, so listing them would recreate the visible-but-dead entry this list just got fixed for. Every candidate was probed live from inside the container before being pinned, which corrected an earlier claim in this file. char-rp and char-rp-reasoning are NOT retired: they are configured and their upstream seat (ana-ml2:8016) is refusing connections while under maintenance, which is a 500, not a 400. Both are retained deliberately — dropping a name because its seat is bouncing makes the list rot in the other direction. Only char-rp-fable is genuinely gone (400, absent from model_info). The comment now spells out the 400-vs-500 distinction so the next editor does not repeat the mistake. Also corrects the credential-posture header, which still described the key as scoped to free local models and cited the glm/kimi rejection as current verification. The key now carries the all-proxy-models access group with no budget ceiling, so that block had become a false security claim in the most load-bearing comment in the file.
133 lines
8.0 KiB
YAML
133 lines
8.0 KiB
YAML
# lobe-chat — evaluation chat frontend over the LiteLLM gateway, on esh-docker-vm.
|
|
#
|
|
# WHY THIS EXISTS
|
|
# `gateway-chat` is a hand-rolled single-file HTML surface the operator does not
|
|
# want to keep improving; it has already cost two defects (a 1024 max_tokens
|
|
# default that read as model degeneracy, and a NaN->null max_tokens bug).
|
|
# Lobe was chosen over Open WebUI on weight, measured from the registries
|
|
# rather than from marketing: 143 MB compressed / 1 layer, against Open WebUI's
|
|
# 1,825 MB / 19 layers. 12.8x.
|
|
#
|
|
# OPEN QUESTION THIS DEPLOY EXISTS TO ANSWER
|
|
# Whether Lobe's TTS base URL / model / voice are settable by ENV or only via
|
|
# the settings UI. Open WebUI has dedicated AUDIO_TTS_* vars; Lobe documents a
|
|
# SHARED OPENAI_PROXY_URL, which should work because LiteLLM serves both
|
|
# /v1/chat/completions and /v1/audio/speech on one base -- but "should" is not
|
|
# "does". If it turns out UI-only it fails the operator's manageable-by-agent
|
|
# criterion even while winning on weight.
|
|
#
|
|
# ⚠️ CREDENTIAL POSTURE — CHANGED 2026-08-21, read this before assuming.
|
|
# This stack uses a purpose-minted LiteLLM virtual key (`key_alias:
|
|
# lobe-chat-esh`), originally scoped to the free LOCAL models so a LAN-exposed
|
|
# chat UI could not spend vendor credits. THAT IS NO LONGER TRUE: on the
|
|
# operator's instruction the key was moved to the `all-proxy-models` access
|
|
# group, so it now reaches EVERYTHING behind the gateway, paid passthroughs
|
|
# (GLM, Kimi, gen-frontier) included, with `max_budget: None`. A paid call
|
|
# from this UI spends real vendor credits and nothing caps it. The UI is
|
|
# LAN-exposed behind ACCESS_CODE and nothing else. A ceiling is available from
|
|
# infra-ops on request; uncapped is the operator's call, not an oversight.
|
|
# The access group also auto-includes future models and only ever resolves
|
|
# LIVE ones, so the key no longer needs editing when a seat is added.
|
|
# Secrets live in the vault, never in git:
|
|
# secret get esh-docker-vm/lobe-chat-litellm-key
|
|
# secret get esh-docker-vm/lobe-chat-access-code
|
|
# secret get esh-docker-vm/lobe-chat-key-vaults-secret
|
|
#
|
|
# TTS (verified with tts-dev 2026-08-16): route via the `ext-tts` LiteLLM alias,
|
|
# never direct to the seat -- engines get swapped behind the gateway and
|
|
# direct-to-seat eats every change. Auth is the gateway key alone.
|
|
# CORRECTION to an earlier note in this file: the OpenAI voice names are
|
|
# ALIASED, not rejected (echo/alloy/onyx/ash->donut, nova->miranda,
|
|
# shimmer/coral->emmie, fable/sage->glados), so a UI mis-click is NOT the
|
|
# hazard I first recorded. Only `ballad` and `verse` 404 -- both aliased since.
|
|
# MODEL NAME -- no longer a manual step (2026-08-17). Lobe defaults to `tts-1`,
|
|
# and while :8198 ignores `model`, LITELLM IN FRONT OF IT DOES NOT -- it resolves
|
|
# the name first, so `tts-1` used to 403 (`key not allowed to access model`) and
|
|
# TTS did nothing from deploy until it was found. infra-ops then aliased `tts-1`,
|
|
# `tts-1-hd` and `gpt-4o-mini-tts` to ext-tts's upstream AND added them to the
|
|
# `lobe-chat-esh` key allow-list, so the stock payload works with ZERO settings.
|
|
# Verified from this host with this .env: {"input":...,"model":"tts-1",
|
|
# "voice":"alloy"} -> 200 audio/mpeg. Setting the UI field to `ext-tts` also
|
|
# works and does no harm.
|
|
# SIZE: Lobe sends only {input, model, voice} -- no `response_format`, no
|
|
# `speed`, and no UI field for either -- so you get the gateway's default (WAV,
|
|
# ~23.5 MB for a 245 s turn), relabelled `audio/mpeg` by LiteLLM. Not fixable
|
|
# from this stack: it is a fleet-gateway default and tts-dev owns it.
|
|
# The old 71.2s per-call cap is dead (that was Zonos-era); dots chunks
|
|
# server-side and a 592-word call renders intact.
|
|
# ⚠️ The seat SERIALIZES generation -- one render at a time, no continuous
|
|
# batching -- so sustained volume from here is a real capacity question for a
|
|
# shared GPU. Report ramp to tts-dev on althing.
|
|
|
|
name: lobe-chat
|
|
|
|
services:
|
|
lobe-chat:
|
|
image: lobehub/lobe-chat:${LOBE_TAG:-latest}
|
|
container_name: ${LOBE_CONTAINER:-lobe-chat}
|
|
restart: unless-stopped
|
|
ports:
|
|
- "${LOBE_PORT:-3210}:3210"
|
|
environment:
|
|
# Chat + TTS both resolve through the one LiteLLM base.
|
|
- OPENAI_API_KEY=${OPENAI_API_KEY:?set in .env, from the vault}
|
|
- OPENAI_PROXY_URL=${OPENAI_PROXY_URL:-http://10.250.50.70:4000/v1}
|
|
# Pin the visible model list so the picker shows fleet seats rather than
|
|
# Lobe's built-in OpenAI catalogue (which would 404 against the gateway).
|
|
# ⚠️ THIS NEVER AUTO-DISCOVERS. `-all` clears Lobe's catalogue and each
|
|
# `+name` re-adds ONE model, so a seat added to LiteLLM stays invisible
|
|
# here until this list is edited and the container bounced. That is the
|
|
# price of the pin, paid deliberately: an unpinned picker offers Lobe's
|
|
# built-in OpenAI catalogue, all of which fails against the gateway.
|
|
#
|
|
# 2026-08-21 (operator: "pass everything through, lobe is a test surface
|
|
# for me"): the list is now EVERY chat-capable model the gateway serves,
|
|
# paid passthroughs included. Membership is decided by LiteLLM's declared
|
|
# `mode` from /v1/model_info, not by taste — chat + completion are in;
|
|
# audio_speech (ext-tts, tts-1, tts-1-hd, gpt-4o-mini-tts), embedding
|
|
# (qwen3-embedding) and rerank (reranker*) are out because they cannot
|
|
# answer a chat request at all. Re-derive the list from `mode` when
|
|
# editing rather than appending by hand.
|
|
#
|
|
# Distinguish the two failure shapes before removing a name:
|
|
# 400 = not configured in LiteLLM at all -> genuinely gone, drop it
|
|
# (char-rp-fable, 2026-08-21)
|
|
# 500 = configured but its upstream seat is down -> KEEP it, or the list
|
|
# rots every time a seat bounces (char-rp + char-rp-reasoning were
|
|
# 500ing on 2026-08-21 with ana-ml2:8016 refusing connections;
|
|
# seat under maintenance, both deliberately retained)
|
|
- OPENAI_MODEL_LIST=${OPENAI_MODEL_LIST:--all,+gen,+gen-reasoning,+sec,+sec-reasoning,+char-rp,+char-rp-reasoning,+summarizer,+summarizer-large,+classifier,+chat-judge,+selene-1-mini-8b,+image-judge,+qwen-image-bench,+coder-fast,+gen-frontier,+gen-frontier-reasoning,+glm-5.2,+glm-5.2-reasoning,+glm-5.1,+glm-5.1-reasoning,+glm-5-turbo,+glm-4.7,+glm-4.5-air,+kimi-k3,+kimi-k3-gen-api}
|
|
# SYSTEM AGENT — the background model Lobe uses for auto-naming
|
|
# conversations (topic), history summarization, translation, query
|
|
# rewrite, thread naming, and assistant-meta generation. ITS DEFAULT IS
|
|
# `openai/gpt-5-mini`, which our OpenAI provider (= the gateway) forwards
|
|
# verbatim; the scoped key does not grant gpt-5-mini, so every background
|
|
# task 403s and the log fills with "Tried to access gpt-5-mini". Repoint
|
|
# ALL keys at fleet models (any key left unset falls back to gpt-5-mini).
|
|
# summarizer/classifier are the same seat as gen at temp 0 -- the right
|
|
# fit for naming/summarizing; gen where output quality matters.
|
|
- SYSTEM_AGENT=${SYSTEM_AGENT:-topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen,queryRewrite=openai/summarizer,historyCompress=openai/summarizer,thread=openai/summarizer}
|
|
# LAN-exposed on a home-lab segment with no auth in front: gate it.
|
|
- ACCESS_CODE=${ACCESS_CODE:?set in .env, from the vault}
|
|
# Encrypts any provider keys a user stores in the UI.
|
|
- KEY_VAULTS_SECRET=${KEY_VAULTS_SECRET:?set in .env, from the vault}
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:3210/ >/dev/null 2>&1 || exit 1"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 60s
|
|
networks:
|
|
- tnet
|
|
labels:
|
|
- homepage.group=Apps
|
|
- homepage.name=Lobe Chat
|
|
- homepage.icon=mdi-chat-processing
|
|
- homepage.description=Chat frontend over the LiteLLM gateway (eval)
|
|
- homepage.href=http://10.0.50.45:${LOBE_PORT:-3210}
|
|
|
|
networks:
|
|
tnet:
|
|
name: traefik-net
|
|
external: true
|