vh
39da1d4a97
The board mixed tools with endpoints. A vLLM seat whose href is a /docs page
sat in the same band as ComfyUI; the MQTT broker and the RustDesk relay, which
have no page at all, sat in Apps; and `Service Networking` was thirteen members
spanning three AdGuards, five Dockges, two Traefiks and four headless agents.
Every group is now one of two kinds and they never mix. TOOLS are expanded and
sit at the top of their tab. ENDPOINTS — an API, a broker, a background agent,
an href that is /docs or /ping or nothing — carry `initiallyCollapsed: true`
and sit at the bottom. Collapsed is not hidden: the eyebrow and its rule still
render, so the tab still says the thing exists and one click expands it.
A second rule fell out of the same pass and now shapes the group boundaries: a
group's members should all carry a widget or none should. A stat strip makes a
card ~50px taller, so one widget card in a row of plain ones opens a void under
the plain ones. That is why AdGuard and Traefik get their own groups rather
than sharing one with Dockge, and it is most of why the old Service Networking
band looked broken. AdGuard (ANA) was the last short card in its row and now
carries the same query/blocked/latency strip as its two siblings — one
infra-ops AdGuard login authenticates against all three instances, verified
against each; it lives in that stack's .env on the host and is vaulted.
The sixteen GPU-backed model seats were deliberately NOT relabelled.
`homepage.group` is read at container creation, so clearer names for
`AI - Inference` and friends would have cost a recreate on six vLLM seats, four
eval seats and four TTS engines — multi-minute model reloads on endpoints peers
reach through the gateway. Order plus `initiallyCollapsed` buys the same
separation for nothing, so those names stay as they are on purpose.
28 containers that ARE cheap to bounce were relabelled, across five hosts, via
rerunnable elway playbooks. Their label steps are gated on the old value still
being present, so a second run reports skipped rather than churning. Two verify
steps were wrong on first contact and are fixed with the reason recorded: the
traefik check raced its own recreate, and asserting a model seat is "running"
cannot answer "did I bounce it" when a seat may be legitimately stopped —
container age can, and now does.
The canonical stacks/ tree was synced to the deployed labels afterwards, so
intent and reality agree again on all fourteen tracked stacks.
Also documents the real nature of the post-recreate blank dashboard, which cost
~25 minutes here and an hour on 2026-08-19. `initialSettings":{}` in the served
HTML is the catch branch of the page's data loader, not a warm-up and not a
cache — and the error can vanish entirely, because the logger is assigned inside
the same try and the catch only logs if the logger exists. Ruled out by
measurement this time: all four API routes return 200 with correct content while
the page serves {}, and the previous known-good settings.yaml reproduces it
identically. The README now carries the one-command test and the next lead.
Before/after, all four tabs: http://10.100.10.50:8090/b/homepage-relayout/
133 lines
8.0 KiB
YAML
133 lines
8.0 KiB
YAML
# lobe-chat — evaluation chat frontend over the LiteLLM gateway, on esh-docker-vm.
|
|
#
|
|
# WHY THIS EXISTS
|
|
# `gateway-chat` is a hand-rolled single-file HTML surface the operator does not
|
|
# want to keep improving; it has already cost two defects (a 1024 max_tokens
|
|
# default that read as model degeneracy, and a NaN->null max_tokens bug).
|
|
# Lobe was chosen over Open WebUI on weight, measured from the registries
|
|
# rather than from marketing: 143 MB compressed / 1 layer, against Open WebUI's
|
|
# 1,825 MB / 19 layers. 12.8x.
|
|
#
|
|
# OPEN QUESTION THIS DEPLOY EXISTS TO ANSWER
|
|
# Whether Lobe's TTS base URL / model / voice are settable by ENV or only via
|
|
# the settings UI. Open WebUI has dedicated AUDIO_TTS_* vars; Lobe documents a
|
|
# SHARED OPENAI_PROXY_URL, which should work because LiteLLM serves both
|
|
# /v1/chat/completions and /v1/audio/speech on one base -- but "should" is not
|
|
# "does". If it turns out UI-only it fails the operator's manageable-by-agent
|
|
# criterion even while winning on weight.
|
|
#
|
|
# ⚠️ CREDENTIAL POSTURE — CHANGED 2026-08-21, read this before assuming.
|
|
# This stack uses a purpose-minted LiteLLM virtual key (`key_alias:
|
|
# lobe-chat-esh`), originally scoped to the free LOCAL models so a LAN-exposed
|
|
# chat UI could not spend vendor credits. THAT IS NO LONGER TRUE: on the
|
|
# operator's instruction the key was moved to the `all-proxy-models` access
|
|
# group, so it now reaches EVERYTHING behind the gateway, paid passthroughs
|
|
# (GLM, Kimi, gen-frontier) included, with `max_budget: None`. A paid call
|
|
# from this UI spends real vendor credits and nothing caps it. The UI is
|
|
# LAN-exposed behind ACCESS_CODE and nothing else. A ceiling is available from
|
|
# infra-ops on request; uncapped is the operator's call, not an oversight.
|
|
# The access group also auto-includes future models and only ever resolves
|
|
# LIVE ones, so the key no longer needs editing when a seat is added.
|
|
# Secrets live in the vault, never in git:
|
|
# secret get esh-docker-vm/lobe-chat-litellm-key
|
|
# secret get esh-docker-vm/lobe-chat-access-code
|
|
# secret get esh-docker-vm/lobe-chat-key-vaults-secret
|
|
#
|
|
# TTS (verified with tts-dev 2026-08-16): route via the `ext-tts` LiteLLM alias,
|
|
# never direct to the seat -- engines get swapped behind the gateway and
|
|
# direct-to-seat eats every change. Auth is the gateway key alone.
|
|
# CORRECTION to an earlier note in this file: the OpenAI voice names are
|
|
# ALIASED, not rejected (echo/alloy/onyx/ash->donut, nova->miranda,
|
|
# shimmer/coral->emmie, fable/sage->glados), so a UI mis-click is NOT the
|
|
# hazard I first recorded. Only `ballad` and `verse` 404 -- both aliased since.
|
|
# MODEL NAME -- no longer a manual step (2026-08-17). Lobe defaults to `tts-1`,
|
|
# and while :8198 ignores `model`, LITELLM IN FRONT OF IT DOES NOT -- it resolves
|
|
# the name first, so `tts-1` used to 403 (`key not allowed to access model`) and
|
|
# TTS did nothing from deploy until it was found. infra-ops then aliased `tts-1`,
|
|
# `tts-1-hd` and `gpt-4o-mini-tts` to ext-tts's upstream AND added them to the
|
|
# `lobe-chat-esh` key allow-list, so the stock payload works with ZERO settings.
|
|
# Verified from this host with this .env: {"input":...,"model":"tts-1",
|
|
# "voice":"alloy"} -> 200 audio/mpeg. Setting the UI field to `ext-tts` also
|
|
# works and does no harm.
|
|
# SIZE: Lobe sends only {input, model, voice} -- no `response_format`, no
|
|
# `speed`, and no UI field for either -- so you get the gateway's default (WAV,
|
|
# ~23.5 MB for a 245 s turn), relabelled `audio/mpeg` by LiteLLM. Not fixable
|
|
# from this stack: it is a fleet-gateway default and tts-dev owns it.
|
|
# The old 71.2s per-call cap is dead (that was Zonos-era); dots chunks
|
|
# server-side and a 592-word call renders intact.
|
|
# ⚠️ The seat SERIALIZES generation -- one render at a time, no continuous
|
|
# batching -- so sustained volume from here is a real capacity question for a
|
|
# shared GPU. Report ramp to tts-dev on althing.
|
|
|
|
name: lobe-chat
|
|
|
|
services:
|
|
lobe-chat:
|
|
image: lobehub/lobe-chat:${LOBE_TAG:-latest}
|
|
container_name: ${LOBE_CONTAINER:-lobe-chat}
|
|
restart: unless-stopped
|
|
ports:
|
|
- "${LOBE_PORT:-3210}:3210"
|
|
environment:
|
|
# Chat + TTS both resolve through the one LiteLLM base.
|
|
- OPENAI_API_KEY=${OPENAI_API_KEY:?set in .env, from the vault}
|
|
- OPENAI_PROXY_URL=${OPENAI_PROXY_URL:-http://10.250.50.70:4000/v1}
|
|
# Pin the visible model list so the picker shows fleet seats rather than
|
|
# Lobe's built-in OpenAI catalogue (which would 404 against the gateway).
|
|
# ⚠️ THIS NEVER AUTO-DISCOVERS. `-all` clears Lobe's catalogue and each
|
|
# `+name` re-adds ONE model, so a seat added to LiteLLM stays invisible
|
|
# here until this list is edited and the container bounced. That is the
|
|
# price of the pin, paid deliberately: an unpinned picker offers Lobe's
|
|
# built-in OpenAI catalogue, all of which fails against the gateway.
|
|
#
|
|
# 2026-08-21 (operator: "pass everything through, lobe is a test surface
|
|
# for me"): the list is now EVERY chat-capable model the gateway serves,
|
|
# paid passthroughs included. Membership is decided by LiteLLM's declared
|
|
# `mode` from /v1/model_info, not by taste — chat + completion are in;
|
|
# audio_speech (ext-tts, tts-1, tts-1-hd, gpt-4o-mini-tts), embedding
|
|
# (qwen3-embedding) and rerank (reranker*) are out because they cannot
|
|
# answer a chat request at all. Re-derive the list from `mode` when
|
|
# editing rather than appending by hand.
|
|
#
|
|
# Distinguish the two failure shapes before removing a name:
|
|
# 400 = not configured in LiteLLM at all -> genuinely gone, drop it
|
|
# (char-rp-fable, 2026-08-21)
|
|
# 500 = configured but its upstream seat is down -> KEEP it, or the list
|
|
# rots every time a seat bounces (char-rp + char-rp-reasoning were
|
|
# 500ing on 2026-08-21 with ana-ml2:8016 refusing connections;
|
|
# seat under maintenance, both deliberately retained)
|
|
- OPENAI_MODEL_LIST=${OPENAI_MODEL_LIST:--all,+gen,+gen-reasoning,+sec,+sec-reasoning,+char-rp,+char-rp-reasoning,+summarizer,+summarizer-large,+classifier,+chat-judge,+selene-1-mini-8b,+image-judge,+qwen-image-bench,+coder-fast,+gen-frontier,+gen-frontier-reasoning,+glm-5.2,+glm-5.2-reasoning,+glm-5.1,+glm-5.1-reasoning,+glm-5-turbo,+glm-4.7,+glm-4.5-air,+kimi-k3,+kimi-k3-gen-api}
|
|
# SYSTEM AGENT — the background model Lobe uses for auto-naming
|
|
# conversations (topic), history summarization, translation, query
|
|
# rewrite, thread naming, and assistant-meta generation. ITS DEFAULT IS
|
|
# `openai/gpt-5-mini`, which our OpenAI provider (= the gateway) forwards
|
|
# verbatim; the scoped key does not grant gpt-5-mini, so every background
|
|
# task 403s and the log fills with "Tried to access gpt-5-mini". Repoint
|
|
# ALL keys at fleet models (any key left unset falls back to gpt-5-mini).
|
|
# summarizer/classifier are the same seat as gen at temp 0 -- the right
|
|
# fit for naming/summarizing; gen where output quality matters.
|
|
- SYSTEM_AGENT=${SYSTEM_AGENT:-topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen,queryRewrite=openai/summarizer,historyCompress=openai/summarizer,thread=openai/summarizer}
|
|
# LAN-exposed on a home-lab segment with no auth in front: gate it.
|
|
- ACCESS_CODE=${ACCESS_CODE:?set in .env, from the vault}
|
|
# Encrypts any provider keys a user stores in the UI.
|
|
- KEY_VAULTS_SECRET=${KEY_VAULTS_SECRET:?set in .env, from the vault}
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:3210/ >/dev/null 2>&1 || exit 1"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 60s
|
|
networks:
|
|
- tnet
|
|
labels:
|
|
- homepage.group=AI - Gateways & Chat
|
|
- homepage.name=Lobe Chat
|
|
- homepage.icon=mdi-chat-processing
|
|
- homepage.description=Chat frontend over the LiteLLM gateway (eval)
|
|
- homepage.href=http://10.0.50.45:${LOBE_PORT:-3210}
|
|
|
|
networks:
|
|
tnet:
|
|
name: traefik-net
|
|
external: true
|