From 3d30a6530ba9c600d1f7326846dce48b120b747f Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 21 Aug 2026 08:11:17 -0700 Subject: [PATCH] feat(lobe-chat): pass every chat-capable model through the picker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator: "pass everything through, lobe is a test surface for me." The picker goes from 11 curated seats to all 25 chat-capable models the gateway serves, paid passthroughs included. Membership is derived from LiteLLM's declared `mode` rather than by taste: chat (15) and completion (1) are in, and the nine GLM/frontier entries whose mode is unset are chat models in practice. Out: audio_speech (ext-tts, tts-1, tts-1-hd, gpt-4o-mini-tts), embedding (qwen3-embedding) and rerank (reranker, reranker-a3-bge-v2-m3) — seven models that cannot answer a chat request at all, so listing them would recreate the visible-but-dead entry this list just got fixed for. Every candidate was probed live from inside the container before being pinned, which corrected an earlier claim in this file. char-rp and char-rp-reasoning are NOT retired: they are configured and their upstream seat (ana-ml2:8016) is refusing connections while under maintenance, which is a 500, not a 400. Both are retained deliberately — dropping a name because its seat is bouncing makes the list rot in the other direction. Only char-rp-fable is genuinely gone (400, absent from model_info). The comment now spells out the 400-vs-500 distinction so the next editor does not repeat the mistake. Also corrects the credential-posture header, which still described the key as scoped to free local models and cited the glm/kimi rejection as current verification. The key now carries the all-proxy-models access group with no budget ceiling, so that block had become a false security claim in the most load-bearing comment in the file. --- stacks/lobe-chat/.env.example | 2 +- stacks/lobe-chat/compose.yaml | 46 ++++++++++++++++++++++++----------- 2 files changed, 33 insertions(+), 15 deletions(-) diff --git a/stacks/lobe-chat/.env.example b/stacks/lobe-chat/.env.example index 9089c66..9e1ec71 100644 --- a/stacks/lobe-chat/.env.example +++ b/stacks/lobe-chat/.env.example @@ -15,4 +15,4 @@ SYSTEM_AGENT=topic=openai/summarizer,translation=openai/gen,agentMeta=openai/gen # invisible until it is listed here AND the container is bounced. Keep it to # LIVE, CHAT-CAPABLE models: a retired name left here becomes a picker entry # that 400s on click, and TTS/embedding/reranker seats cannot chat at all. -OPENAI_MODEL_LIST=-all,+gen,+gen-reasoning,+char-rp,+summarizer,+summarizer-large,+classifier,+image-judge,+coder-fast,+chat-judge,+sec,+sec-reasoning +OPENAI_MODEL_LIST=-all,+gen,+gen-reasoning,+sec,+sec-reasoning,+char-rp,+char-rp-reasoning,+summarizer,+summarizer-large,+classifier,+chat-judge,+selene-1-mini-8b,+image-judge,+qwen-image-bench,+coder-fast,+gen-frontier,+gen-frontier-reasoning,+glm-5.2,+glm-5.2-reasoning,+glm-5.1,+glm-5.1-reasoning,+glm-5-turbo,+glm-4.7,+glm-4.5-air,+kimi-k3,+kimi-k3-gen-api diff --git a/stacks/lobe-chat/compose.yaml b/stacks/lobe-chat/compose.yaml index e039ad4..df62b0e 100644 --- a/stacks/lobe-chat/compose.yaml +++ b/stacks/lobe-chat/compose.yaml @@ -16,13 +16,18 @@ # "does". If it turns out UI-only it fails the operator's manageable-by-agent # criterion even while winning on weight. # -# ⚠️ CREDENTIAL POSTURE — deliberately NOT the shared all-agents key. -# That key reaches the PAID passthroughs (GLM, Kimi) and a LAN-exposed chat UI -# holding it would let anyone who can reach the port spend vendor credits on a -# pool shared across every project. This stack uses a purpose-minted LiteLLM -# virtual key (`key_alias: lobe-chat-esh`) scoped to the 20 free LOCAL models. -# Verified at mint time: `gen` answers; `glm-5.2`, `kimi-k3` and `gen-frontier` -# all return "key not allowed to access model". +# ⚠️ CREDENTIAL POSTURE — CHANGED 2026-08-21, read this before assuming. +# This stack uses a purpose-minted LiteLLM virtual key (`key_alias: +# lobe-chat-esh`), originally scoped to the free LOCAL models so a LAN-exposed +# chat UI could not spend vendor credits. THAT IS NO LONGER TRUE: on the +# operator's instruction the key was moved to the `all-proxy-models` access +# group, so it now reaches EVERYTHING behind the gateway, paid passthroughs +# (GLM, Kimi, gen-frontier) included, with `max_budget: None`. A paid call +# from this UI spends real vendor credits and nothing caps it. The UI is +# LAN-exposed behind ACCESS_CODE and nothing else. A ceiling is available from +# infra-ops on request; uncapped is the operator's call, not an oversight. +# The access group also auto-includes future models and only ever resolves +# LIVE ones, so the key no longer needs editing when a seat is added. # Secrets live in the vault, never in git: # secret get esh-docker-vm/lobe-chat-litellm-key # secret get esh-docker-vm/lobe-chat-access-code @@ -72,13 +77,26 @@ services: # ⚠️ THIS NEVER AUTO-DISCOVERS. `-all` clears Lobe's catalogue and each # `+name` re-adds ONE model, so a seat added to LiteLLM stays invisible # here until this list is edited and the container bounced. That is the - # price of the pin, paid deliberately: an unpinned picker offers models - # that fail on click. Two consequences to remember when editing: - # - a RETIRED model left in this list becomes a visible entry that 400s - # (char-rp-reasoning + char-rp-fable did exactly that until 2026-08-21) - # - the list is curated to LIVE, CHAT-CAPABLE seats: no TTS, embedding or - # reranker models (they cannot chat), and no paid passthroughs. - - OPENAI_MODEL_LIST=${OPENAI_MODEL_LIST:--all,+gen,+gen-reasoning,+char-rp,+summarizer,+summarizer-large,+classifier,+image-judge,+coder-fast,+chat-judge,+sec,+sec-reasoning} + # price of the pin, paid deliberately: an unpinned picker offers Lobe's + # built-in OpenAI catalogue, all of which fails against the gateway. + # + # 2026-08-21 (operator: "pass everything through, lobe is a test surface + # for me"): the list is now EVERY chat-capable model the gateway serves, + # paid passthroughs included. Membership is decided by LiteLLM's declared + # `mode` from /v1/model_info, not by taste — chat + completion are in; + # audio_speech (ext-tts, tts-1, tts-1-hd, gpt-4o-mini-tts), embedding + # (qwen3-embedding) and rerank (reranker*) are out because they cannot + # answer a chat request at all. Re-derive the list from `mode` when + # editing rather than appending by hand. + # + # Distinguish the two failure shapes before removing a name: + # 400 = not configured in LiteLLM at all -> genuinely gone, drop it + # (char-rp-fable, 2026-08-21) + # 500 = configured but its upstream seat is down -> KEEP it, or the list + # rots every time a seat bounces (char-rp + char-rp-reasoning were + # 500ing on 2026-08-21 with ana-ml2:8016 refusing connections; + # seat under maintenance, both deliberately retained) + - OPENAI_MODEL_LIST=${OPENAI_MODEL_LIST:--all,+gen,+gen-reasoning,+sec,+sec-reasoning,+char-rp,+char-rp-reasoning,+summarizer,+summarizer-large,+classifier,+chat-judge,+selene-1-mini-8b,+image-judge,+qwen-image-bench,+coder-fast,+gen-frontier,+gen-frontier-reasoning,+glm-5.2,+glm-5.2-reasoning,+glm-5.1,+glm-5.1-reasoning,+glm-5-turbo,+glm-4.7,+glm-4.5-air,+kimi-k3,+kimi-k3-gen-api} # SYSTEM AGENT — the background model Lobe uses for auto-naming # conversations (topic), history summarization, translation, query # rewrite, thread naming, and assistant-meta generation. ITS DEFAULT IS