vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(lobe-chat): correct the TTS notes — the deploy's TTS never worked

Browse Source 
Two claims in this stack's docs were reasoned from the wrong hop, and one of
them hid a dead feature since deploy. Re-measured from esh-docker-vm against
the live .env:

1. "An unknown `model` routes to the gateway default" — true of :8198, false of
   the path Lobe takes. LiteLLM resolves the model name first, so Lobe's default
   `tts-1` returns 403 (`key not allowed to access model`) and never reaches the
   gateway. `ext-tts` returns 200 + audio. The endpoint inheriting
   OPENAI_PROXY_URL is necessary but not sufficient: Settings -> TTS -> OpenAI
   TTS model -> `ext-tts` is a required one-time step per browser, and removing
   it needs a LiteLLM alias plus a key allow-list entry (both master-key, so
   infra-ops).

2. "`response_format: mp3` ... set it in the UI" — not possible. Lobe's OpenAI
   TTS client sends `{input, model, voice}` and nothing else (server bundle
   chunks/29685.js), so format is not selectable from this stack at any level.
   The deploy gets the fleet gateway's default (WAV, ~23.5 MB for a 245 s turn),
   relabelled `audio/mpeg` by LiteLLM. That is tts-dev's fence, not this one's.

Comment/doc only — no functional change, so the host copy needs no redeploy.
This commit is contained in:
vh
2026-08-17 21:28:59 -07:00
parent d1f4f1cb96
commit ca8c0a318e
2 changed files with 69 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/lobe-chat/README.md
+56 -20
View File
@@ -29,22 +29,46 @@ not docs:
- **There is a server-side route** `(backend)/webapi/tts/openai/route.js` — TTS goes
browser → Lobe server → the **OpenAI provider**, whose server-side base URL is
`OPENAI_PROXY_URL`. So the TTS *endpoint* inherits the gateway and reaches `ext-tts`
with no extra config. (Probing the route bare returns 401 — it exists and wants the
browser's provider payload — not 404.)
`OPENAI_PROXY_URL`. So the TTS *endpoint* inherits the gateway. (Probing the route
bare returns 401 — it exists and wants the browser's provider payload — not 404.)
- **No `process.env.*TTS*` / `*AUDIO*` / `*SPEECH*` vars exist at all.** Voice, model,
`response_format`, and the enable-toggle live in a **client-side settings store**
and the enable-toggle live in a **client-side settings store**
(bundle key `TTS_SETTING_KEY = 'tts'`), configured in Settings, per browser. There is
no `AUDIO_TTS_*` equivalent to Open WebUI's.
**Verdict against the operator's "manageable OR scriptable" criterion:** the
load-bearing part (endpoint → gateway → ext-tts) IS env-scriptable and already wired;
voice/model/format are a **one-time UI setup per browser**, not a maintenance surface.
That is "manageable", which was acceptable. Harmless-by-now footnotes: a voice
mis-click can't 404 (tts-dev aliased the full OpenAI set), and an unknown `model`
routes to the gateway default. The ONE thing not forceable by env is
`response_format: mp3` — if Lobe defaults to WAV you get 23.5 MB/turn; verify at first
use and set it in the UI.
load-bearing part (endpoint → gateway) IS env-scriptable and already wired; voice and
model are a **one-time UI setup per browser**, not a maintenance surface. That is
"manageable", which was acceptable.
> ### ⚠ CORRECTION 2026-08-17 (tts-dev) — this section's two footnotes were wrong,
> and one of them meant TTS did not work at all
>
> Both claims below were reasoned from the wrong hop. Re-measured end-to-end from
> esh-docker-vm with this stack's own `.env` key:
>
> 1. **"An unknown `model` routes to the gateway default" — false on this path.**
> True direct-to-`:8198`, which accepts-and-ignores `model`. But requests go
> through **LiteLLM**, which resolves the model name *first*. Lobe's default is
> `tts-1`, and `POST $OPENAI_PROXY_URL/audio/speech {"model":"tts-1",...}` returns
> **403 `key not allowed to access model`** (the scoped key's allow-list holds
> `ext-tts` alone; on an unscoped key it is a 400 `Invalid model name`). The same
> call with `"model":"ext-tts"` returns 200 + audio. **So TTS was dead on arrival
> from deploy until the model field was set** — the endpoint being wired is
> necessary, not sufficient.
> 2. **"`response_format: mp3` … set it in the UI" — not possible.** Lobe's OpenAI
> TTS client sends exactly three fields (server bundle `chunks/29685.js`):
> `{ input, model: options?.model || 'tts-1', voice }`. `response_format` is never
> sent and has no UI setting, so **format cannot be chosen from this side at all.**
> You get the fleet gateway's default, which today is WAV (~123 KB for three words;
> 23.5 MB for a 245 s turn), and LiteLLM labels it `content-type: audio/mpeg`
> regardless of the bytes. Chrome sniffs and plays it. Fixing the size is
> tts-dev's side of the fence, not a setting here.
>
> **To make TTS work in a browser today:** Settings → TTS → *OpenAI TTS model* →
> `ext-tts`. Per browser, because the settings store is client-side. Removing that
> manual step means aliasing `tts-1` → `ext-tts` in LiteLLM **and** adding it to the
> `lobe-chat-esh` key's allow-list — both master-key operations (infra-ops).
Trade recorded for the record: Open WebUI has dedicated `AUDIO_TTS_*` env vars (fully
scriptable) at 1,825 MB; Lobe is endpoint-env + UI-cosmetic at 143 MB. Operator chose
@@ -62,10 +86,15 @@ eating every engine change.
**Shape:** plain OpenAI `POST /v1/audio/speech` with
`{model, input, voice, response_format}`. No deviations; a stock OpenAI client works
drop-in. Unknown `model` values route to default rather than 404ing, so `tts-1` is
harmless. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. `speed`
drop-in. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. `speed`
honoured 0.25–4.0.
⚠️ **`model` is the one field that is NOT free-form on this route.** That tolerance
belongs to `:8198`, which accepts-and-ignores `model`; **LiteLLM in front of it does
not** — it resolves the name first, so `tts-1` never reaches the gateway (403 on the
scoped key, 400 on an unscoped one). Set Lobe's TTS model to **`ext-tts`**. See the
2026-08-17 correction above.
### ⚠️ CORRECTION — my earlier voice foot-gun warning was wrong
This file previously warned that any non-fleet OpenAI voice name 404s and could trip
@@ -88,8 +117,9 @@ Authoritative list (13): `computer`, `computer-soft`, `computer-urgent`, `donut`
now aliased and live. Verified: the entire modern OpenAI voice set returns 200, and
only a genuinely-unknown string (`wharrgarbl`) 404s. **A stock Lobe voice picker
cannot produce a 404 from any voice it would plausibly offer, so it cannot trip the
router cooldown.** The env-vs-UI voice question is therefore moot for safety — pin
`mp3` for size, but the voice string itself no longer needs pinning.
router cooldown.** The env-vs-UI voice question is therefore moot for safety — the
voice string does not need pinning. (Format is a different matter; see below — and
it is not pinnable from here at all.)
### The real constraint is SIZE, not length
@@ -98,11 +128,17 @@ client-side chunking. dots chunks server-side. tts-dev threw a 592-word single c
at it: HTTP 200, 66.4 s wall, **245 s of audio**, byte-identical ending under ASR
diff. Long assistant turns are handled, not clipped.
But: **245 s of WAV is 23.5 MB.** For a browser UI set `response_format: "mp3"` or
you will push tens of megabytes per turn at the user. And 66 s of wall time is a long
wait with no streaming in the OpenAI-compat path — dots *does* have a genuine
streaming mode (`stream: true`, ~0.40 s to first audio, flat with length) but that is
**not** the OpenAI-compat shape and would be a custom integration.
But: **245 s of WAV is 23.5 MB**, and WAV is what this deploy gets — Lobe never sends
`response_format`, so the fleet gateway's default (native lane → engine → wav) is what
lands. **That is not fixable from this stack**: no env var, no UI field, not in the
payload. It is a fleet-gateway default, and tts-dev owns it. Until it changes, expect
a few MB per spoken turn on the LAN — wasteful, not fatal.
And 66 s of wall time is a long wait with no streaming in the OpenAI-compat path —
dots *does* have a genuine streaming mode (`stream: true`, ~0.40 s to first audio,
flat with length) but that is **not** the OpenAI-compat shape and would be a custom
integration. Lobe sends no `speed` either, so the gateway's atempo path is unreachable
from here.
### ⚠️ Capacity — tell tts-dev if this ramps
stacks/lobe-chat/compose.yaml
+13 -4
View File
@@ -34,10 +34,19 @@
# CORRECTION to an earlier note in this file: the OpenAI voice names are
# ALIASED, not rejected (echo/alloy/onyx/ash->donut, nova->miranda,
# shimmer/coral->emmie, fable/sage->glados), so a UI mis-click is NOT the
# hazard I first recorded. Only `ballad` and `verse` 404.
# The real constraint is SIZE: 245s of WAV is 23.5 MB, so a browser UI should
# request `response_format: "mp3"`. The old 71.2s per-call cap is dead (that
# was Zonos-era); dots chunks server-side and a 592-word call renders intact.
# hazard I first recorded. Only `ballad` and `verse` 404 -- both aliased since.
# ⚠️ REQUIRED UI STEP (tts-dev 2026-08-17): Settings -> TTS -> OpenAI TTS model
# -> `ext-tts`. There is no env var for it. Lobe's default is `tts-1`, and while
# :8198 ignores `model`, LITELLM IN FRONT OF IT DOES NOT -- it resolves the name
# first, so `tts-1` returns 403 (`key not allowed to access model`; 400 on an
# unscoped key) and TTS does nothing. Verified from this host with this .env.
# Per browser -- the TTS settings store is client-side.
# SIZE: Lobe sends only {input, model, voice} -- no `response_format`, no
# `speed`, and no UI field for either -- so you get the gateway's default (WAV,
# ~23.5 MB for a 245 s turn), relabelled `audio/mpeg` by LiteLLM. Not fixable
# from this stack: it is a fleet-gateway default and tts-dev owns it.
# The old 71.2s per-call cap is dead (that was Zonos-era); dots chunks
# server-side and a 592-word call renders intact.
# ⚠️ The seat SERIALIZES generation -- one render at a time, no continuous
# batching -- so sustained volume from here is a real capacity question for a
# shared GPU. Report ramp to tts-dev on althing.