diff --git a/stacks/lobe-chat/README.md b/stacks/lobe-chat/README.md index 0ad57d3..69322fd 100644 --- a/stacks/lobe-chat/README.md +++ b/stacks/lobe-chat/README.md @@ -29,22 +29,46 @@ not docs: - **There is a server-side route** `(backend)/webapi/tts/openai/route.js` — TTS goes browser → Lobe server → the **OpenAI provider**, whose server-side base URL is - `OPENAI_PROXY_URL`. So the TTS *endpoint* inherits the gateway and reaches `ext-tts` - with no extra config. (Probing the route bare returns 401 — it exists and wants the - browser's provider payload — not 404.) + `OPENAI_PROXY_URL`. So the TTS *endpoint* inherits the gateway. (Probing the route + bare returns 401 — it exists and wants the browser's provider payload — not 404.) - **No `process.env.*TTS*` / `*AUDIO*` / `*SPEECH*` vars exist at all.** Voice, model, - `response_format`, and the enable-toggle live in a **client-side settings store** + and the enable-toggle live in a **client-side settings store** (bundle key `TTS_SETTING_KEY = 'tts'`), configured in Settings, per browser. There is no `AUDIO_TTS_*` equivalent to Open WebUI's. **Verdict against the operator's "manageable OR scriptable" criterion:** the -load-bearing part (endpoint → gateway → ext-tts) IS env-scriptable and already wired; -voice/model/format are a **one-time UI setup per browser**, not a maintenance surface. -That is "manageable", which was acceptable. Harmless-by-now footnotes: a voice -mis-click can't 404 (tts-dev aliased the full OpenAI set), and an unknown `model` -routes to the gateway default. The ONE thing not forceable by env is -`response_format: mp3` — if Lobe defaults to WAV you get 23.5 MB/turn; verify at first -use and set it in the UI. +load-bearing part (endpoint → gateway) IS env-scriptable and already wired; voice and +model are a **one-time UI setup per browser**, not a maintenance surface. That is +"manageable", which was acceptable. + +> ### ⚠ CORRECTION 2026-08-17 (tts-dev) — this section's two footnotes were wrong, +> and one of them meant TTS did not work at all +> +> Both claims below were reasoned from the wrong hop. Re-measured end-to-end from +> esh-docker-vm with this stack's own `.env` key: +> +> 1. **"An unknown `model` routes to the gateway default" — false on this path.** +> True direct-to-`:8198`, which accepts-and-ignores `model`. But requests go +> through **LiteLLM**, which resolves the model name *first*. Lobe's default is +> `tts-1`, and `POST $OPENAI_PROXY_URL/audio/speech {"model":"tts-1",...}` returns +> **403 `key not allowed to access model`** (the scoped key's allow-list holds +> `ext-tts` alone; on an unscoped key it is a 400 `Invalid model name`). The same +> call with `"model":"ext-tts"` returns 200 + audio. **So TTS was dead on arrival +> from deploy until the model field was set** — the endpoint being wired is +> necessary, not sufficient. +> 2. **"`response_format: mp3` … set it in the UI" — not possible.** Lobe's OpenAI +> TTS client sends exactly three fields (server bundle `chunks/29685.js`): +> `{ input, model: options?.model || 'tts-1', voice }`. `response_format` is never +> sent and has no UI setting, so **format cannot be chosen from this side at all.** +> You get the fleet gateway's default, which today is WAV (~123 KB for three words; +> 23.5 MB for a 245 s turn), and LiteLLM labels it `content-type: audio/mpeg` +> regardless of the bytes. Chrome sniffs and plays it. Fixing the size is +> tts-dev's side of the fence, not a setting here. +> +> **To make TTS work in a browser today:** Settings → TTS → *OpenAI TTS model* → +> `ext-tts`. Per browser, because the settings store is client-side. Removing that +> manual step means aliasing `tts-1` → `ext-tts` in LiteLLM **and** adding it to the +> `lobe-chat-esh` key's allow-list — both master-key operations (infra-ops). Trade recorded for the record: Open WebUI has dedicated `AUDIO_TTS_*` env vars (fully scriptable) at 1,825 MB; Lobe is endpoint-env + UI-cosmetic at 143 MB. Operator chose @@ -62,10 +86,15 @@ eating every engine change. **Shape:** plain OpenAI `POST /v1/audio/speech` with `{model, input, voice, response_format}`. No deviations; a stock OpenAI client works -drop-in. Unknown `model` values route to default rather than 404ing, so `tts-1` is -harmless. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. `speed` +drop-in. mp3/opus/aac/flac transcode; wav/pcm pass through byte-verbatim. `speed` honoured 0.25–4.0. +⚠️ **`model` is the one field that is NOT free-form on this route.** That tolerance +belongs to `:8198`, which accepts-and-ignores `model`; **LiteLLM in front of it does +not** — it resolves the name first, so `tts-1` never reaches the gateway (403 on the +scoped key, 400 on an unscoped one). Set Lobe's TTS model to **`ext-tts`**. See the +2026-08-17 correction above. + ### ⚠️ CORRECTION — my earlier voice foot-gun warning was wrong This file previously warned that any non-fleet OpenAI voice name 404s and could trip @@ -88,8 +117,9 @@ Authoritative list (13): `computer`, `computer-soft`, `computer-urgent`, `donut` now aliased and live. Verified: the entire modern OpenAI voice set returns 200, and only a genuinely-unknown string (`wharrgarbl`) 404s. **A stock Lobe voice picker cannot produce a 404 from any voice it would plausibly offer, so it cannot trip the -router cooldown.** The env-vs-UI voice question is therefore moot for safety — pin -`mp3` for size, but the voice string itself no longer needs pinning. +router cooldown.** The env-vs-UI voice question is therefore moot for safety — the +voice string does not need pinning. (Format is a different matter; see below — and +it is not pinnable from here at all.) ### The real constraint is SIZE, not length @@ -98,11 +128,17 @@ client-side chunking. dots chunks server-side. tts-dev threw a 592-word single c at it: HTTP 200, 66.4 s wall, **245 s of audio**, byte-identical ending under ASR diff. Long assistant turns are handled, not clipped. -But: **245 s of WAV is 23.5 MB.** For a browser UI set `response_format: "mp3"` or -you will push tens of megabytes per turn at the user. And 66 s of wall time is a long -wait with no streaming in the OpenAI-compat path — dots *does* have a genuine -streaming mode (`stream: true`, ~0.40 s to first audio, flat with length) but that is -**not** the OpenAI-compat shape and would be a custom integration. +But: **245 s of WAV is 23.5 MB**, and WAV is what this deploy gets — Lobe never sends +`response_format`, so the fleet gateway's default (native lane → engine → wav) is what +lands. **That is not fixable from this stack**: no env var, no UI field, not in the +payload. It is a fleet-gateway default, and tts-dev owns it. Until it changes, expect +a few MB per spoken turn on the LAN — wasteful, not fatal. + +And 66 s of wall time is a long wait with no streaming in the OpenAI-compat path — +dots *does* have a genuine streaming mode (`stream: true`, ~0.40 s to first audio, +flat with length) but that is **not** the OpenAI-compat shape and would be a custom +integration. Lobe sends no `speed` either, so the gateway's atempo path is unreachable +from here. ### ⚠️ Capacity — tell tts-dev if this ramps diff --git a/stacks/lobe-chat/compose.yaml b/stacks/lobe-chat/compose.yaml index 325b1d3..a738571 100644 --- a/stacks/lobe-chat/compose.yaml +++ b/stacks/lobe-chat/compose.yaml @@ -34,10 +34,19 @@ # CORRECTION to an earlier note in this file: the OpenAI voice names are # ALIASED, not rejected (echo/alloy/onyx/ash->donut, nova->miranda, # shimmer/coral->emmie, fable/sage->glados), so a UI mis-click is NOT the -# hazard I first recorded. Only `ballad` and `verse` 404. -# The real constraint is SIZE: 245s of WAV is 23.5 MB, so a browser UI should -# request `response_format: "mp3"`. The old 71.2s per-call cap is dead (that -# was Zonos-era); dots chunks server-side and a 592-word call renders intact. +# hazard I first recorded. Only `ballad` and `verse` 404 -- both aliased since. +# ⚠️ REQUIRED UI STEP (tts-dev 2026-08-17): Settings -> TTS -> OpenAI TTS model +# -> `ext-tts`. There is no env var for it. Lobe's default is `tts-1`, and while +# :8198 ignores `model`, LITELLM IN FRONT OF IT DOES NOT -- it resolves the name +# first, so `tts-1` returns 403 (`key not allowed to access model`; 400 on an +# unscoped key) and TTS does nothing. Verified from this host with this .env. +# Per browser -- the TTS settings store is client-side. +# SIZE: Lobe sends only {input, model, voice} -- no `response_format`, no +# `speed`, and no UI field for either -- so you get the gateway's default (WAV, +# ~23.5 MB for a 245 s turn), relabelled `audio/mpeg` by LiteLLM. Not fixable +# from this stack: it is a fleet-gateway default and tts-dev owns it. +# The old 71.2s per-call cap is dead (that was Zonos-era); dots chunks +# server-side and a 592-word call renders intact. # ⚠️ The seat SERIALIZES generation -- one render at a time, no continuous # batching -- so sustained volume from here is a real capacity question for a # shared GPU. Report ramp to tts-dev on althing.