cosyvoice: verify /api/tts streaming accepts cloned voice_ids #1

New Issue

2026-04-24T09:55:49-07:00

vh commented

2026-04-24 09:55:49 -07:00

The current stacks/cosyvoice/README.md documents streaming synthesis via POST /api/tts with stream=true, reusing the same voice_id that comes from POST /v1/voices/create. This assumption is based on the OpenAPI schema (which shows a voice field on /api/tts) but was not empirically verified during the deploy session on 2026-04-23/24.

What we know

/v1/audio/speech with a cloned voice_id works (confirmed end-to-end with the glados voice).
/api/tts has a voice: string field and a stream: boolean [default=False] field per GET /openapi.json.
The native endpoint also has mode (zero_shot / instruct / sft), prompt_wav, prompt_text, and spk_id fields — the role of voice relative to those is undocumented upstream.
/api/speakers returns [] in this image (neosun/cosyvoice:v1.3.2), so the spk_id path is not usable.

Risk

If /api/tts treats the voice field differently from /v1/audio/speech (e.g., expects an spk_id rather than a cloned-voice id), the streaming curl in the README will fail with a 400/500 the first time someone tries it.

What to do

Run the streaming curl from the README against an existing cloned voice (e.g., glados). Capture:
- HTTP status code
- Whether chunks arrive as chunked-transfer or one blob
- Approximate TTFB (should be ~150 ms per upstream claim)
If it works, add a confirming note to the README noting the verified voice_id shape.
If it fails, switch the documented streaming recipe to the inline-reference-audio form (prompt_wav=@ref.wav + prompt_text=<transcript> + mode=zero_shot), which is documented in the field-differences table and is the safer fallback.

Context

Endpoint: http://10.100.79.3:8190 (WG-only, irv-ml1)
Stack source: stacks/cosyvoice/ in this repo
Image: neosun/cosyvoice:v1.3.2 (Fun-CosyVoice3-0.5B-2512)

The current stacks/cosyvoice/README.md documents streaming synthesis via `POST /api/tts` with `stream=true`, reusing the same `voice_id` that comes from `POST /v1/voices/create`. This assumption is based on the OpenAPI schema (which shows a `voice` field on `/api/tts`) but **was not empirically verified** during the deploy session on 2026-04-23/24. ### What we know - `/v1/audio/speech` with a cloned `voice_id` works (confirmed end-to-end with the `glados` voice). - `/api/tts` has a `voice: string` field and a `stream: boolean [default=False]` field per `GET /openapi.json`. - The native endpoint also has `mode` (`zero_shot` / `instruct` / `sft`), `prompt_wav`, `prompt_text`, and `spk_id` fields — the role of `voice` relative to those is undocumented upstream. - `/api/speakers` returns `[]` in this image (neosun/cosyvoice:v1.3.2), so the `spk_id` path is not usable. ### Risk If `/api/tts` treats the `voice` field differently from `/v1/audio/speech` (e.g., expects an `spk_id` rather than a cloned-voice id), the streaming curl in the README will fail with a 400/500 the first time someone tries it. ### What to do 1. Run the streaming curl from the README against an existing cloned voice (e.g., `glados`). Capture: - HTTP status code - Whether chunks arrive as chunked-transfer or one blob - Approximate TTFB (should be ~150 ms per upstream claim) 2. If it works, add a confirming note to the README noting the verified voice_id shape. 3. If it fails, switch the documented streaming recipe to the inline-reference-audio form (`prompt_wav=@ref.wav` + `prompt_text=<transcript>` + `mode=zero_shot`), which is documented in the field-differences table and is the safer fallback. ### Context - Endpoint: `http://10.100.79.3:8190` (WG-only, irv-ml1) - Stack source: `stacks/cosyvoice/` in this repo - Image: `neosun/cosyvoice:v1.3.2` (Fun-CosyVoice3-0.5B-2512)

vh added the investigation label 2026-04-24 09:57:40 -07:00

vh closed this issue

2026-04-24 16:39:30 -07:00

vh commented

2026-04-24 17:32:54 -07:00

Verified end-to-end against the live deploy on irv-ml1 (voice id 9053c5a52f80):

Endpoint	Status	Notes
POST /api/tts stream=true	200	Raw PCM chunks streamed (no RIFF header — intentional for streaming)
POST /api/tts stream=false	200	RIFF WAV, 16-bit mono 24kHz
POST /v1/audio/speech with XML ...	200	RIFF WAV
POST /v1/audio/speech with instruct: "..."	200	RIFF WAV

Cloned voice_ids from /v1/voices/create work uniformly across all four paths. The streaming recipe in the README's smoke test is correct as written.

Verified end-to-end against the live deploy on irv-ml1 (voice id `9053c5a52f80`): | Endpoint | Status | Notes | |---|---|---| | POST /api/tts stream=true | 200 | Raw PCM chunks streamed (no RIFF header — intentional for streaming) | | POST /api/tts stream=false | 200 | RIFF WAV, 16-bit mono 24kHz | | POST /v1/audio/speech with XML <angry>...</angry> | 200 | RIFF WAV | | POST /v1/audio/speech with instruct: "..." | 200 | RIFF WAV | Cloned voice_ids from /v1/voices/create work uniformly across all four paths. The streaming recipe in the README's smoke test is correct as written.

vh referenced this issue from a commit

2026-08-28 05:52:26 -07:00

memory: run 3c had TWO launches — the third was an untimestamped report

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: vh/esh-pfi-infrastructure#1