diff --git a/stacks/cosyvoice/README.md b/stacks/cosyvoice/README.md index 6661dff..a6940db 100644 --- a/stacks/cosyvoice/README.md +++ b/stacks/cosyvoice/README.md @@ -113,6 +113,20 @@ ssh irv-ml1 ' - **Model choice.** `cosyvoice-v3` is default and best; `cosyvoice-v2` is available as a fallback if you have voices tuned against the v2 model's idiosyncrasies. +- **The `instruct` field is Chinese-context-only — DON'T use it + for English emotion control.** Verified empirically 2026-04-24: + any value in `instruct` (English, Chinese, or empty hints inside + it) routes through CosyVoice's `inference_instruct2` which expects + a directive in the upstream Chinese template + (`You are a helpful assistant. 请用…<|endofprompt|>`). + English values like `instruct: "speak with cold contempt"` cause + the model to *vocalize the instruction text itself* — you'll hear + Chinese-flavored phonemes reading the English directive instead + of your `input`. Workaround: **use the XML inline tags** (``, + ``, ``, ``, ``, ``, + ``, ``, ``, ``) for English emotion + control. They're parsed as control tokens, not content. See + upstream issue [reference TBD]. ## Smoke test (full clone → synthesize) @@ -131,29 +145,31 @@ curl -X POST http://10.100.79.3:8190/v1/voices/create \ -F "text=" # → {"voice_id": "", "name": "glados", ...} -# 4. Synthesize via OpenAI-compat endpoint (single-shot) +# 4. Synthesize via OpenAI-compat endpoint (single-shot). +# For English emotion control, use the XML inline tag — NOT the +# `instruct` field (that's Chinese-context only; see Gotchas). curl -X POST http://10.100.79.3:8190/v1/audio/speech \ -H 'Content-Type: application/json' \ -d '{ "model": "cosyvoice-v3", "voice": "", - "input": "Hello. I did not ask for visitors today.", - "instruct": "speak with a flat, menacing calm", + "input": "I did not ask for visitors today.", "response_format": "wav", "speed": 1.0 }' \ -o out.wav # 5. Or synthesize via native endpoint with streaming — curl writes -# audio chunks to the file as they arrive (~150 ms TTFB) +# audio chunks to the file as they arrive (~150 ms TTFB). +# `instruct_text` here has the same Chinese-only caveat as the +# OpenAI path's `instruct` — leave it empty for English work. curl -N -X POST http://10.100.79.3:8190/api/tts \ - -F "text=Hello. I did not ask for visitors today." \ + -F "text=I did not ask for visitors today." \ -F "voice=" \ -F "mode=zero_shot" \ - -F "instruct_text=speak with a flat, menacing calm" \ -F "speed=1.0" \ -F "stream=true" \ - -o out_stream.wav + -o out_stream.pcm ``` Field naming differs between the two synthesis endpoints: @@ -170,16 +186,21 @@ Field naming differs between the two synthesis endpoints: ## Emotion / style control -Two syntaxes, combine freely: +**For English content, use XML inline tags. Do not use the `instruct` +field** — see the Gotcha above; `instruct` routes through CosyVoice's +Chinese-context inference path and English values get spoken aloud +instead of treated as direction. -| Syntax | Where | Example | -|---|---|---| -| XML tags in the text | inline in `input` / `text` | `You're late.` | -| Natural-language directive | `instruct` / `instruct_text` | `"speak with a flat, menacing calm"` | +| Syntax | Where | Example | English-safe? | +|---|---|---|---| +| XML tags in the text | inline in `input` / `text` | `You're late.` | ✅ | +| Natural-language directive | `instruct` / `instruct_text` | `"用愤怒的语气说"` (Chinese) | ⚠️ Chinese only | Tag vocabulary includes ``, ``, ``, ``, ``, ``, ``, ``, plus -character/style tags like ``, ``. +character/style tags like ``, ``. Tags can be nested +or applied at sentence granularity (the model accepts paragraph- +length input with multiple tagged segments). ## Version bump