From 93cedd695163ac5141f68d8d7ec7ab380069df75 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 1 Jun 2026 01:18:59 -0700 Subject: [PATCH] catalog(chatterbox): route to /tts, expose emotion levers + sane defaults MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Chatterbox was producing poor output because the catalog pointed at the thin OpenAI /v1/audio/speech endpoint, which exposes none of Resemble's emotion/ pacing knobs — and the devnen server's shipped default exaggeration is 1.3 (tuned for its theatrical demo presets), which over-acts. Re-point to the wrapper's richer /tts and expose the real control surface (exaggeration, cfg_weight, temperature, speed_factor, seed, voice_mode), mirroring the sibling dia stack (same devnen author). Defaults sourced live 2026-06-01: exaggeration + cfg_weight = 0.5 (Resemble README 'works well for most prompts'), temperature 0.8 / speed 1.0 / seed 0 (server generation_ defaults). The shipped 1.3 exaggeration is deliberately NOT adopted. Voices: expose the 28 built-in predefined voices via /get_predefined_voices (default Emily.wav, the server default_voice_id) + clone via /get_reference_ files — replacing the wrong 'OpenAI aliases only' claim. Corrected seedable: false -> true (/tts has seed) and image_tag_mutable -> true (:latest). Bumped service version 1 -> 2 (breaking field-shape change); status down -> ready (live + healthy). catalog_version unchanged (no new field types). --- docs/asset-engine/services.yaml | 183 ++++++++++++++++++++++++++------ 1 file changed, 150 insertions(+), 33 deletions(-) diff --git a/docs/asset-engine/services.yaml b/docs/asset-engine/services.yaml index 84fed16..8a8c511 100644 --- a/docs/asset-engine/services.yaml +++ b/docs/asset-engine/services.yaml @@ -199,68 +199,185 @@ services: renderer ships. Once present, flip to status: ready. - id: chatterbox - name: Chatterbox Turbo TTS + name: Chatterbox TTS description: > - Resemble AI's low-latency English TTS (350M, ~75ms TTFB, 6× realtime). - Zero-shot voice cloning from ~5s reference. 9 paralinguistic tags. + Resemble AI's low-latency English TTS (Chatterbox-Turbo, 350M, ~75ms TTFB, + 6× realtime). 28 built-in predefined voices + zero-shot cloning from a + 5–30s reference. Inline paralinguistic tags, plus Resemble's signature + exaggeration / cfg_weight emotion + pacing control. category: tts - version: 1 - status: down + version: 2 + status: ready host: irv-ml1 lifecycle: stack: chatterbox vram_gb: 4 gpu_device_id: 0 - endpoint: http://10.100.79.3:8196/v1/audio/speech + endpoint: http://10.100.79.3:8196/tts method: POST content_type: application/json model: id: ResembleAI/chatterbox-turbo revision: null image: devnen/Chatterbox-TTS-Server:latest + section_groups: + - id: basic + label: Text & voice + - id: sampling + label: Expression & sampling + hint: Resemble's neutral defaults (exaggeration 0.5 / cfg_weight 0.5). Raise exaggeration or lower cfg_weight for drama. + - id: advanced + label: Advanced fields: - - name: input + - name: text type: textarea label: Text (with optional [tags]) + section: basic required: true max_length: 5000 description: > - Inline tags: [laugh] [chuckle] [sigh] [gasp] [cough] [clear throat] - [sniff] [groan] [shush]. Turbo loses base-Chatterbox's exaggeration knob. - - name: model + Inline paralinguistic tags honored by Turbo: [laugh] [chuckle] [sigh] + [gasp] [cough] [clear throat] [sniff] [groan] [shush]. Best results + when a physical tag is paired with surrounding emotional context. + - name: voice_mode type: select - options: [chatterbox-turbo] - default: chatterbox-turbo - - name: voice - type: select - label: Voice - default: alloy + label: Voice mode + section: basic + options: [predefined, clone] + default: predefined description: > - Built-in OpenAI-compat aliases (alloy, echo, fable, onyx, nova, shimmer). - Cloned: 5–15s WAV files in /worktank/chatterbox/reference_audio/. - - name: response_format + `predefined` -> a built-in voice (predefined_voice_id below). + `clone` -> a reference clip (reference_audio_filename). predefined is + the out-of-box default; the empty/"undefined" case is avoided by + defaulting the voice below. + - name: predefined_voice_id type: select - options: [wav, opus, aac, flac, pcm_s16] + label: Voice (built-in) + section: basic + optional: true + default: "Emily.wav" + source_url: http://10.100.79.3:8196/get_predefined_voices + source_jsonpath: $[*].filename + description: > + Required when voice_mode=predefined. 28 built-in voices staged in the + devnen image (Abigail, Adrian, Alexander, Alice, Austin, Axel, Connor, + Cora, Elena, Eli, Emily, Everett, Gabriel, Gianna, Henry, Ian, Jade, + Jeremiah, Jordan, Julian, Layla, Leonardo, Michael, Miles, Olivia, + Ryan, Taylor, Thomas — each .wav). Default Emily.wav is the + server's own default_voice_id. Verified live via /get_predefined_voices. + - name: reference_audio_filename + type: select + label: Voice (clone reference) + section: basic + optional: true + source_url: http://10.100.79.3:8196/get_reference_files + source_jsonpath: $[*] + description: > + Required when voice_mode=clone. 5–30s clean WAV (16 kHz+ mono) under + /worktank/chatterbox/reference_audio/; upload via the server's + /upload_reference. Match the clip's language to `language` to avoid + accent transfer (or set cfg_weight=0). + - name: exaggeration + type: slider + section: sampling + min: 0.25 + max: 2.0 + step: 0.05 + default: 0.5 + description: > + Emotional intensity. Resemble's docs: 0.5 "works well for most prompts + across all languages"; ~0.7+ for dramatic delivery (which also speeds + speech up). NOTE: the devnen server *ships* 1.3 (tuned for its + theatrical demo presets) — 0.5 is the general-use value and the catalog + default; the shipped 1.3 is the likely cause of over-acted/unstable output. + - name: cfg_weight + type: slider + section: sampling + min: 0.0 + max: 1.0 + step: 0.05 + default: 0.5 + description: > + Pacing / prompt adherence (Resemble default 0.5). Lower to ~0.3 to + slow delivery, for fast/intense reference speakers, or alongside a + raised exaggeration for drama; 0 effectively disables guidance (useful + to reduce reference-accent transfer). + - name: temperature + type: slider + section: sampling + min: 0.05 + max: 2.0 + step: 0.05 + default: 0.8 + description: Sampling temperature; lower = steadier. Server + Resemble default 0.8. + - name: speed_factor + type: slider + section: sampling + min: 0.5 + max: 2.0 + step: 0.05 + default: 1.0 + description: Post-hoc playback speed. Server default 1.0. + - name: seed + type: number + section: sampling + required: false + default: 0 + description: 0 = random; a fixed integer repeats the same take. + - name: output_format + type: select + section: basic + options: [wav, opus, mp3] default: wav - - name: stream + description: 24 kHz. Live-verified enum (wav/opus/mp3). + - name: language + type: text + section: advanced + required: false + default: en + description: > + Language override. Base Turbo is English; the multilingual variant + (23 languages, via the stack .env) honors other codes. Leave `en`. + - name: split_text type: bool - default: false + section: advanced + default: true + description: Auto-split long text into chunks. + - name: chunk_size + type: slider + section: advanced + min: 100 + max: 1000 + step: 10 + default: 120 + description: Target chunk length in chars when splitting (server default 120). response: type: audio - mime_from_field: response_format + mime_from_field: output_format reproducibility: - seedable: false - deterministic: true + seedable: true + deterministic: false + seed_field: seed notes: > - No seed. Wrapper repo updates ~weekly; pin SHA in .env. PerTh watermark - unconditionally applied (Resemble policy). + /tts exposes `seed` (0=random); a fixed seed + identical params repeats a + take. Temperature-sampled → not guaranteed byte-exact, and Resemble's + PerTh watermark is applied unconditionally. (Prior catalog claimed no + seed support — corrected against the live OpenAPI 2026-06-01.) estimated_latency: cold_start_s: 3 warm_per_unit: "~75ms TTFB, 6× realtime" license: MIT notes: | - Python 3.10 only (wrapper hardcoding). - Multilingual variant (23 languages) also available via .env. + Routes to the devnen wrapper's richer /tts (full control surface: + exaggeration / cfg_weight / temperature / speed_factor / seed / voice_mode) + instead of the thin OpenAI /v1/audio/speech, which exposes NONE of the + emotion knobs — that omission was why prior output was poor. Same wrapper + author as the `dia` stack; identical predefined/clone voice model. + Defaults sourced from Resemble's README (exaggeration + cfg_weight = 0.5) + and the server's generation_defaults (temperature 0.8, speed 1.0, seed 0), + read live 2026-06-01; the server's shipped exaggeration 1.3 is demo-tuned + and deliberately NOT adopted. Python 3.10 only (wrapper hardcoding); + multilingual (23-language) variant available via the stack .env. - id: index-tts name: IndexTTS-2 @@ -1724,10 +1841,10 @@ reproducibility_audit: image_tag_mutable: true notes: "Same image as kokoro proper; same mutability story. Response carries timestamps." - service: chatterbox - seedable: false - model_deterministic: true - image_tag_mutable: false - notes: "PerTh watermark unconditional (Resemble policy)." + seedable: true + model_deterministic: false + image_tag_mutable: true + notes: "/tts exposes seed (0=random); temperature-sampled, not byte-exact. PerTh watermark unconditional (Resemble policy). image :latest is mutable — pin a digest/SHA for true repro." - service: index-tts seedable: false model_deterministic: true