catalog(canonical): remove chatterbox dead levers + add csm-expressiva
The asset-engine catalog source of truth. Removes the chatterbox exaggeration/cfg_weight sliders (proven Turbo no-ops) — reconciling the canonical with the fix previously applied only to asset-engine's vendored copy — and adds the csm-expressiva whisper TTS entry (irv-ml1:8198).
This commit is contained in:
+109
-39
@@ -203,8 +203,8 @@ services:
|
|||||||
description: >
|
description: >
|
||||||
Resemble AI's low-latency English TTS (Chatterbox-Turbo, 350M, ~75ms TTFB,
|
Resemble AI's low-latency English TTS (Chatterbox-Turbo, 350M, ~75ms TTFB,
|
||||||
6× realtime). 28 built-in predefined voices + zero-shot cloning from a
|
6× realtime). 28 built-in predefined voices + zero-shot cloning from a
|
||||||
5–30s reference. Inline paralinguistic tags, plus Resemble's signature
|
5–30s reference. Inline paralinguistic tags. (Resemble's exaggeration /
|
||||||
exaggeration / cfg_weight emotion + pacing control.
|
cfg_weight emotion knobs are no-ops on the Turbo checkpoint — omitted.)
|
||||||
category: tts
|
category: tts
|
||||||
version: 2
|
version: 2
|
||||||
status: ready
|
status: ready
|
||||||
@@ -225,7 +225,7 @@ services:
|
|||||||
label: Text & voice
|
label: Text & voice
|
||||||
- id: sampling
|
- id: sampling
|
||||||
label: Expression & sampling
|
label: Expression & sampling
|
||||||
hint: Resemble's neutral defaults (exaggeration 0.5 / cfg_weight 0.5). Raise exaggeration or lower cfg_weight for drama.
|
hint: Turbo honors temperature / seed / speed_factor. (exaggeration / cfg_weight are Turbo no-ops — removed.)
|
||||||
- id: advanced
|
- id: advanced
|
||||||
label: Advanced
|
label: Advanced
|
||||||
fields:
|
fields:
|
||||||
@@ -276,32 +276,12 @@ services:
|
|||||||
Required when voice_mode=clone. 5–30s clean WAV (16 kHz+ mono) under
|
Required when voice_mode=clone. 5–30s clean WAV (16 kHz+ mono) under
|
||||||
/worktank/chatterbox/reference_audio/; upload via the server's
|
/worktank/chatterbox/reference_audio/; upload via the server's
|
||||||
/upload_reference. Match the clip's language to `language` to avoid
|
/upload_reference. Match the clip's language to `language` to avoid
|
||||||
accent transfer (or set cfg_weight=0).
|
accent transfer.
|
||||||
- name: exaggeration
|
# exaggeration / cfg_weight intentionally removed: Chatterbox-Turbo IGNORES
|
||||||
type: slider
|
# both. Proven 2026-06-04 — same text/voice/seed across cfg 0.0/0.5/0.9 gave
|
||||||
section: sampling
|
# byte-identical audio, and the model logs "CFG, min_p and exaggeration are
|
||||||
min: 0.25
|
# not supported by Turbo version and will be ignored." They were dead UI
|
||||||
max: 2.0
|
# sliders. They'd only do something on a non-turbo Chatterbox checkpoint.
|
||||||
step: 0.05
|
|
||||||
default: 0.5
|
|
||||||
description: >
|
|
||||||
Emotional intensity. Resemble's docs: 0.5 "works well for most prompts
|
|
||||||
across all languages"; ~0.7+ for dramatic delivery (which also speeds
|
|
||||||
speech up). NOTE: the devnen server *ships* 1.3 (tuned for its
|
|
||||||
theatrical demo presets) — 0.5 is the general-use value and the catalog
|
|
||||||
default; the shipped 1.3 is the likely cause of over-acted/unstable output.
|
|
||||||
- name: cfg_weight
|
|
||||||
type: slider
|
|
||||||
section: sampling
|
|
||||||
min: 0.0
|
|
||||||
max: 1.0
|
|
||||||
step: 0.05
|
|
||||||
default: 0.5
|
|
||||||
description: >
|
|
||||||
Pacing / prompt adherence (Resemble default 0.5). Lower to ~0.3 to
|
|
||||||
slow delivery, for fast/intense reference speakers, or alongside a
|
|
||||||
raised exaggeration for drama; 0 effectively disables guidance (useful
|
|
||||||
to reduce reference-accent transfer).
|
|
||||||
- name: temperature
|
- name: temperature
|
||||||
type: slider
|
type: slider
|
||||||
section: sampling
|
section: sampling
|
||||||
@@ -368,16 +348,16 @@ services:
|
|||||||
warm_per_unit: "~75ms TTFB, 6× realtime"
|
warm_per_unit: "~75ms TTFB, 6× realtime"
|
||||||
license: MIT
|
license: MIT
|
||||||
notes: |
|
notes: |
|
||||||
Routes to the devnen wrapper's richer /tts (full control surface:
|
Routes to the devnen wrapper's /tts (control surface: temperature /
|
||||||
exaggeration / cfg_weight / temperature / speed_factor / seed / voice_mode)
|
speed_factor / seed / voice_mode) rather than the thin OpenAI
|
||||||
instead of the thin OpenAI /v1/audio/speech, which exposes NONE of the
|
/v1/audio/speech. exaggeration / cfg_weight were DROPPED 2026-06-04 — the
|
||||||
emotion knobs — that omission was why prior output was poor. Same wrapper
|
Turbo checkpoint ignores them (proven byte-identical across cfg values; the
|
||||||
author as the `dia` stack; identical predefined/clone voice model.
|
model warns they're "not supported by Turbo version and will be ignored"),
|
||||||
Defaults sourced from Resemble's README (exaggeration + cfg_weight = 0.5)
|
so they were dead UI sliders. Same wrapper author as the `dia` stack;
|
||||||
and the server's generation_defaults (temperature 0.8, speed 1.0, seed 0),
|
identical predefined/clone voice model. Defaults from the server's
|
||||||
read live 2026-06-01; the server's shipped exaggeration 1.3 is demo-tuned
|
generation_defaults (temperature 0.8, speed 1.0, seed 0), read live
|
||||||
and deliberately NOT adopted. Python 3.10 only (wrapper hardcoding);
|
2026-06-01. Python 3.10 only (wrapper hardcoding); multilingual (23-language)
|
||||||
multilingual (23-language) variant available via the stack .env.
|
variant available via the stack .env.
|
||||||
|
|
||||||
- id: chatterbox-fast
|
- id: chatterbox-fast
|
||||||
name: Chatterbox Fast (streaming)
|
name: Chatterbox Fast (streaming)
|
||||||
@@ -673,6 +653,91 @@ services:
|
|||||||
notes: |
|
notes: |
|
||||||
Voice cloning shape differs from CosyVoice: profile-based, not voice-id-based.
|
Voice cloning shape differs from CosyVoice: profile-based, not voice-id-based.
|
||||||
|
|
||||||
|
- id: csm-expressiva
|
||||||
|
name: CSM Expressiva (whisper)
|
||||||
|
description: >
|
||||||
|
Experimental LoRA fine-tune of Sesame's CSM-1B (Conversational Speech Model)
|
||||||
|
on Expresso's whispering voice — WHISPER-style speech only, trained speaker
|
||||||
|
= 4. Custom thin server (no upstream CSM server image exists). Autoregressive,
|
||||||
|
so ~RTF 1.8 on the A6000 — good for offline/clip generation, not live
|
||||||
|
streaming. License cc-by-nc-4.0 (non-commercial). No output watermark.
|
||||||
|
category: tts
|
||||||
|
version: 1
|
||||||
|
status: experimental
|
||||||
|
host: irv-ml1
|
||||||
|
lifecycle:
|
||||||
|
stack: csm-expressiva
|
||||||
|
vram_gb: 4
|
||||||
|
gpu_device_id: 1
|
||||||
|
endpoint: http://10.100.79.3:8198/tts
|
||||||
|
method: POST
|
||||||
|
content_type: application/json
|
||||||
|
model:
|
||||||
|
id: senstella/csm-expressiva-1b
|
||||||
|
revision: null
|
||||||
|
image: local/csm-expressiva:v1
|
||||||
|
section_groups:
|
||||||
|
- id: basic
|
||||||
|
label: Text
|
||||||
|
- id: sampling
|
||||||
|
label: Sampling
|
||||||
|
fields:
|
||||||
|
- name: text
|
||||||
|
type: textarea
|
||||||
|
label: Text
|
||||||
|
section: basic
|
||||||
|
required: true
|
||||||
|
max_length: 2000
|
||||||
|
description: >
|
||||||
|
Text to synthesize as a whisper. CSM is conversational — short
|
||||||
|
utterances work best. No inline paralinguistic tag support.
|
||||||
|
- name: speaker
|
||||||
|
type: number
|
||||||
|
section: basic
|
||||||
|
required: false
|
||||||
|
default: 4
|
||||||
|
description: >
|
||||||
|
Speaker id. The fine-tune only trained speaker 4 (Expresso whisper);
|
||||||
|
leave at 4 — other ids fall back to off-distribution base-CSM behavior.
|
||||||
|
- name: max_audio_length_ms
|
||||||
|
type: number
|
||||||
|
section: sampling
|
||||||
|
required: false
|
||||||
|
default: 20000
|
||||||
|
description: Hard cap on generated audio length (ms); generation stops early on EOS.
|
||||||
|
- name: temperature
|
||||||
|
type: slider
|
||||||
|
section: sampling
|
||||||
|
min: 0.05
|
||||||
|
max: 1.5
|
||||||
|
step: 0.05
|
||||||
|
default: 0.8
|
||||||
|
description: Sampling temperature. Repo (csm-mlx) example default 0.8.
|
||||||
|
- name: topk
|
||||||
|
type: number
|
||||||
|
section: sampling
|
||||||
|
required: false
|
||||||
|
default: 50
|
||||||
|
description: Top-k sampling cutoff. Repo example default 50.
|
||||||
|
response:
|
||||||
|
type: audio
|
||||||
|
mime: audio/wav
|
||||||
|
reproducibility:
|
||||||
|
seedable: false
|
||||||
|
deterministic: false
|
||||||
|
notes: >
|
||||||
|
No seed field exposed; temperature-sampled, so takes vary run to run.
|
||||||
|
estimated_latency:
|
||||||
|
cold_start_s: 30
|
||||||
|
warm_per_unit: "~RTF 1.8 (≈9s compute per ≈5s of audio on the A6000)"
|
||||||
|
license: cc-by-nc-4.0
|
||||||
|
notes: |
|
||||||
|
Whisper-only experimental fine-tune. Custom thin FastAPI wrapper around the
|
||||||
|
vendored Sesame csm generator (watermark + silentcipher stripped; ungated
|
||||||
|
Llama-3.2 tokenizer mirror). Runs eager — CSM has no torch.compile hot path,
|
||||||
|
so the compiled image gained nothing. NOT behind the LiteLLM gateway (custom
|
||||||
|
/tts schema, not OpenAI /v1/audio/speech).
|
||||||
|
|
||||||
- id: cosyvoice
|
- id: cosyvoice
|
||||||
name: CosyVoice 3 (Multilingual)
|
name: CosyVoice 3 (Multilingual)
|
||||||
description: >
|
description: >
|
||||||
@@ -2132,6 +2197,11 @@ reproducibility_audit:
|
|||||||
model_deterministic: false
|
model_deterministic: false
|
||||||
image_tag_mutable: true
|
image_tag_mutable: true
|
||||||
notes: "/tts exposes seed (0=random) — one-shot byte-exact for fixed seed+params (verified 2026-06-02); STREAMING non-reproducible (timing-dependent chunk boundaries), auditions ephemeral. PerTh watermark unconditional. image local/chatterbox-fast:v1 is tag-pinned + mutable — pin a digest for true repro."
|
notes: "/tts exposes seed (0=random) — one-shot byte-exact for fixed seed+params (verified 2026-06-02); STREAMING non-reproducible (timing-dependent chunk boundaries), auditions ephemeral. PerTh watermark unconditional. image local/chatterbox-fast:v1 is tag-pinned + mutable — pin a digest for true repro."
|
||||||
|
- service: csm-expressiva
|
||||||
|
seedable: false
|
||||||
|
model_deterministic: false
|
||||||
|
image_tag_mutable: true
|
||||||
|
notes: "No seed field exposed; temperature-sampled, takes vary run to run. No watermark (stripped). image local/csm-expressiva:v1 is tag-pinned + mutable — pin a digest for true repro. Runs eager (CSM has no torch.compile hot path)."
|
||||||
- service: index-tts
|
- service: index-tts
|
||||||
seedable: false
|
seedable: false
|
||||||
model_deterministic: true
|
model_deterministic: true
|
||||||
|
|||||||
Reference in New Issue
Block a user