feat(omnivoice): expose full generation surface (voice-design, language, diffusion params)
Wrapper /v1/audio/speech now accepts OmniVoice's whole surface: - voice (clone, now OPTIONAL) and/or instruct (voice DESIGN). instruct is a CONTROLLED vocabulary (gender/age/pitch/accent/whisper tags, comma-separated), not free prose — discoverable at the new /v1/audio/instruct-items endpoint (23 items). - language (Auto + 647, new /v1/audio/languages endpoint), speed, duration. - diffusion controls: num_step, guidance_scale, denoise, preprocess_prompt, postprocess_output; plus a generation_overrides JSON passthrough for expert GenerationConfig knobs (t_shift, layer_penalty_factor, position/class temperature, audio_chunk_*). - at least one of voice/instruct required (else 400). Catalog (services.yaml): omnivoice v1 -> v2, 13 schema-valid fields; instruct as a controlled-vocab text field sourced from the items endpoint. Verified live on irv-ml1: clone, voice-design (instruct-only), and tuned-param synths all -> 24 kHz PCM_16 WAV; 647 languages; 23 instruct items.
This commit is contained in:
@@ -572,7 +572,7 @@ services:
|
||||
(diffusion-LM, RTF ~0.025). Apache-2.0. Behind our own FastAPI wrapper
|
||||
(stacks/omnivoice/app.py); voices are the reused chatterbox reference clips.
|
||||
category: tts
|
||||
version: 1
|
||||
version: 2
|
||||
status: ready
|
||||
host: irv-ml1
|
||||
lifecycle:
|
||||
@@ -592,15 +592,90 @@ services:
|
||||
label: Text
|
||||
required: true
|
||||
max_length: 5000
|
||||
# Voice source — at least one of voice (clone) / instruct (design) is required.
|
||||
- name: voice
|
||||
type: select
|
||||
label: Speaker Voice
|
||||
label: Speaker Voice (clone)
|
||||
optional: true
|
||||
source_url: http://10.100.79.3:8199/v1/audio/voices
|
||||
source_jsonpath: $.voices[*]
|
||||
description: >
|
||||
Zero-shot clone target — a reference clip staged in
|
||||
/worktank/omnivoice/voices/ (reused chatterbox voices; 33 at deploy).
|
||||
Live list at /v1/audio/voices.
|
||||
Zero-shot clone target — a reference clip in /worktank/omnivoice/voices/
|
||||
(reused chatterbox voices; 33 at deploy). Omit to design a voice via
|
||||
instruct instead. Live list at /v1/audio/voices.
|
||||
- name: instruct
|
||||
type: text
|
||||
label: Voice Design (instruct)
|
||||
optional: true
|
||||
source_url: http://10.100.79.3:8199/v1/audio/instruct-items
|
||||
source_jsonpath: $.instruct_items[*]
|
||||
description: >
|
||||
Voice DESIGN — a comma-separated list of CONTROLLED attribute tags (not
|
||||
free prose), e.g. "british accent, elderly, male, low pitch". Valid tags
|
||||
(gender/age/pitch/accent/whisper) at /v1/audio/instruct-items. Use instead
|
||||
of, or together with, a clone voice.
|
||||
- name: language
|
||||
type: select
|
||||
label: Language
|
||||
optional: true
|
||||
default: Auto
|
||||
source_url: http://10.100.79.3:8199/v1/audio/languages
|
||||
source_jsonpath: $.languages[*]
|
||||
description: "Auto-detects when left as Auto; 600+ languages supported."
|
||||
- name: speed
|
||||
type: slider
|
||||
label: Speed
|
||||
optional: true
|
||||
min: 0.5
|
||||
max: 1.5
|
||||
default: 1.0
|
||||
description: "1.0 = normal; >1 faster, <1 slower. Ignored if duration is set."
|
||||
- name: duration
|
||||
type: number
|
||||
label: Duration (seconds)
|
||||
optional: true
|
||||
description: "Fixed output length in seconds; overrides speed when set."
|
||||
- name: num_step
|
||||
type: slider
|
||||
label: Inference Steps
|
||||
optional: true
|
||||
min: 4
|
||||
max: 64
|
||||
default: 32
|
||||
description: "Diffusion steps. Lower = faster, higher = better quality."
|
||||
- name: guidance_scale
|
||||
type: slider
|
||||
label: Guidance Scale (CFG)
|
||||
optional: true
|
||||
min: 0.0
|
||||
max: 4.0
|
||||
default: 2.0
|
||||
- name: denoise
|
||||
type: bool
|
||||
label: Denoise
|
||||
optional: true
|
||||
default: true
|
||||
- name: preprocess_prompt
|
||||
type: bool
|
||||
label: Preprocess Prompt
|
||||
optional: true
|
||||
default: true
|
||||
description: "Silence-trim + punctuate the reference (clone mode)."
|
||||
- name: postprocess_output
|
||||
type: bool
|
||||
label: Postprocess Output
|
||||
optional: true
|
||||
default: true
|
||||
description: "Remove long silences from the generated audio."
|
||||
- name: generation_overrides
|
||||
type: json
|
||||
label: Advanced (GenerationConfig)
|
||||
optional: true
|
||||
description: >
|
||||
Expert OmniVoiceGenerationConfig overrides as a JSON object — keys:
|
||||
t_shift (0.1), layer_penalty_factor (5.0), position_temperature (5.0),
|
||||
class_temperature (0.0), audio_chunk_duration (15.0),
|
||||
audio_chunk_threshold (30.0). Unknown keys ignored.
|
||||
- name: response_format
|
||||
type: select
|
||||
options: [wav]
|
||||
@@ -621,9 +696,12 @@ services:
|
||||
warm_per_unit: "full-utterance (no streaming)"
|
||||
license: "Apache-2.0"
|
||||
notes: |
|
||||
v1 is clone-only (voice = a staged reference clip); OmniVoice's voice-DESIGN
|
||||
and language/instruct controls are not yet exposed in the wrapper. No streaming.
|
||||
Voices reused from chatterbox /refs (staged into /worktank/omnivoice/voices/).
|
||||
Two voice sources, combinable: voice (clone a staged reference clip) and/or
|
||||
instruct (free-text voice DESIGN); at least one required. Full generation
|
||||
surface exposed — language (600+), speed, duration, num_step, guidance_scale,
|
||||
denoise, preprocess/postprocess — with expert GenerationConfig knobs (t_shift,
|
||||
layer/position/class temperature, audio_chunk_*) via the generation_overrides
|
||||
JSON field. No streaming. Voices reused from chatterbox /refs.
|
||||
|
||||
- id: qwen3-tts
|
||||
name: Qwen3-TTS 1.7B
|
||||
|
||||
Reference in New Issue
Block a user