feat(omnivoice): expose full generation surface (voice-design, language, diffusion params)

Wrapper /v1/audio/speech now accepts OmniVoice's whole surface:
- voice (clone, now OPTIONAL) and/or instruct (voice DESIGN). instruct is a CONTROLLED
  vocabulary (gender/age/pitch/accent/whisper tags, comma-separated), not free prose —
  discoverable at the new /v1/audio/instruct-items endpoint (23 items).
- language (Auto + 647, new /v1/audio/languages endpoint), speed, duration.
- diffusion controls: num_step, guidance_scale, denoise, preprocess_prompt,
  postprocess_output; plus a generation_overrides JSON passthrough for expert
  GenerationConfig knobs (t_shift, layer_penalty_factor, position/class temperature,
  audio_chunk_*).
- at least one of voice/instruct required (else 400).

Catalog (services.yaml): omnivoice v1 -> v2, 13 schema-valid fields; instruct as a
controlled-vocab text field sourced from the items endpoint.

Verified live on irv-ml1: clone, voice-design (instruct-only), and tuned-param synths
all -> 24 kHz PCM_16 WAV; 647 languages; 23 instruct items.
This commit is contained in:
vh
2026-06-18 23:25:39 -07:00
parent 71f5784016
commit 981ae4e6a1
2 changed files with 179 additions and 22 deletions
+86 -8
View File
@@ -572,7 +572,7 @@ services:
(diffusion-LM, RTF ~0.025). Apache-2.0. Behind our own FastAPI wrapper
(stacks/omnivoice/app.py); voices are the reused chatterbox reference clips.
category: tts
version: 1
version: 2
status: ready
host: irv-ml1
lifecycle:
@@ -592,15 +592,90 @@ services:
label: Text
required: true
max_length: 5000
# Voice source — at least one of voice (clone) / instruct (design) is required.
- name: voice
type: select
label: Speaker Voice
label: Speaker Voice (clone)
optional: true
source_url: http://10.100.79.3:8199/v1/audio/voices
source_jsonpath: $.voices[*]
description: >
Zero-shot clone target — a reference clip staged in
/worktank/omnivoice/voices/ (reused chatterbox voices; 33 at deploy).
Live list at /v1/audio/voices.
Zero-shot clone target — a reference clip in /worktank/omnivoice/voices/
(reused chatterbox voices; 33 at deploy). Omit to design a voice via
instruct instead. Live list at /v1/audio/voices.
- name: instruct
type: text
label: Voice Design (instruct)
optional: true
source_url: http://10.100.79.3:8199/v1/audio/instruct-items
source_jsonpath: $.instruct_items[*]
description: >
Voice DESIGN — a comma-separated list of CONTROLLED attribute tags (not
free prose), e.g. "british accent, elderly, male, low pitch". Valid tags
(gender/age/pitch/accent/whisper) at /v1/audio/instruct-items. Use instead
of, or together with, a clone voice.
- name: language
type: select
label: Language
optional: true
default: Auto
source_url: http://10.100.79.3:8199/v1/audio/languages
source_jsonpath: $.languages[*]
description: "Auto-detects when left as Auto; 600+ languages supported."
- name: speed
type: slider
label: Speed
optional: true
min: 0.5
max: 1.5
default: 1.0
description: "1.0 = normal; >1 faster, <1 slower. Ignored if duration is set."
- name: duration
type: number
label: Duration (seconds)
optional: true
description: "Fixed output length in seconds; overrides speed when set."
- name: num_step
type: slider
label: Inference Steps
optional: true
min: 4
max: 64
default: 32
description: "Diffusion steps. Lower = faster, higher = better quality."
- name: guidance_scale
type: slider
label: Guidance Scale (CFG)
optional: true
min: 0.0
max: 4.0
default: 2.0
- name: denoise
type: bool
label: Denoise
optional: true
default: true
- name: preprocess_prompt
type: bool
label: Preprocess Prompt
optional: true
default: true
description: "Silence-trim + punctuate the reference (clone mode)."
- name: postprocess_output
type: bool
label: Postprocess Output
optional: true
default: true
description: "Remove long silences from the generated audio."
- name: generation_overrides
type: json
label: Advanced (GenerationConfig)
optional: true
description: >
Expert OmniVoiceGenerationConfig overrides as a JSON object — keys:
t_shift (0.1), layer_penalty_factor (5.0), position_temperature (5.0),
class_temperature (0.0), audio_chunk_duration (15.0),
audio_chunk_threshold (30.0). Unknown keys ignored.
- name: response_format
type: select
options: [wav]
@@ -621,9 +696,12 @@ services:
warm_per_unit: "full-utterance (no streaming)"
license: "Apache-2.0"
notes: |
v1 is clone-only (voice = a staged reference clip); OmniVoice's voice-DESIGN
and language/instruct controls are not yet exposed in the wrapper. No streaming.
Voices reused from chatterbox /refs (staged into /worktank/omnivoice/voices/).
Two voice sources, combinable: voice (clone a staged reference clip) and/or
instruct (free-text voice DESIGN); at least one required. Full generation
surface exposed — language (600+), speed, duration, num_step, guidance_scale,
denoise, preprocess/postprocess — with expert GenerationConfig knobs (t_shift,
layer/position/class temperature, audio_chunk_*) via the generation_overrides
JSON field. No streaming. Voices reused from chatterbox /refs.
- id: qwen3-tts
name: Qwen3-TTS 1.7B