diff --git a/docs/asset-engine/services.yaml b/docs/asset-engine/services.yaml index 6c7bb8b..4cce4b4 100644 --- a/docs/asset-engine/services.yaml +++ b/docs/asset-engine/services.yaml @@ -1241,7 +1241,7 @@ services: [S1]/[S2] speaker tags and nonverbals (laughs)/(coughs)/(sighs). Dia2 family (1B streaming, 2B high-quality) selectable on the host. category: tts - version: 1 + version: 2 status: down host: irv-ml1 lifecycle: @@ -1275,10 +1275,13 @@ services: - name: voice type: select label: Voice / mode + options: [S1, S2, dialogue] default: S1 description: > - S1 | S2 | dialogue, or a predefined/clone reference filename under - /worktank/dia/reference_audio (e.g. "my_ref.wav"). + S1 | S2 | dialogue speaker mode. Clone-reference filenames under + /worktank/dia/reference_audio are NOT selectable here (a closed + select can't carry free-text); promote to a source_url-backed + picker or a separate free-text field if clone-by-file is wanted. - name: response_format type: select options: [opus, wav] @@ -1408,6 +1411,149 @@ services: first start. Because it is non-deterministic, when started it belongs at status: experimental (not ready). Pin CSM_SHA before build. + - id: zonos + name: Zonos-v0.1 TTS + description: > + Zyphra's expressive multilingual open-weight TTS (Apache-2.0) — 44 kHz, + zero-shot voice cloning from a 10–30s clip, and an 8-axis emotion + vector plus pitch/rate/fmax conditioning. First genuinely seedable TTS + in the fleet: the adapter returns the seed so regenerate/fork can pin it. + category: tts + version: 1 + status: down + host: irv-ml1 + lifecycle: + stack: zonos + # Adapter alone (~6 GB). NOTE: the zonos stack currently ships TWO + # services (the Gradio eval UI + this adapter); if orchestration does a + # whole-stack `compose up` it loads the model twice (~12 GB). Target the + # zonos-api service, or drop the Gradio service at promotion. + vram_gb: 6 + gpu_device_id: 0 + endpoint: http://10.100.79.3:8201/v1/audio/speech + method: POST + content_type: application/json + model: + id: Zyphra/Zonos-v0.1-transformer + revision: null + image: local/zonos-api:v1 + section_groups: + - id: text + label: Text + - id: voice + label: Voice + - id: expression + label: Expression + hint: Conditioning knobs; leave emotion blank to keep it unconditional. + - id: generation + label: Generation + fields: + - name: input + type: textarea + label: Text to synthesize + section: text + required: true + max_length: 500 + description: eSpeak-phonemized. Upstream Gradio caps the box at ~500 chars. + - name: language + type: select + label: Language code + section: text + default: en-us + # eSpeak code list is large; these are the documented multilingual + # set. The adapter accepts any code in zonos.supported_language_codes. + options: [en-us, ja, cmn, fr-fr, de] + description: Any eSpeak code Zonos supports; 400 on an unknown code. + - name: voice + type: select + label: Clone voice (optional) + section: voice + optional: true + source_url: http://10.100.79.3:8201/v1/audio/voices + source_jsonpath: $.voices[*] + description: > + Reference clip filename under /worktank/zonos/voices/ (10–30s clean + WAV). Omit for Zonos's default unconditioned voice. + - name: response_format + type: select + section: generation + options: [wav, flac] + default: wav + description: 44.1 kHz. Both lossless (libsndfile); no mp3/opus in v1. + - name: emotion + type: json + label: Emotion vector (optional) + section: expression + optional: true + description: > + 8 floats 0.0–1.0 [happy, sad, disgust, fear, surprise, anger, other, + neutral]. Omit to leave emotion unconditional (the Gradio default; + upstream warns the sliders are finicky and can destabilize output). + - name: speaking_rate + type: slider + section: expression + min: 5.0 + max: 30.0 + step: 0.5 + default: 15.0 + - name: pitch_std + type: slider + label: Pitch std + section: expression + min: 0.0 + max: 300.0 + step: 1 + default: 45.0 + - name: fmax + type: slider + label: Fmax (Hz) + section: expression + min: 0 + max: 24000 + step: 1 + default: 24000 + - name: cfg_scale + type: slider + label: CFG scale + section: generation + min: 1.0 + max: 5.0 + step: 0.1 + default: 2.0 + - name: seed + type: number + section: generation + optional: true + description: Omit for a random seed; the seed used is returned in the envelope. + response: + # JSON envelope (like kokoro-captioned) so the seed rides back with the + # audio. Adapter returns: {audio: , + # audio_format: "audio/wav"|"audio/flac", seed: }. + type: audio + audio_field: audio + audio_format_field: audio_format + reproducibility: + seedable: true + deterministic: true + seed_field: seed + notes: > + torch.manual_seed(seed); the adapter echoes the seed it used in the + envelope's `seed` key. Same seed + params + GPU → byte-stable in + practice; cross-GPU (3090 vs A6000) may differ at bf16. Wire shape + is defined by the adapter (stacks/zonos/adapter/server.py) — verify + against the live endpoint at deploy per CATALOG-CONTRACT. + estimated_latency: + cold_start_s: 30 + warm_per_unit: "~2x realtime on the 3090 (transformer variant)" + license: Apache-2.0 + notes: | + Served via a thin OpenAI-ish FastAPI adapter (stacks/zonos/adapter/) in + front of the Zonos Python SDK — upstream ships only Gradio + SDK, no REST. + Transformer variant only (mamba-ssm absent → no hybrid). Fields read from + upstream gradio_interface.py (2026-05-31), the CATALOG-CONTRACT blessed + source for defaults/ranges. Adapter not yet deployed/verified — flip to + ready (or experimental) after the first successful generation through 8201. + # Reproducibility audit — answers per service: (a) seedable, (b) model # deterministic without seed, (c) image tag mutable (security/reproducibility risk). reproducibility_audit: @@ -1488,3 +1634,8 @@ reproducibility_audit: model_deterministic: false image_tag_mutable: false notes: "⚠️ No seed + temperature-sampled → non-reproducible. Catalogued experimental by operator direction; upstream seed support is the fix. Gated Sesame license." + - service: zonos + seedable: true + model_deterministic: true + image_tag_mutable: false + notes: "Adapter echoes the seed used (reproducibility.seed_field=seed). Byte-stable same-GPU; bf16 may drift cross-GPU. local/zonos-api:v1 built FROM local/zonos (pin ZONOS_SHA for true repro)."