diff --git a/docs/asset-engine/services.yaml b/docs/asset-engine/services.yaml index 4cce4b4..8442386 100644 --- a/docs/asset-engine/services.yaml +++ b/docs/asset-engine/services.yaml @@ -1234,27 +1234,34 @@ services: /history. Until then, expose ComfyUI as an external link in the UI. User state at /worktank/comfyui/basedir/. - - id: dia - name: Dia / Dia2 TTS + # Dia2 runs as TWO fixed-model instances: the devnen wrapper is + # single-model and IGNORES the OpenAI `model` field (serves whatever + # checkpoint is loaded), so the only way to offer both Dia2 models to + # asset-engine as real per-request choices is one endpoint per model. + # Both live in the `dia` stack (one compose, two services); `compose up` + # starts both. The legacy Dia 1.6B entry was retired 2026-05-31 in favor + # of these — drop the old `dia` tile when re-vendoring. + - id: dia2-2b + name: Dia2 2B (High Quality) description: > - Nari Labs' dialogue TTS — multi-speaker turn-taking in one pass with - [S1]/[S2] speaker tags and nonverbals (laughs)/(coughs)/(sighs). - Dia2 family (1B streaming, 2B high-quality) selectable on the host. + Nari Labs' Dia2 dialogue TTS, 2B high-quality checkpoint — multi-speaker + turn-taking in one pass with [S1]/[S2] speaker tags and nonverbals + (laughs)/(coughs)/(sighs). 24 kHz. Served via devnen/Dia-TTS-Server. category: tts - version: 2 - status: down + version: 1 + status: ready host: irv-ml1 lifecycle: stack: dia - vram_gb: 7 + vram_gb: 8 gpu_device_id: 0 endpoint: http://10.100.79.3:8200/v1/audio/speech method: POST content_type: application/json model: - id: nari-labs/Dia-1.6B + id: nari-labs/Dia2-2B revision: null - image: local/dia:v1 + image: local/dia:v2 fields: - name: input type: textarea @@ -1263,29 +1270,20 @@ services: max_length: 5000 description: > [S1]/[S2] tags mark speaker turns; nonverbals like (laughs), - (coughs), (sighs), (clears throat) go inline. For a clone voice, - prepend the reference transcript manually. - - name: model - type: select - options: [dia-1.6b] - default: dia-1.6b - description: > - Ignored by /v1/audio/speech (serves whatever checkpoint is loaded). - Switch Dia 1.6B / Dia2-1B / Dia2-2B via the wrapper config or Web UI. + (coughs), (sighs), (clears throat) go inline. - name: voice type: select label: Voice / mode - options: [S1, S2, dialogue] - default: S1 + options: [dialogue, single_s1, single_s2, predefined] + default: dialogue description: > - S1 | S2 | dialogue speaker mode. Clone-reference filenames under - /worktank/dia/reference_audio are NOT selectable here (a closed - select can't carry free-text); promote to a source_url-backed - picker or a separate free-text field if clone-by-file is wanted. + Speaker mode from the model's voice set. `clone` (prefix_speaker) + needs a reference clip, so it isn't a plain picker choice here. - name: response_format type: select options: [opus, wav] default: opus + description: 24 kHz output (Dia2; note dia1 was 44.1 kHz). - name: speed type: slider min: 0.5 @@ -1305,21 +1303,95 @@ services: seedable: true deterministic: false notes: > - Seed gives consistent voice/prosody but NOT byte-exact output — - upstream notes float-arithmetic variance across hardware/versions. - A richer custom /tts endpoint exposes cfg_scale/temperature/top_p/ - cfg_filter_top_k for finer control. + Seed gives consistent voice/prosody but NOT byte-exact output (Dia2 is + temperature-sampled). Raw-bytes response carries no seed echo, so no + seed_field. A richer custom /tts endpoint exposes + cfg_scale/temperature/top_p/cfg_filter_top_k. estimated_latency: - cold_start_s: 10 + cold_start_s: 30 warm_per_unit: "dialogue one-pass; ~realtime on the 3090" - license: "Apache-2.0 (Dia weights); MIT (devnen wrapper)" + license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)" notes: | - OpenAI-compat /v1/audio/speech plus a richer custom /tts (cfg_scale 3.0, - temperature 1.3, top_p 0.95, cfg_filter_top_k 35, max_tokens, - split_text/chunk_size). Fields read from devnen documentation.md - (2026-05-31); confirm against live OpenAPI/Pydantic at deploy. Deployed - to irv-ml1 but PARKED (asset-engine orchestrates start) — flip to ready - after the first successful generation. Pin DIA_SHA before build. + One of two fixed-model instances in the `dia` stack; sibling dia2-1b + (:8202). `docker compose up` on the stack starts BOTH services — the + lifecycle vram_gb here is this instance alone (~8 GB); budget for both + if gating the whole stack. Image local/dia:v2 = upstream devnen wrapper + + the dia2 package baked in (see stacks/dia/dia2-image/). Verified + end-to-end 2026-05-31 (HTTP 200, Ogg/Opus 24 kHz). + + - id: dia2-1b + name: Dia2 1B (Streaming) + description: > + Nari Labs' Dia2 dialogue TTS, 1B streaming checkpoint — same [S1]/[S2] + dialogue + nonverbals as dia2-2b, lower latency / lighter weight. 24 kHz. + Served via devnen/Dia-TTS-Server. + category: tts + version: 1 + status: ready + host: irv-ml1 + lifecycle: + stack: dia + vram_gb: 5 + gpu_device_id: 0 + endpoint: http://10.100.79.3:8202/v1/audio/speech + method: POST + content_type: application/json + model: + id: nari-labs/Dia2-1B + revision: null + image: local/dia:v2 + fields: + - name: input + type: textarea + label: Text ([S1]/[S2] dialogue + nonverbals) + required: true + max_length: 5000 + description: > + [S1]/[S2] tags mark speaker turns; nonverbals like (laughs), + (coughs), (sighs), (clears throat) go inline. + - name: voice + type: select + label: Voice / mode + options: [dialogue, single_s1, single_s2, predefined] + default: dialogue + description: > + Speaker mode from the model's voice set. `clone` (prefix_speaker) + needs a reference clip, so it isn't a plain picker choice here. + - name: response_format + type: select + options: [opus, wav] + default: opus + description: 24 kHz output. + - name: speed + type: slider + min: 0.5 + max: 2.0 + step: 0.05 + default: 1.0 + description: Post-generation playback speed multiplier. + - name: seed + type: number + required: false + default: -1 + description: -1 = random; any integer for repeatable (not byte-exact) output. + response: + type: audio + mime_from_field: response_format + reproducibility: + seedable: true + deterministic: false + notes: > + Same determinism story as dia2-2b (temperature-sampled, seed gives + consistent prosody not byte-exact). No seed echo -> no seed_field. + estimated_latency: + cold_start_s: 30 + warm_per_unit: "streaming; lower latency than dia2-2b" + license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)" + notes: | + Sibling of dia2-2b in the same `dia` stack (see its notes for the + single-model-per-endpoint rationale and the both-start-together caveat). + Image local/dia:v2. Verified end-to-end 2026-05-31 (HTTP 200, Ogg/Opus + 24 kHz). - id: csm name: Sesame CSM (conversational) @@ -1430,7 +1502,7 @@ services: # zonos-api service, or drop the Gradio service at promotion. vram_gb: 6 gpu_device_id: 0 - endpoint: http://10.100.79.3:8201/v1/audio/speech + endpoint: http://10.100.79.3:8203/v1/audio/speech method: POST content_type: application/json model: @@ -1469,7 +1541,7 @@ services: label: Clone voice (optional) section: voice optional: true - source_url: http://10.100.79.3:8201/v1/audio/voices + source_url: http://10.100.79.3:8203/v1/audio/voices source_jsonpath: $.voices[*] description: > Reference clip filename under /worktank/zonos/voices/ (10–30s clean @@ -1552,7 +1624,7 @@ services: Transformer variant only (mamba-ssm absent → no hybrid). Fields read from upstream gradio_interface.py (2026-05-31), the CATALOG-CONTRACT blessed source for defaults/ranges. Adapter not yet deployed/verified — flip to - ready (or experimental) after the first successful generation through 8201. + ready (or experimental) after the first successful generation through 8203. # Reproducibility audit — answers per service: (a) seedable, (b) model # deterministic without seed, (c) image tag mutable (security/reproducibility risk). @@ -1624,11 +1696,16 @@ reproducibility_audit: model_deterministic: true image_tag_mutable: false notes: "Reproducibility requires persisting full workflow JSON + seed." - - service: dia + - service: dia2-2b seedable: true model_deterministic: false image_tag_mutable: false - notes: "Seed gives consistent voice, not byte-exact (float variance). Build pin via DIA_SHA." + notes: "Dia2 2B, temperature-sampled (seed gives consistent prosody, not byte-exact). local/dia:v2 (derived; upstream wrapper SHA not pinned)." + - service: dia2-1b + seedable: true + model_deterministic: false + image_tag_mutable: false + notes: "Dia2 1B streaming; same determinism story as dia2-2b. local/dia:v2." - service: csm seedable: false model_deterministic: false diff --git a/stacks/zonos/.env.example b/stacks/zonos/.env.example index 51903d2..d2ed1d3 100644 --- a/stacks/zonos/.env.example +++ b/stacks/zonos/.env.example @@ -20,9 +20,9 @@ ZONOS_TAG=v1 ZONOS_PORT=8199 # Host port for the OpenAI-ish REST adapter (zonos-api service) that -# asset-engine routes to. Container listens on 8000 internally. 8201 is -# the next free slot above the Gradio port. -ZONOS_API_PORT=8201 +# asset-engine routes to. Container listens on 8000 internally. 8203 +# (8200 Dia2-2B, 8201 csm, 8202 Dia2-1B were taken on irv-ml1). +ZONOS_API_PORT=8203 # Bind address. 0.0.0.0 exposes on all interfaces (incl. WG tunnel # interface 10.100.79.3); 127.0.0.1 restricts to local-only. diff --git a/stacks/zonos/README.md b/stacks/zonos/README.md index e748ca4..577d815 100644 --- a/stacks/zonos/README.md +++ b/stacks/zonos/README.md @@ -13,7 +13,7 @@ Models ([`Zonos.from_pretrained()`](https://github.com/Zyphra/Zonos)): | hybrid | [Zyphra/Zonos-v0.1-hybrid](https://huggingface.co/Zyphra/Zonos-v0.1-hybrid) | Mamba-SSM; needs Ampere+ GPU + extra build deps | **Server:** irv-ml1 (Irvine, WireGuard-only) -**Ports:** 8199 Gradio eval UI (container 7860) · 8201 REST adapter (container 8000) +**Ports:** 8199 Gradio eval UI (container 7860) · 8203 REST adapter (container 8000) **GPUs:** pins to device 0 (RTX 3090) by default; ~6 GB VRAM **Image:** `local/zonos:v1` — built locally from a pinned git SHA of the upstream repo via docker buildx's git URL context @@ -29,13 +29,13 @@ natural-language paralinguistic tags. Worth A/B-ing by ear against Chatterbox-Turbo (the research that prompted this stack explicitly said "benchmark Zonos against Chatterbox before choosing"). -## Two surfaces: Gradio eval (8199) + REST adapter (8201) +## Two surfaces: Gradio eval (8199) + REST adapter (8203) The official repo ships a **Gradio WebUI + Python SDK only** — no REST endpoint. So this stack runs two services: - **`zonos`** (8199) — upstream Gradio UI, for *auditioning quality by ear*. -- **`zonos-api`** (8201) — a thin OpenAI-ish adapter we built +- **`zonos-api`** (8203) — a thin OpenAI-ish adapter we built (`adapter/server.py`) exposing `POST /v1/audio/speech` so **asset-engine** can route to Zonos like every other TTS in the catalog. Returns a JSON envelope `{audio: , audio_format, seed}` — the `seed` rides back @@ -52,7 +52,7 @@ unmerged fork. ### Adapter request (example) ```bash -curl -sS http://10.100.79.3:8201/v1/audio/speech \ +curl -sS http://10.100.79.3:8203/v1/audio/speech \ -H 'content-type: application/json' \ -d '{"input":"Hello from Zonos.","language":"en-us","seed":420}' \ | python3 -c 'import sys,json,base64; d=json.load(sys.stdin); open("out.wav","wb").write(base64.b64decode(d["audio"])); print("seed",d["seed"])'