diff --git a/docs/asset-engine/services.yaml b/docs/asset-engine/services.yaml index 45de993..6f1139e 100644 --- a/docs/asset-engine/services.yaml +++ b/docs/asset-engine/services.yaml @@ -1248,77 +1248,159 @@ services: turn-taking in one pass with [S1]/[S2] speaker tags and nonverbals (laughs)/(coughs)/(sighs). 24 kHz. Served via devnen/Dia-TTS-Server. category: tts - version: 1 + version: 2 status: ready host: irv-ml1 lifecycle: stack: dia vram_gb: 8 gpu_device_id: 0 - endpoint: http://10.100.79.3:8200/v1/audio/speech + endpoint: http://10.100.79.3:8200/tts method: POST content_type: application/json model: id: nari-labs/Dia2-2B revision: null image: local/dia:v2 + section_groups: + - id: basic + label: Text & voice + - id: sampling + label: Sampling + hint: Wrapper-canonical Dia2 defaults; lower temperature/cfg for steadier output. + - id: advanced + label: Advanced fields: - - name: input + - name: text type: textarea label: Text ([S1]/[S2] dialogue + nonverbals) + section: basic required: true max_length: 5000 default: "[S1] Hello Dia2!" # canonical: nari-labs/dia2 README usage example description: > [S1]/[S2] tags mark speaker turns; nonverbals like (laughs), (coughs), (sighs), (clears throat) go inline. - - name: voice + - name: voice_mode type: select - label: Voice / mode - options: [dialogue, single_s1, single_s2, predefined] - default: dialogue + label: Voice mode + section: basic + options: [single_s1, single_s2, dialogue, clone] + default: single_s1 description: > - Speaker mode from the model's voice set. `clone` (prefix_speaker) - needs a reference clip, so it isn't a plain picker choice here. - - name: response_format + Canonical default single_s1. Unconditioned modes (single_s1/s2, + dialogue) sample a RANDOM speaker each call — gender varies. For a + STABLE voice, set this to clone and pick a clone_reference_filename. + - name: clone_reference_filename type: select + label: Voice (clone reference) + section: basic + optional: true + source_url: http://10.100.79.3:8200/get_reference_files + source_jsonpath: $[*] + description: > + Required when voice_mode=clone. Lists the reference_audio library — + 43 curated voices (singles + [S1]/[S2] dialogue pairs like + Abigail_Taylor.wav) staged there from the image's /app/voices, plus + any you upload. Pins voice/gender deterministically. + - name: transcript + type: textarea + label: Reference transcript + section: advanced + optional: true + description: > + Optional transcript of a custom clone reference (overrides its local + .txt sidecar). Leave blank for predefined voices. + - name: output_format + type: select + section: basic options: [opus, wav] default: opus - description: 24 kHz output (Dia2; note dia1 was 44.1 kHz). - - name: speed + description: 24 kHz (Dia2). + - name: cfg_scale type: slider + section: sampling + min: 1.0 + max: 5.0 + step: 0.1 + default: 3.0 + description: Classifier-free guidance scale (wrapper default 3.0). + - name: temperature + type: slider + section: sampling + min: 0.1 + max: 1.5 + step: 0.05 + default: 1.3 + description: Sampling temperature; lower = steadier (wrapper default 1.3). + - name: top_p + type: slider + section: sampling + min: 0.1 + max: 1.0 + step: 0.01 + default: 0.95 + - name: cfg_filter_top_k + type: number + section: sampling + required: false + default: 35 + description: Top-k filter for CFG guidance (1–100; wrapper default 35). + - name: speed_factor + type: slider + section: sampling min: 0.5 max: 2.0 - step: 0.05 - default: 1.0 - description: Post-generation playback speed multiplier. + step: 0.01 + default: 0.94 + description: Playback speed (wrapper canonical default 0.94). - name: seed type: number + section: sampling required: false default: -1 - description: -1 = random; any integer for repeatable (not byte-exact) output. + description: -1 = random; a fixed integer repeats the same take. + - name: split_text + type: bool + section: advanced + default: true + description: Auto-split long text into chunks. + - name: chunk_size + type: slider + section: advanced + min: 100 + max: 1000 + step: 10 + default: 300 + description: Target chunk length in chars when splitting (wrapper default 300). + - name: max_tokens + type: number + section: advanced + required: false + description: Max audio tokens per chunk; omit to use the model's internal default. response: type: audio - mime_from_field: response_format + mime_from_field: output_format reproducibility: seedable: true deterministic: false notes: > - Seed gives consistent voice/prosody but NOT byte-exact output (Dia2 is - temperature-sampled). Raw-bytes response carries no seed echo, so no - seed_field. A richer custom /tts endpoint exposes - cfg_scale/temperature/top_p/cfg_filter_top_k. + Per the dia2 README, voices vary per generation unless anchored — set + voice_mode=clone + a clone_reference_filename for a stable voice/gender. + seed (-1=random) repeats a take but doesn't choose the voice; + temperature-sampled, not byte-exact; raw bytes -> no seed_field. All + field defaults are the wrapper's CustomTTSRequest Pydantic blessed values. estimated_latency: cold_start_s: 30 warm_per_unit: "dialogue one-pass; ~realtime on the 3090" license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)" notes: | - One of two fixed-model instances in the `dia` stack; sibling dia2-1b - (:8202). `docker compose up` on the stack starts BOTH services — the - lifecycle vram_gb here is this instance alone (~8 GB); budget for both - if gating the whole stack. Image local/dia:v2 = upstream devnen wrapper - + the dia2 package baked in (see stacks/dia/dia2-image/). Verified - end-to-end 2026-05-31 (HTTP 200, Ogg/Opus 24 kHz). + Routes to the wrapper's richer /tts endpoint (full control surface: + cfg_scale/temperature/top_p/cfg_filter_top_k/voice_mode/clone) rather than + the OpenAI /v1/audio/speech. One of two fixed-model instances in the `dia` + stack; sibling dia2-1b (:8202). `docker compose up` starts BOTH (~8 GB + here; budget both if gating the whole stack). Image local/dia:v2 (see + stacks/dia/dia2-image/). Verified end-to-end 2026-05-31. - id: dia2-1b name: Dia2 1B (Streaming) @@ -1327,73 +1409,156 @@ services: dialogue + nonverbals as dia2-2b, lower latency / lighter weight. 24 kHz. Served via devnen/Dia-TTS-Server. category: tts - version: 1 + version: 2 status: ready host: irv-ml1 lifecycle: stack: dia vram_gb: 5 gpu_device_id: 0 - endpoint: http://10.100.79.3:8202/v1/audio/speech + endpoint: http://10.100.79.3:8202/tts method: POST content_type: application/json model: id: nari-labs/Dia2-1B revision: null image: local/dia:v2 + section_groups: + - id: basic + label: Text & voice + - id: sampling + label: Sampling + hint: Wrapper-canonical Dia2 defaults; lower temperature/cfg for steadier output. + - id: advanced + label: Advanced fields: - - name: input + - name: text type: textarea label: Text ([S1]/[S2] dialogue + nonverbals) + section: basic required: true max_length: 5000 default: "[S1] Hello Dia2!" # canonical: nari-labs/dia2 README usage example description: > [S1]/[S2] tags mark speaker turns; nonverbals like (laughs), (coughs), (sighs), (clears throat) go inline. - - name: voice + - name: voice_mode type: select - label: Voice / mode - options: [dialogue, single_s1, single_s2, predefined] - default: dialogue + label: Voice mode + section: basic + options: [single_s1, single_s2, dialogue, clone] + default: single_s1 description: > - Speaker mode from the model's voice set. `clone` (prefix_speaker) - needs a reference clip, so it isn't a plain picker choice here. - - name: response_format + Canonical default single_s1. Unconditioned modes (single_s1/s2, + dialogue) sample a RANDOM speaker each call — gender varies. For a + STABLE voice, set this to clone and pick a clone_reference_filename. + - name: clone_reference_filename type: select + label: Voice (clone reference) + section: basic + optional: true + source_url: http://10.100.79.3:8202/get_reference_files + source_jsonpath: $[*] + description: > + Required when voice_mode=clone. Lists the reference_audio library — + 43 curated voices (singles + [S1]/[S2] dialogue pairs like + Abigail_Taylor.wav) staged there from the image's /app/voices, plus + any you upload. Pins voice/gender deterministically. + - name: transcript + type: textarea + label: Reference transcript + section: advanced + optional: true + description: > + Optional transcript of a custom clone reference (overrides its local + .txt sidecar). Leave blank for predefined voices. + - name: output_format + type: select + section: basic options: [opus, wav] default: opus - description: 24 kHz output. - - name: speed + description: 24 kHz (Dia2). + - name: cfg_scale type: slider + section: sampling + min: 1.0 + max: 5.0 + step: 0.1 + default: 3.0 + description: Classifier-free guidance scale (wrapper default 3.0). + - name: temperature + type: slider + section: sampling + min: 0.1 + max: 1.5 + step: 0.05 + default: 1.3 + description: Sampling temperature; lower = steadier (wrapper default 1.3). + - name: top_p + type: slider + section: sampling + min: 0.1 + max: 1.0 + step: 0.01 + default: 0.95 + - name: cfg_filter_top_k + type: number + section: sampling + required: false + default: 35 + description: Top-k filter for CFG guidance (1–100; wrapper default 35). + - name: speed_factor + type: slider + section: sampling min: 0.5 max: 2.0 - step: 0.05 - default: 1.0 - description: Post-generation playback speed multiplier. + step: 0.01 + default: 0.94 + description: Playback speed (wrapper canonical default 0.94). - name: seed type: number + section: sampling required: false default: -1 - description: -1 = random; any integer for repeatable (not byte-exact) output. + description: -1 = random; a fixed integer repeats the same take. + - name: split_text + type: bool + section: advanced + default: true + description: Auto-split long text into chunks. + - name: chunk_size + type: slider + section: advanced + min: 100 + max: 1000 + step: 10 + default: 300 + description: Target chunk length in chars when splitting (wrapper default 300). + - name: max_tokens + type: number + section: advanced + required: false + description: Max audio tokens per chunk; omit to use the model's internal default. response: type: audio - mime_from_field: response_format + mime_from_field: output_format reproducibility: seedable: true deterministic: false notes: > - Same determinism story as dia2-2b (temperature-sampled, seed gives - consistent prosody not byte-exact). No seed echo -> no seed_field. + Same story as dia2-2b: voices vary per generation unless anchored + (voice_mode=clone + clone_reference_filename); temperature-sampled, + not byte-exact; raw bytes -> no seed_field. Defaults are the wrapper's + CustomTTSRequest Pydantic blessed values. estimated_latency: cold_start_s: 30 warm_per_unit: "streaming; lower latency than dia2-2b" license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)" notes: | - Sibling of dia2-2b in the same `dia` stack (see its notes for the - single-model-per-endpoint rationale and the both-start-together caveat). - Image local/dia:v2. Verified end-to-end 2026-05-31 (HTTP 200, Ogg/Opus - 24 kHz). + Routes to the wrapper's richer /tts endpoint (same control surface as + dia2-2b). Sibling of dia2-2b in the same `dia` stack (see its notes for + the single-model-per-endpoint rationale + both-start-together caveat). + Image local/dia:v2. Verified end-to-end 2026-05-31. - id: csm name: Sesame CSM (conversational) diff --git a/stacks/dia/README.md b/stacks/dia/README.md index 9468811..11c744e 100644 --- a/stacks/dia/README.md +++ b/stacks/dia/README.md @@ -54,15 +54,50 @@ scripts/deploy-stack.sh irv-ml1 dia First boot pulls the checkpoint (~6-10 GB) into `DIA_CACHE_DIR` and can take several minutes; the healthcheck's 600 s `start_period` covers it. +## Deployment shape (as of 2026-05-31) + +This stack now runs **two fixed-model instances** from `local/dia:v2` +(the dia2-capable image — see [`dia2-image/Dockerfile`](dia2-image/Dockerfile)): + +| service | model | port | notes | +|---|---|---|---| +| `dia2-2b` | nari-labs/Dia2-2B | 8200 | highest quality | +| `dia2-1b` | nari-labs/Dia2-1B | 8202 | streaming / lower latency | + +The wrapper is single-model and ignores per-request model selection, so +one fixed instance per model is the only way to offer both as real +asset-engine choices. Legacy Dia 1.6B was retired. `local/dia:v2` is built +in two stages: upstream wrapper → `local/dia:v1`, then `dia2-image/` layers +in the dia2 package + its deps. Each instance pins its model via a mounted +`/opt/docker/conf/dia2-*/config.yaml`. + +## Voices — stabilizing the random-voice behavior + +Dia2 samples a **random speaker (random gender) per generation** unless +anchored (per the dia2 README: "voices vary per generation … use with +prefix … for stable output"). To pin a voice, use the richer **`/tts`** +endpoint with `voice_mode: clone` + a `clone_reference_filename`. + +The image bakes **43 curated voices** at `/app/voices` (singles + +`[S1]/[S2]` dialogue pairs like `Abigail_Taylor.wav`), but `/tts`'s clone +path only reads the **reference_audio** dir — so they're **staged** into it: + +```bash +# one-time per host (writes through the shared /worktank/dia/reference_audio mount): +docker exec dia2-2b sh -c 'cp -n /app/voices/* /app/reference_audio/' +``` + +After staging, `GET /get_reference_files` lists them and asset-engine's +`clone_reference_filename` picker (sourced from that endpoint) offers a +stable, known voice. Restic-included, so it survives once staged. + ## Notes -- **Pin `DIA_SHA`** to a full 40-char commit before relying on this — - `.env.example` ships `main` for convenience, but `main` is not - reproducible (chatterbox learned this when an upstream restructure - broke its `main` build). -- **Model switching** is config.yaml-driven in the wrapper, or live from - the Web UI at `http://10.100.79.3:8200/`. To pin a non-default model - declaratively, mount a host `config.yaml` (see the commented volume in - `compose.yaml`). -- Endpoints: `/v1/audio/speech` (OpenAI-compat), `/health` (liveness), - `/api/model-status` (download/load progress), `/api/model-info`. +- **Model switching** within an instance is config.yaml-driven (the mounted + `config.yaml` pins `model.repo_id`); the Web UI can hot-swap live but only + the mounted config survives recreate. +- Endpoints: `/tts` (rich: cfg_scale/temperature/top_p/cfg_filter_top_k/ + voice_mode/clone — what asset-engine targets), `/v1/audio/speech` + (OpenAI-compat; its `voice` param also resolves predefined voices by name), + `/get_reference_files`, `/get_predefined_voices`, `/health`, + `/api/model-status`, `/api/model-info`.