catalog(dia2): expose full /tts control surface + stable predefined voices

Repoint both dia2 entries from /v1/audio/speech to the wrapper's richer /tts
endpoint (CustomTTSRequest), exposing the levers that fix the random-voice
problem: voice_mode, clone_reference_filename, cfg_scale, temperature, top_p,
cfg_filter_top_k, speed_factor, seed, split_text, chunk_size, transcript,
max_tokens. All defaults are the wrapper's Pydantic blessed values (cfg 3.0 /
temp 1.3 / top_p 0.95 / top_k 35 / speed_factor 0.94 / chunk 300). Fields
grouped (basic/sampling/advanced). dia2 -> version 2 (field-shape change).

Voice stability: Dia2 samples a random speaker per call unless anchored. The
43 curated voices baked at /app/voices aren't reachable from /tts's clone path
(reference_audio dir only), so they're staged into reference_audio; the
clone_reference_filename picker now sources /get_reference_files. voice_mode=
clone + a reference filename pins voice/gender. Verified /tts clone end-to-end
(HTTP 200, Ogg/Opus 24 kHz). README documents the staging + two-instance shape.
This commit is contained in:
2026-05-31 15:18:53 -07:00
parent e97d80cb8e
commit 55602b7251
2 changed files with 260 additions and 60 deletions
+215 -50
View File
@@ -1248,77 +1248,159 @@ services:
turn-taking in one pass with [S1]/[S2] speaker tags and nonverbals turn-taking in one pass with [S1]/[S2] speaker tags and nonverbals
(laughs)/(coughs)/(sighs). 24 kHz. Served via devnen/Dia-TTS-Server. (laughs)/(coughs)/(sighs). 24 kHz. Served via devnen/Dia-TTS-Server.
category: tts category: tts
version: 1 version: 2
status: ready status: ready
host: irv-ml1 host: irv-ml1
lifecycle: lifecycle:
stack: dia stack: dia
vram_gb: 8 vram_gb: 8
gpu_device_id: 0 gpu_device_id: 0
endpoint: http://10.100.79.3:8200/v1/audio/speech endpoint: http://10.100.79.3:8200/tts
method: POST method: POST
content_type: application/json content_type: application/json
model: model:
id: nari-labs/Dia2-2B id: nari-labs/Dia2-2B
revision: null revision: null
image: local/dia:v2 image: local/dia:v2
section_groups:
- id: basic
label: Text & voice
- id: sampling
label: Sampling
hint: Wrapper-canonical Dia2 defaults; lower temperature/cfg for steadier output.
- id: advanced
label: Advanced
fields: fields:
- name: input - name: text
type: textarea type: textarea
label: Text ([S1]/[S2] dialogue + nonverbals) label: Text ([S1]/[S2] dialogue + nonverbals)
section: basic
required: true required: true
max_length: 5000 max_length: 5000
default: "[S1] Hello Dia2!" # canonical: nari-labs/dia2 README usage example default: "[S1] Hello Dia2!" # canonical: nari-labs/dia2 README usage example
description: > description: >
[S1]/[S2] tags mark speaker turns; nonverbals like (laughs), [S1]/[S2] tags mark speaker turns; nonverbals like (laughs),
(coughs), (sighs), (clears throat) go inline. (coughs), (sighs), (clears throat) go inline.
- name: voice - name: voice_mode
type: select type: select
label: Voice / mode label: Voice mode
options: [dialogue, single_s1, single_s2, predefined] section: basic
default: dialogue options: [single_s1, single_s2, dialogue, clone]
default: single_s1
description: > description: >
Speaker mode from the model's voice set. `clone` (prefix_speaker) Canonical default single_s1. Unconditioned modes (single_s1/s2,
needs a reference clip, so it isn't a plain picker choice here. dialogue) sample a RANDOM speaker each call — gender varies. For a
- name: response_format STABLE voice, set this to clone and pick a clone_reference_filename.
- name: clone_reference_filename
type: select type: select
label: Voice (clone reference)
section: basic
optional: true
source_url: http://10.100.79.3:8200/get_reference_files
source_jsonpath: $[*]
description: >
Required when voice_mode=clone. Lists the reference_audio library —
43 curated voices (singles + [S1]/[S2] dialogue pairs like
Abigail_Taylor.wav) staged there from the image's /app/voices, plus
any you upload. Pins voice/gender deterministically.
- name: transcript
type: textarea
label: Reference transcript
section: advanced
optional: true
description: >
Optional transcript of a custom clone reference (overrides its local
.txt sidecar). Leave blank for predefined voices.
- name: output_format
type: select
section: basic
options: [opus, wav] options: [opus, wav]
default: opus default: opus
description: 24 kHz output (Dia2; note dia1 was 44.1 kHz). description: 24 kHz (Dia2).
- name: speed - name: cfg_scale
type: slider type: slider
section: sampling
min: 1.0
max: 5.0
step: 0.1
default: 3.0
description: Classifier-free guidance scale (wrapper default 3.0).
- name: temperature
type: slider
section: sampling
min: 0.1
max: 1.5
step: 0.05
default: 1.3
description: Sampling temperature; lower = steadier (wrapper default 1.3).
- name: top_p
type: slider
section: sampling
min: 0.1
max: 1.0
step: 0.01
default: 0.95
- name: cfg_filter_top_k
type: number
section: sampling
required: false
default: 35
description: Top-k filter for CFG guidance (1100; wrapper default 35).
- name: speed_factor
type: slider
section: sampling
min: 0.5 min: 0.5
max: 2.0 max: 2.0
step: 0.05 step: 0.01
default: 1.0 default: 0.94
description: Post-generation playback speed multiplier. description: Playback speed (wrapper canonical default 0.94).
- name: seed - name: seed
type: number type: number
section: sampling
required: false required: false
default: -1 default: -1
description: -1 = random; any integer for repeatable (not byte-exact) output. description: -1 = random; a fixed integer repeats the same take.
- name: split_text
type: bool
section: advanced
default: true
description: Auto-split long text into chunks.
- name: chunk_size
type: slider
section: advanced
min: 100
max: 1000
step: 10
default: 300
description: Target chunk length in chars when splitting (wrapper default 300).
- name: max_tokens
type: number
section: advanced
required: false
description: Max audio tokens per chunk; omit to use the model's internal default.
response: response:
type: audio type: audio
mime_from_field: response_format mime_from_field: output_format
reproducibility: reproducibility:
seedable: true seedable: true
deterministic: false deterministic: false
notes: > notes: >
Seed gives consistent voice/prosody but NOT byte-exact output (Dia2 is Per the dia2 README, voices vary per generation unless anchored — set
temperature-sampled). Raw-bytes response carries no seed echo, so no voice_mode=clone + a clone_reference_filename for a stable voice/gender.
seed_field. A richer custom /tts endpoint exposes seed (-1=random) repeats a take but doesn't choose the voice;
cfg_scale/temperature/top_p/cfg_filter_top_k. temperature-sampled, not byte-exact; raw bytes -> no seed_field. All
field defaults are the wrapper's CustomTTSRequest Pydantic blessed values.
estimated_latency: estimated_latency:
cold_start_s: 30 cold_start_s: 30
warm_per_unit: "dialogue one-pass; ~realtime on the 3090" warm_per_unit: "dialogue one-pass; ~realtime on the 3090"
license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)" license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)"
notes: | notes: |
One of two fixed-model instances in the `dia` stack; sibling dia2-1b Routes to the wrapper's richer /tts endpoint (full control surface:
(:8202). `docker compose up` on the stack starts BOTH services — the cfg_scale/temperature/top_p/cfg_filter_top_k/voice_mode/clone) rather than
lifecycle vram_gb here is this instance alone (~8 GB); budget for both the OpenAI /v1/audio/speech. One of two fixed-model instances in the `dia`
if gating the whole stack. Image local/dia:v2 = upstream devnen wrapper stack; sibling dia2-1b (:8202). `docker compose up` starts BOTH (~8 GB
+ the dia2 package baked in (see stacks/dia/dia2-image/). Verified here; budget both if gating the whole stack). Image local/dia:v2 (see
end-to-end 2026-05-31 (HTTP 200, Ogg/Opus 24 kHz). stacks/dia/dia2-image/). Verified end-to-end 2026-05-31.
- id: dia2-1b - id: dia2-1b
name: Dia2 1B (Streaming) name: Dia2 1B (Streaming)
@@ -1327,73 +1409,156 @@ services:
dialogue + nonverbals as dia2-2b, lower latency / lighter weight. 24 kHz. dialogue + nonverbals as dia2-2b, lower latency / lighter weight. 24 kHz.
Served via devnen/Dia-TTS-Server. Served via devnen/Dia-TTS-Server.
category: tts category: tts
version: 1 version: 2
status: ready status: ready
host: irv-ml1 host: irv-ml1
lifecycle: lifecycle:
stack: dia stack: dia
vram_gb: 5 vram_gb: 5
gpu_device_id: 0 gpu_device_id: 0
endpoint: http://10.100.79.3:8202/v1/audio/speech endpoint: http://10.100.79.3:8202/tts
method: POST method: POST
content_type: application/json content_type: application/json
model: model:
id: nari-labs/Dia2-1B id: nari-labs/Dia2-1B
revision: null revision: null
image: local/dia:v2 image: local/dia:v2
section_groups:
- id: basic
label: Text & voice
- id: sampling
label: Sampling
hint: Wrapper-canonical Dia2 defaults; lower temperature/cfg for steadier output.
- id: advanced
label: Advanced
fields: fields:
- name: input - name: text
type: textarea type: textarea
label: Text ([S1]/[S2] dialogue + nonverbals) label: Text ([S1]/[S2] dialogue + nonverbals)
section: basic
required: true required: true
max_length: 5000 max_length: 5000
default: "[S1] Hello Dia2!" # canonical: nari-labs/dia2 README usage example default: "[S1] Hello Dia2!" # canonical: nari-labs/dia2 README usage example
description: > description: >
[S1]/[S2] tags mark speaker turns; nonverbals like (laughs), [S1]/[S2] tags mark speaker turns; nonverbals like (laughs),
(coughs), (sighs), (clears throat) go inline. (coughs), (sighs), (clears throat) go inline.
- name: voice - name: voice_mode
type: select type: select
label: Voice / mode label: Voice mode
options: [dialogue, single_s1, single_s2, predefined] section: basic
default: dialogue options: [single_s1, single_s2, dialogue, clone]
default: single_s1
description: > description: >
Speaker mode from the model's voice set. `clone` (prefix_speaker) Canonical default single_s1. Unconditioned modes (single_s1/s2,
needs a reference clip, so it isn't a plain picker choice here. dialogue) sample a RANDOM speaker each call — gender varies. For a
- name: response_format STABLE voice, set this to clone and pick a clone_reference_filename.
- name: clone_reference_filename
type: select type: select
label: Voice (clone reference)
section: basic
optional: true
source_url: http://10.100.79.3:8202/get_reference_files
source_jsonpath: $[*]
description: >
Required when voice_mode=clone. Lists the reference_audio library —
43 curated voices (singles + [S1]/[S2] dialogue pairs like
Abigail_Taylor.wav) staged there from the image's /app/voices, plus
any you upload. Pins voice/gender deterministically.
- name: transcript
type: textarea
label: Reference transcript
section: advanced
optional: true
description: >
Optional transcript of a custom clone reference (overrides its local
.txt sidecar). Leave blank for predefined voices.
- name: output_format
type: select
section: basic
options: [opus, wav] options: [opus, wav]
default: opus default: opus
description: 24 kHz output. description: 24 kHz (Dia2).
- name: speed - name: cfg_scale
type: slider type: slider
section: sampling
min: 1.0
max: 5.0
step: 0.1
default: 3.0
description: Classifier-free guidance scale (wrapper default 3.0).
- name: temperature
type: slider
section: sampling
min: 0.1
max: 1.5
step: 0.05
default: 1.3
description: Sampling temperature; lower = steadier (wrapper default 1.3).
- name: top_p
type: slider
section: sampling
min: 0.1
max: 1.0
step: 0.01
default: 0.95
- name: cfg_filter_top_k
type: number
section: sampling
required: false
default: 35
description: Top-k filter for CFG guidance (1100; wrapper default 35).
- name: speed_factor
type: slider
section: sampling
min: 0.5 min: 0.5
max: 2.0 max: 2.0
step: 0.05 step: 0.01
default: 1.0 default: 0.94
description: Post-generation playback speed multiplier. description: Playback speed (wrapper canonical default 0.94).
- name: seed - name: seed
type: number type: number
section: sampling
required: false required: false
default: -1 default: -1
description: -1 = random; any integer for repeatable (not byte-exact) output. description: -1 = random; a fixed integer repeats the same take.
- name: split_text
type: bool
section: advanced
default: true
description: Auto-split long text into chunks.
- name: chunk_size
type: slider
section: advanced
min: 100
max: 1000
step: 10
default: 300
description: Target chunk length in chars when splitting (wrapper default 300).
- name: max_tokens
type: number
section: advanced
required: false
description: Max audio tokens per chunk; omit to use the model's internal default.
response: response:
type: audio type: audio
mime_from_field: response_format mime_from_field: output_format
reproducibility: reproducibility:
seedable: true seedable: true
deterministic: false deterministic: false
notes: > notes: >
Same determinism story as dia2-2b (temperature-sampled, seed gives Same story as dia2-2b: voices vary per generation unless anchored
consistent prosody not byte-exact). No seed echo -> no seed_field. (voice_mode=clone + clone_reference_filename); temperature-sampled,
not byte-exact; raw bytes -> no seed_field. Defaults are the wrapper's
CustomTTSRequest Pydantic blessed values.
estimated_latency: estimated_latency:
cold_start_s: 30 cold_start_s: 30
warm_per_unit: "streaming; lower latency than dia2-2b" warm_per_unit: "streaming; lower latency than dia2-2b"
license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)" license: "Apache-2.0 (Dia2 weights); MIT (devnen wrapper)"
notes: | notes: |
Sibling of dia2-2b in the same `dia` stack (see its notes for the Routes to the wrapper's richer /tts endpoint (same control surface as
single-model-per-endpoint rationale and the both-start-together caveat). dia2-2b). Sibling of dia2-2b in the same `dia` stack (see its notes for
Image local/dia:v2. Verified end-to-end 2026-05-31 (HTTP 200, Ogg/Opus the single-model-per-endpoint rationale + both-start-together caveat).
24 kHz). Image local/dia:v2. Verified end-to-end 2026-05-31.
- id: csm - id: csm
name: Sesame CSM (conversational) name: Sesame CSM (conversational)
+45 -10
View File
@@ -54,15 +54,50 @@ scripts/deploy-stack.sh irv-ml1 dia
First boot pulls the checkpoint (~6-10 GB) into `DIA_CACHE_DIR` and can First boot pulls the checkpoint (~6-10 GB) into `DIA_CACHE_DIR` and can
take several minutes; the healthcheck's 600 s `start_period` covers it. take several minutes; the healthcheck's 600 s `start_period` covers it.
## Deployment shape (as of 2026-05-31)
This stack now runs **two fixed-model instances** from `local/dia:v2`
(the dia2-capable image — see [`dia2-image/Dockerfile`](dia2-image/Dockerfile)):
| service | model | port | notes |
|---|---|---|---|
| `dia2-2b` | nari-labs/Dia2-2B | 8200 | highest quality |
| `dia2-1b` | nari-labs/Dia2-1B | 8202 | streaming / lower latency |
The wrapper is single-model and ignores per-request model selection, so
one fixed instance per model is the only way to offer both as real
asset-engine choices. Legacy Dia 1.6B was retired. `local/dia:v2` is built
in two stages: upstream wrapper → `local/dia:v1`, then `dia2-image/` layers
in the dia2 package + its deps. Each instance pins its model via a mounted
`/opt/docker/conf/dia2-*/config.yaml`.
## Voices — stabilizing the random-voice behavior
Dia2 samples a **random speaker (random gender) per generation** unless
anchored (per the dia2 README: "voices vary per generation … use with
prefix … for stable output"). To pin a voice, use the richer **`/tts`**
endpoint with `voice_mode: clone` + a `clone_reference_filename`.
The image bakes **43 curated voices** at `/app/voices` (singles +
`[S1]/[S2]` dialogue pairs like `Abigail_Taylor.wav`), but `/tts`'s clone
path only reads the **reference_audio** dir — so they're **staged** into it:
```bash
# one-time per host (writes through the shared /worktank/dia/reference_audio mount):
docker exec dia2-2b sh -c 'cp -n /app/voices/* /app/reference_audio/'
```
After staging, `GET /get_reference_files` lists them and asset-engine's
`clone_reference_filename` picker (sourced from that endpoint) offers a
stable, known voice. Restic-included, so it survives once staged.
## Notes ## Notes
- **Pin `DIA_SHA`** to a full 40-char commit before relying on this — - **Model switching** within an instance is config.yaml-driven (the mounted
`.env.example` ships `main` for convenience, but `main` is not `config.yaml` pins `model.repo_id`); the Web UI can hot-swap live but only
reproducible (chatterbox learned this when an upstream restructure the mounted config survives recreate.
broke its `main` build). - Endpoints: `/tts` (rich: cfg_scale/temperature/top_p/cfg_filter_top_k/
- **Model switching** is config.yaml-driven in the wrapper, or live from voice_mode/clone — what asset-engine targets), `/v1/audio/speech`
the Web UI at `http://10.100.79.3:8200/`. To pin a non-default model (OpenAI-compat; its `voice` param also resolves predefined voices by name),
declaratively, mount a host `config.yaml` (see the commented volume in `/get_reference_files`, `/get_predefined_voices`, `/health`,
`compose.yaml`). `/api/model-status`, `/api/model-info`.
- Endpoints: `/v1/audio/speech` (OpenAI-compat), `/health` (liveness),
`/api/model-status` (download/load progress), `/api/model-info`.