vh
55602b7251
Repoint both dia2 entries from /v1/audio/speech to the wrapper's richer /tts endpoint (CustomTTSRequest), exposing the levers that fix the random-voice problem: voice_mode, clone_reference_filename, cfg_scale, temperature, top_p, cfg_filter_top_k, speed_factor, seed, split_text, chunk_size, transcript, max_tokens. All defaults are the wrapper's Pydantic blessed values (cfg 3.0 / temp 1.3 / top_p 0.95 / top_k 35 / speed_factor 0.94 / chunk 300). Fields grouped (basic/sampling/advanced). dia2 -> version 2 (field-shape change). Voice stability: Dia2 samples a random speaker per call unless anchored. The 43 curated voices baked at /app/voices aren't reachable from /tts's clone path (reference_audio dir only), so they're staged into reference_audio; the clone_reference_filename picker now sources /get_reference_files. voice_mode= clone + a reference filename pins voice/gender. Verified /tts clone end-to-end (HTTP 200, Ogg/Opus 24 kHz). README documents the staging + two-instance shape.
104 lines
4.6 KiB
Markdown
104 lines
4.6 KiB
Markdown
# Dia / Dia2
|
|
|
|
Nari Labs' dialogue-focused TTS — generates ultra-realistic
|
|
multi-speaker conversation in a single pass — served via
|
|
[devnen/Dia-TTS-Server](https://github.com/devnen/Dia-TTS-Server), the
|
|
same actively-maintained OpenAI-compat wrapper author as our
|
|
[chatterbox](../chatterbox/) stack. The wrapper carries multi-model
|
|
support for the whole Dia family:
|
|
|
|
| Model | HF repo | Notes |
|
|
|---|---|---|
|
|
| Dia 1.6B | [nari-labs/Dia-1.6B](https://huggingface.co/nari-labs/Dia-1.6B) | original, dialogue in one pass (in-image default) |
|
|
| Dia2-1B | Nari Labs Dia2 family | streaming, lower latency |
|
|
| Dia2-2B | Nari Labs Dia2 family | highest quality |
|
|
|
|
Dia2 was released 2025-11-19 ([nari-labs/dia2](https://github.com/nari-labs/dia2)).
|
|
|
|
**Server:** irv-ml1 (Irvine, WireGuard-only)
|
|
**Port:** 8200 (container listens on 8003)
|
|
**GPUs:** pins to device 0 (RTX 3090) by default; ~7 GB VRAM at BF16
|
|
**Image:** `local/dia:v1` — built locally from a pinned git SHA of the
|
|
wrapper repo via docker buildx's git URL context
|
|
**Upstream wrapper:** [devnen/Dia-TTS-Server](https://github.com/devnen/Dia-TTS-Server) (MIT)
|
|
**Upstream model:** [nari-labs/dia](https://github.com/nari-labs/dia) / [nari-labs/dia2](https://github.com/nari-labs/dia2) (Apache-2.0 weights)
|
|
|
|
## Why this stack exists
|
|
|
|
Fills the **dialogue-scene** slot none of the other TTS own cleanly.
|
|
Dia generates multi-speaker turn-taking in one pass with inline `[S1]`/
|
|
`[S2]` speaker tags and nonverbal cues — `(laughs)`, `(coughs)`,
|
|
`(sighs)`, `(clears throat)` — directly in the prompt. That's a
|
|
different shape from the single-speaker engines:
|
|
|
|
- **Fish S2-Pro / IndexTTS-2 / Chatterbox** are excellent single-voice
|
|
readers (rich emotion, cloning) but you'd have to stitch turns
|
|
yourself.
|
|
- **VibeVoice** does long-form multi-speaker but is podcast/narration
|
|
shaped, not fast turn-taking with nonverbals.
|
|
- **Dia** is the one built for *scene dialogue* — the
|
|
multi-character-storytelling case skaldsong is aimed at.
|
|
|
|
OpenAI-compatible (`POST /v1/audio/speech`), so skaldsong can target it
|
|
by base-URL once we add a `dia` engine option to its router.
|
|
|
|
## Deploy
|
|
|
|
```bash
|
|
# from this workstation (irv-ml1 is WG-only — routes via ana-wg):
|
|
scripts/deploy-stack.sh irv-ml1 dia
|
|
# then on irv-ml1, first run builds the image from the pinned SHA:
|
|
# docker compose up -d --build
|
|
```
|
|
|
|
First boot pulls the checkpoint (~6-10 GB) into `DIA_CACHE_DIR` and can
|
|
take several minutes; the healthcheck's 600 s `start_period` covers it.
|
|
|
|
## Deployment shape (as of 2026-05-31)
|
|
|
|
This stack now runs **two fixed-model instances** from `local/dia:v2`
|
|
(the dia2-capable image — see [`dia2-image/Dockerfile`](dia2-image/Dockerfile)):
|
|
|
|
| service | model | port | notes |
|
|
|---|---|---|---|
|
|
| `dia2-2b` | nari-labs/Dia2-2B | 8200 | highest quality |
|
|
| `dia2-1b` | nari-labs/Dia2-1B | 8202 | streaming / lower latency |
|
|
|
|
The wrapper is single-model and ignores per-request model selection, so
|
|
one fixed instance per model is the only way to offer both as real
|
|
asset-engine choices. Legacy Dia 1.6B was retired. `local/dia:v2` is built
|
|
in two stages: upstream wrapper → `local/dia:v1`, then `dia2-image/` layers
|
|
in the dia2 package + its deps. Each instance pins its model via a mounted
|
|
`/opt/docker/conf/dia2-*/config.yaml`.
|
|
|
|
## Voices — stabilizing the random-voice behavior
|
|
|
|
Dia2 samples a **random speaker (random gender) per generation** unless
|
|
anchored (per the dia2 README: "voices vary per generation … use with
|
|
prefix … for stable output"). To pin a voice, use the richer **`/tts`**
|
|
endpoint with `voice_mode: clone` + a `clone_reference_filename`.
|
|
|
|
The image bakes **43 curated voices** at `/app/voices` (singles +
|
|
`[S1]/[S2]` dialogue pairs like `Abigail_Taylor.wav`), but `/tts`'s clone
|
|
path only reads the **reference_audio** dir — so they're **staged** into it:
|
|
|
|
```bash
|
|
# one-time per host (writes through the shared /worktank/dia/reference_audio mount):
|
|
docker exec dia2-2b sh -c 'cp -n /app/voices/* /app/reference_audio/'
|
|
```
|
|
|
|
After staging, `GET /get_reference_files` lists them and asset-engine's
|
|
`clone_reference_filename` picker (sourced from that endpoint) offers a
|
|
stable, known voice. Restic-included, so it survives once staged.
|
|
|
|
## Notes
|
|
|
|
- **Model switching** within an instance is config.yaml-driven (the mounted
|
|
`config.yaml` pins `model.repo_id`); the Web UI can hot-swap live but only
|
|
the mounted config survives recreate.
|
|
- Endpoints: `/tts` (rich: cfg_scale/temperature/top_p/cfg_filter_top_k/
|
|
voice_mode/clone — what asset-engine targets), `/v1/audio/speech`
|
|
(OpenAI-compat; its `voice` param also resolves predefined voices by name),
|
|
`/get_reference_files`, `/get_predefined_voices`, `/health`,
|
|
`/api/model-status`, `/api/model-info`.
|