Files

T

vh 55602b7251 catalog(dia2): expose full /tts control surface + stable predefined voices

Repoint both dia2 entries from /v1/audio/speech to the wrapper's richer /tts
endpoint (CustomTTSRequest), exposing the levers that fix the random-voice
problem: voice_mode, clone_reference_filename, cfg_scale, temperature, top_p,
cfg_filter_top_k, speed_factor, seed, split_text, chunk_size, transcript,
max_tokens. All defaults are the wrapper's Pydantic blessed values (cfg 3.0 /
temp 1.3 / top_p 0.95 / top_k 35 / speed_factor 0.94 / chunk 300). Fields
grouped (basic/sampling/advanced). dia2 -> version 2 (field-shape change).

Voice stability: Dia2 samples a random speaker per call unless anchored. The
43 curated voices baked at /app/voices aren't reachable from /tts's clone path
(reference_audio dir only), so they're staged into reference_audio; the
clone_reference_filename picker now sources /get_reference_files. voice_mode=
clone + a reference filename pins voice/gender. Verified /tts clone end-to-end
(HTTP 200, Ogg/Opus 24 kHz). README documents the staging + two-instance shape.

2026-05-31 15:18:53 -07:00

4.6 KiB

Raw Blame History

Dia / Dia2

Nari Labs' dialogue-focused TTS — generates ultra-realistic multi-speaker conversation in a single pass — served via devnen/Dia-TTS-Server, the same actively-maintained OpenAI-compat wrapper author as our chatterbox stack. The wrapper carries multi-model support for the whole Dia family:

Model	HF repo	Notes
Dia 1.6B	nari-labs/Dia-1.6B	original, dialogue in one pass (in-image default)
Dia2-1B	Nari Labs Dia2 family	streaming, lower latency
Dia2-2B	Nari Labs Dia2 family	highest quality

Dia2 was released 2025-11-19 (nari-labs/dia2).

Server: irv-ml1 (Irvine, WireGuard-only) Port: 8200 (container listens on 8003) GPUs: pins to device 0 (RTX 3090) by default; ~7 GB VRAM at BF16 Image: local/dia:v1 — built locally from a pinned git SHA of the wrapper repo via docker buildx's git URL context Upstream wrapper: devnen/Dia-TTS-Server (MIT) Upstream model: nari-labs/dia / nari-labs/dia2 (Apache-2.0 weights)

Why this stack exists

Fills the dialogue-scene slot none of the other TTS own cleanly. Dia generates multi-speaker turn-taking in one pass with inline [S1]/ [S2] speaker tags and nonverbal cues — (laughs), (coughs), (sighs), (clears throat) — directly in the prompt. That's a different shape from the single-speaker engines:

Fish S2-Pro / IndexTTS-2 / Chatterbox are excellent single-voice readers (rich emotion, cloning) but you'd have to stitch turns yourself.
VibeVoice does long-form multi-speaker but is podcast/narration shaped, not fast turn-taking with nonverbals.
Dia is the one built for scene dialogue — the multi-character-storytelling case skaldsong is aimed at.

OpenAI-compatible (POST /v1/audio/speech), so skaldsong can target it by base-URL once we add a dia engine option to its router.

Deploy

# from this workstation (irv-ml1 is WG-only — routes via ana-wg):
scripts/deploy-stack.sh irv-ml1 dia
# then on irv-ml1, first run builds the image from the pinned SHA:
#   docker compose up -d --build

First boot pulls the checkpoint (~6-10 GB) into DIA_CACHE_DIR and can take several minutes; the healthcheck's 600 s start_period covers it.

Deployment shape (as of 2026-05-31)

This stack now runs two fixed-model instances from local/dia:v2 (the dia2-capable image — see dia2-image/Dockerfile):

service	model	port	notes
`dia2-2b`	nari-labs/Dia2-2B	8200	highest quality
`dia2-1b`	nari-labs/Dia2-1B	8202	streaming / lower latency

The wrapper is single-model and ignores per-request model selection, so one fixed instance per model is the only way to offer both as real asset-engine choices. Legacy Dia 1.6B was retired. local/dia:v2 is built in two stages: upstream wrapper → local/dia:v1, then dia2-image/ layers in the dia2 package + its deps. Each instance pins its model via a mounted /opt/docker/conf/dia2-*/config.yaml.

Voices — stabilizing the random-voice behavior

Dia2 samples a random speaker (random gender) per generation unless anchored (per the dia2 README: "voices vary per generation … use with prefix … for stable output"). To pin a voice, use the richer /tts endpoint with voice_mode: clone + a clone_reference_filename.

The image bakes 43 curated voices at /app/voices (singles + [S1]/[S2] dialogue pairs like Abigail_Taylor.wav), but /tts's clone path only reads the reference_audio dir — so they're staged into it:

# one-time per host (writes through the shared /worktank/dia/reference_audio mount):
docker exec dia2-2b sh -c 'cp -n /app/voices/* /app/reference_audio/'

After staging, GET /get_reference_files lists them and asset-engine's clone_reference_filename picker (sourced from that endpoint) offers a stable, known voice. Restic-included, so it survives once staged.

Notes

Model switching within an instance is config.yaml-driven (the mounted config.yaml pins model.repo_id); the Web UI can hot-swap live but only the mounted config survives recreate.
Endpoints: /tts (rich: cfg_scale/temperature/top_p/cfg_filter_top_k/ voice_mode/clone — what asset-engine targets), /v1/audio/speech (OpenAI-compat; its voice param also resolves predefined voices by name), /get_reference_files, /get_predefined_voices, /health, /api/model-status, /api/model-info.

4.6 KiB Raw Blame History