refactor(dots-tts): extract TTS stack to tts-stack repo; pointer stub + move voices out

TTS development moves to a dedicated repo (~/development/tts-stack) so a separate
agent can own tuning/dev. Mirrors the chatterbox-fast extraction:

- stacks/dots-tts/ reduced to a pointer README (code/Dockerfile/compose/tests/env
  now canonical in tts-stack).
- voices/ canonical corpus moved out to tts-stack/voices/. Blast-radius checked:
  no eshpfi playbook/script reads the corpus (other voices/ refs are unrelated
  host paths under /worktank/...).
- persistent-memory updated: TTS dev extracted + stood down; reverses the earlier
  "corpus home = eshpfi voices/" call.

The ~15 experimental TTS compose wrappers stay here as reference (catalogued in
tts-stack/KNOWLEDGE.md). Live service on irv-ml1:8198 is unaffected (runs from a
copy on the host).
This commit is contained in:
vh
2026-08-11 07:46:11 -07:00
parent a80f6e958f
commit 62672c9850
20 changed files with 24 additions and 746 deletions
-31
View File
@@ -1,31 +0,0 @@
# dots-tts stack tunables. Copy to `.env` on irv-ml1 before deploying.
# ── image ────────────────────────────────────────────────────────────
# v2 (2026-08-10): curly->ASCII sanitize (fixes "Donut's"->"donut ess" on
# typographic apostrophes) + server-side sentence-chunking (long turns no longer
# truncate at dots' ~40s single-generate cap).
DOTS_TAG=v2
# ── network ──────────────────────────────────────────────────────────
DOTS_BIND=0.0.0.0
DOTS_PORT=8198
# ── GPU ──────────────────────────────────────────────────────────────
# 0 = 3090 in Docker (PCI order), co-resident with chatterbox-fast. soar needs
# ~6GB; the 3090 has headroom with Zonos parked down.
DOTS_GPU_DEVICES=0
# ── model / inference ────────────────────────────────────────────────
DOTS_MODEL=dots-studio/dots.tts-soar
DOTS_DEFAULT_VOICE=donut
DOTS_NUM_STEPS=10 # 10 = full quality @ RTF ~0.22; lower = faster/rougher
DOTS_GUIDANCE_SCALE=1.2
DOTS_CHUNK_MAX_CHARS=280 # max chars per generate() chunk (dots caps ~40s/~500 patches)
# ── host mounts ──────────────────────────────────────────────────────
# HF cache holding the downloaded soar snapshot (~5GB). Reuse the burn-in cache.
DOTS_HFCACHE_DIR=/home/lkraven/dots-tts/hf_cache
# dots-derived voice references (derive.py dots -> derived/dots/<name>.{wav,txt}).
# Burn-in points at the corpus output directly; for a durable deploy, copy the
# derived set to /opt/docker/conf/dots-tts/voices and point here.
DOTS_VOICES_HOST_DIR=/home/lkraven/voice-corpus/derived/dots
-37
View File
@@ -1,37 +0,0 @@
# dots.tts OpenAI-compatible TTS server (thin FastAPI over DotsTtsRuntime).
# GPU access is via `runtime: nvidia` at run time (torch ships its own CUDA
# runtime; no CUDA toolkit / nvcc needed to build — the model uses no custom
# compiled kernels, confirmed on the irv-ml1 venv).
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
libsndfile1 ffmpeg git curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir uv
WORKDIR /app
# Pin the proven-working version set (captured from the running v1 image).
# NOT using upstream constraints/recommended.txt: as of 2026-08-10 it pins
# gradio==6.17.0, which does not exist on PyPI and makes a fresh resolve
# unsatisfiable (upstream regression). dots.tts 0.2.1 pulls a working gradio
# (6.17.3) on its own; torch/numpy/soundfile pinned to the v1-image versions.
RUN uv pip install --system \
dots.tts==0.2.1 torch==2.8.0 torchaudio==2.8.0 numpy==2.2.6 soundfile==0.13.1 \
fastapi "uvicorn[standard]"
# C compiler for the RUNTIME (not build): optimize=True drives torch.compile /
# inductor / triton, which JIT-compile kernels via gcc on model load. Without it
# the runtime dies with "Failed to find C compiler". Placed after the pip layer
# so it doesn't invalidate the expensive torch install cache.
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
&& rm -rf /var/lib/apt/lists/*
COPY app.py /app/app.py
# Persist the inductor compile cache on the mounted (rw) HF cache so kernel
# JIT doesn't re-run on every container restart (~70s warmup otherwise).
ENV HF_HOME=/hf_cache DOTS_PORT=8198 CC=gcc CXX=g++ TORCHINDUCTOR_CACHE_DIR=/hf_cache/inductor
EXPOSE 8198
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8198"]
+21 -55
View File
@@ -1,64 +1,30 @@
# dots-tts
# dots-tts — moved to its own repository
OpenAI-compatible zero-shot voice-clone TTS over **dots.tts** (rednote-hilab) —
2B continuous-AR, native **48kHz**, `optimize=True` CUDA graphs → **RTF ~0.22** on
the irv-ml1 3090. Thin FastAPI wrapper around `DotsTtsRuntime` (chosen over SGLang
Omni: Omni's batching is MeanFlow-only and unneeded for a single consumer; the raw
runtime already streams at the same RTF and is ~100 lines we control).
The dots.tts serving stack now lives in the dedicated **tts-stack** repo:
- **Host:** irv-ml1, port **8198** (chatterbox-fast is :8197 — they co-reside on the 3090)
- **Model:** `dots-studio/dots.tts-soar`, bf16, num_steps=10
- **Voices:** every `<name>.wav` (+ `<name>.txt` transcript) in the mounted voices dir,
sourced from the [`voices/`](../../voices/) canonical corpus via `derive.py dots`.
> **`~/development/tts-stack`** → `stacks/dots-tts/`
> (gitea `vh/tts-stack` on gitea.phasefinal.com once pushed)
## API
Extracted from this workspace on 2026-08-11 so a dedicated agent can drive all TTS
tuning / development. Like chatterbox-fast, dots-tts is **authored software with a
test suite**, so it follows the sister-repo pattern rather than staying a thin
compose wrapper here. The new repo owns the code, Dockerfile, tests, the voice
corpus (formerly `voices/` here), the deploy runbook, and the running TTS
knowledge base.
```
GET /health -> {status, model, sample_rate, voices[]}
GET /v1/voices -> {voices[]}
POST /v1/audio/speech -> audio
body: {input, voice, response_format?("wav"|"pcm"), stream?}
```
## Deployed service
`stream:true` returns a WAV stream (placeholder-header + PCM frames, 48kHz mono
s16le) — the same shape the Zonos/chatterbox consumers already handle. Non-stream
returns a complete WAV (or raw PCM with `response_format:"pcm"`).
Live on **irv-ml1:8198** (`local/dots-tts:v3`, OpenAI `/v1/audio/speech`). The
host stack dir is `/opt/docker/compose/dots-tts/`. Deploy + rollback runbook and
all engine knowledge live in tts-stack (`docs/infrastructure.md`, `KNOWLEDGE.md`).
```bash
curl -X POST http://10.100.79.3:8198/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{"input":"Well, look who finally showed up.","voice":"glados"}' \
--output out.wav
```
## Voice corpus
## Deploy
The canonical voice corpus (was `eshpfi-management/voices/`) moved to
`tts-stack/voices/`.
Reference sets come from the canonical corpus, not this stack — derive then point
the mount at them:
## Other TTS engines
```bash
# 1. produce dots refs from the corpus (on a box with the whisper venv):
python voices/derive.py dots # -> voices/derived/dots/*.{wav,txt}
# 2. build + run on irv-ml1 (cp .env.example .env first; adjust mounts):
scripts/deploy-stack.sh irv-ml1 dots-tts # or, on the host:
docker compose build && docker compose up -d
```
The model (~5GB) is **not** baked — it's read from the mounted `HF_HOME`
(`DOTS_HFCACHE_DIR`). First boot downloads it there if absent.
## Voice cloning gotcha
dots.tts clones from `(reference wav + its transcript)` and **leaks reference
audio into the output** if the transcript is inaccurate or ends mid-clause. The
`voices/` corpus + `derive.py` handle this (sentence-bounded trim + accurate
transcript); don't hand this server a raw reference wav without a matching `.txt`.
## Notes
- **GPU:** `NVIDIA_VISIBLE_DEVICES=0` = the 3090 in Docker (PCI order). `optimize=True`
is **incompatible with `PYTORCH_CUDA_ALLOC_CONF=expandable_segments`** (CUDA-graph
capture error) — don't set it.
- **Variants:** `dots.tts-mf` (MeanFlow, faster) is a drop-in via `DOTS_MODEL`; soar
is the quality pick and single-consumer doesn't need mf's batching.
The experimental TTS compose wrappers evaluated along the way (cosyvoice, dia,
kokoro, vibevoice, zonos, …) remain under `stacks/` here as reference; their
verdicts are catalogued in `tts-stack/KNOWLEDGE.md`.
-215
View File
@@ -1,215 +0,0 @@
"""OpenAI-compatible /v1/audio/speech server over dots.tts (rednote-hilab).
Thin wrapper around DotsTtsRuntime — chosen over SGLang Omni because Omni's edge
(continuous batching) is MeanFlow-only and unneeded for a single-consumer surface,
while the raw runtime with optimize=True already streams at RTF ~0.22 on our 3090.
Voice registry: every <name>.wav (+ optional <name>.txt transcript) under
DOTS_VOICES_DIR becomes a callable voice. dots.tts REQUIRES an accurate,
sentence-bounded transcript to clone cleanly (see the voices/ corpus) — the .txt
is that transcript; without it the model leaks reference audio into the output.
"""
import io
import os
import glob
import re
import struct
import threading
import wave
import numpy as np
from fastapi import FastAPI, HTTPException
from fastapi.responses import Response, StreamingResponse
from pydantic import BaseModel
from dots_tts.runtime import DotsTtsRuntime
MODEL = os.environ.get("DOTS_MODEL", "dots-studio/dots.tts-soar")
VOICES_DIR = os.environ.get("DOTS_VOICES_DIR", "/voices")
DEFAULT_VOICE = os.environ.get("DOTS_DEFAULT_VOICE", "donut")
NUM_STEPS = int(os.environ.get("DOTS_NUM_STEPS", "10"))
GUIDANCE = float(os.environ.get("DOTS_GUIDANCE_SCALE", "1.2"))
SAMPLE_RATE = 48000 # dots.tts fixed native output
# ratatoskr's LLM emits typographic (curly) punctuation, and dots' tokenizer
# mispronounces curly apostrophes ("Donut's" -> "donut ess"). Fold curly -> ASCII
# before synthesis. normalize_text stays ON (operator call — keeps number/date
# expansion); the sanitize just removes the curly trigger the model chokes on.
CURLY_MAP = str.maketrans({
"’": "'", "‘": "'", "“": '"', "”": '"',
"–": "-", "…": "...", " ": " ",
})
# dots caps a single generate() at ~500 audio patches (~40s). Long turns (RP
# monologues) truncate without chunking, so split into <=CHUNK_MAX_CHARS pieces
# on sentence (then clause) boundaries and stitch. A short input is one chunk =
# unchanged behavior.
CHUNK_MAX_CHARS = int(os.environ.get("DOTS_CHUNK_MAX_CHARS", "280"))
app = FastAPI(title="dots.tts")
_rt = None
_voices: dict = {}
# One DotsTtsRuntime, and it is NOT safe to call concurrently (CUDA-graph capture
# + shared state). uvicorn runs sync endpoints in a threadpool, so we must
# serialize generation ourselves: requests queue and run one at a time. This is
# the deliberate trade for the thin-wrapper design — no vLLM-style continuous
# batching. If concurrency demand appears, swap the backend to SGLang Omni + the
# mf variant behind this same API (see README).
_gen_lock = threading.Lock()
def _load_voices() -> dict:
reg = {}
for wav in sorted(glob.glob(os.path.join(VOICES_DIR, "*.wav"))):
name = os.path.splitext(os.path.basename(wav))[0]
txt = os.path.splitext(wav)[0] + ".txt"
reg[name] = {
"wav": wav,
"text": open(txt).read().strip() if os.path.exists(txt) else "",
}
return reg
@app.on_event("startup")
def _startup():
global _rt, _voices
_voices = _load_voices()
_rt = DotsTtsRuntime.from_pretrained(MODEL, precision="bfloat16", optimize=True)
@app.get("/health")
def health():
return {
"status": "ok" if _rt is not None else "loading",
"model": MODEL,
"sample_rate": SAMPLE_RATE,
"voices": sorted(_voices),
}
@app.get("/v1/voices")
def list_voices():
return {"voices": sorted(_voices)}
class SpeechRequest(BaseModel):
input: str
voice: str = DEFAULT_VOICE
model: str | None = None # accepted, ignored (single served model)
response_format: str = "wav" # wav | pcm
stream: bool = False
def _sanitize(text: str) -> str:
"""Fold curly punctuation to ASCII, then map the clause breaks dots runs flat
on (semicolon, clause colon, em-dash — each measured ~+0.03s vs no pause) to a
period, which dots honors as a believable ~0.3s pause (ellipsis, at ~+0.43s,
read as too much). Number contexts are guarded: times (3:45) and ratios (2:1)
keep their colon, and en-dash ranges (folded to hyphen in CURLY_MAP) never
become "10.20"."""
text = text.translate(CURLY_MAP)
text = re.sub(r"\s*;\s*", ". ", text) # semicolon -> period
text = re.sub(r"(?<!\d)\s*:\s*(?!\d)", ". ", text) # clause colon (not 3:45)
text = re.sub(r"\s*—\s*", ". ", text) # em-dash clause break
return text
def _chunk(text: str, max_chars: int = CHUNK_MAX_CHARS) -> list:
"""Pack sentences into <=max_chars chunks (sub-splitting an over-long sentence
on commas) so each generate() stays under dots' ~40s cap. One chunk for short
input."""
text = text.strip()
if len(text) <= max_chars:
return [text]
sentences = re.split(r"(?<=[.!?])\s+", text)
chunks, cur = [], ""
for s in sentences:
s = s.strip()
if not s:
continue
pieces = [s]
if len(s) > max_chars: # rare: a single sentence over the cap
pieces = [p.strip() for p in re.split(r"(?<=,)\s+", s) if p.strip()]
for p in pieces:
if not cur:
cur = p
elif len(cur) + 1 + len(p) <= max_chars:
cur = cur + " " + p
else:
chunks.append(cur)
cur = p
if cur:
chunks.append(cur)
return chunks or [text]
def _to_pcm16(audio: np.ndarray) -> bytes:
return np.round(np.clip(audio, -1.0, 1.0) * 32767.0).astype("<i2").tobytes()
def _wav_bytes(pcm: bytes) -> bytes:
buf = io.BytesIO()
w = wave.open(buf, "wb")
w.setnchannels(1)
w.setsampwidth(2)
w.setframerate(SAMPLE_RATE)
w.writeframes(pcm)
w.close()
return buf.getvalue()
def _streaming_wav_header() -> bytes:
"""WAV header with placeholder (max) sizes — lets a client start playing the
stream before the total length is known (the pattern the Zonos/chatterbox
consumers already expect)."""
return (
b"RIFF" + struct.pack("<I", 0xFFFFFFFF) + b"WAVE"
+ b"fmt " + struct.pack("<IHHIIHH", 16, 1, 1, SAMPLE_RATE, SAMPLE_RATE * 2, 2, 16)
+ b"data" + struct.pack("<I", 0xFFFFFFFF)
)
@app.post("/v1/audio/speech")
def speech(req: SpeechRequest):
rt = _rt
if rt is None:
raise HTTPException(503, "model still loading")
if req.voice not in _voices:
raise HTTPException(404, f"unknown voice '{req.voice}'; have {sorted(_voices)}")
if not req.input.strip():
raise HTTPException(400, "empty input")
v = _voices[req.voice]
text = _sanitize(req.input)
chunks = _chunk(text)
kw = dict(
prompt_audio_path=v["wav"],
prompt_text=v["text"],
num_steps=NUM_STEPS,
guidance_scale=GUIDANCE,
normalize_text=True,
)
gap = np.zeros(int(0.08 * SAMPLE_RATE), dtype=np.float32) # 80ms seam between chunks
if req.stream:
def gen():
# Hold the lock for the whole stream — a second generation on the
# shared runtime mid-stream would corrupt both.
with _gen_lock:
yield _streaming_wav_header()
for i, ch in enumerate(chunks):
if i:
yield _to_pcm16(gap)
for piece in rt.generate_stream(text=ch, **kw):
yield _to_pcm16(piece.float().cpu().squeeze().numpy())
return StreamingResponse(gen(), media_type="audio/wav")
with _gen_lock:
parts = []
for i, ch in enumerate(chunks):
if i:
parts.append(gap)
parts.append(rt.generate(text=ch, **kw)["audio"].float().cpu().squeeze().numpy())
pcm = _to_pcm16(np.concatenate(parts))
if req.response_format == "pcm":
return Response(pcm, media_type="audio/L16;rate=48000")
return Response(_wav_bytes(pcm), media_type="audio/wav")
-50
View File
@@ -1,50 +0,0 @@
# dots.tts — OpenAI-compatible 48kHz zero-shot voice-clone TTS (rednote-hilab).
# Deployed on irv-ml1 ALONGSIDE chatterbox-fast (:8197) for burn-in; both share
# the 3090. Mirrors the chatterbox-fast sibling: GPU via `runtime: nvidia` +
# NVIDIA_VISIBLE_DEVICES, host IP:port (no traefik). All tunables in .env.
#
# GPU note: in Docker, NVIDIA_VISIBLE_DEVICES=0 resolves to the 3090 (PCI order),
# same as chatterbox-fast — NOT the A6000 that *native* CUDA calls device 0 on
# this host. CUDA_DEVICE_ORDER=PCI_BUS_ID is set belt-and-suspenders.
services:
dots-tts:
image: local/dots-tts:${DOTS_TAG:-v1}
build:
context: .
dockerfile: Dockerfile
container_name: dots-tts
restart: unless-stopped
runtime: nvidia
ports:
- "${DOTS_BIND:-0.0.0.0}:${DOTS_PORT:-8198}:8198"
environment:
- NVIDIA_VISIBLE_DEVICES=${DOTS_GPU_DEVICES:-0}
- CUDA_DEVICE_ORDER=PCI_BUS_ID
- HF_HOME=/hf_cache
- DOTS_MODEL=${DOTS_MODEL:-dots-studio/dots.tts-soar}
- DOTS_VOICES_DIR=/voices
- DOTS_DEFAULT_VOICE=${DOTS_DEFAULT_VOICE:-donut}
- DOTS_NUM_STEPS=${DOTS_NUM_STEPS:-10}
- DOTS_GUIDANCE_SCALE=${DOTS_GUIDANCE_SCALE:-1.2}
- DOTS_PORT=8198
volumes:
# HF model cache — reuse the already-downloaded soar snapshot (~5GB).
- ${DOTS_HFCACHE_DIR:-/home/lkraven/dots-tts/hf_cache}:/hf_cache
# Voice references — the dots-derived set from the voices/ corpus
# (derive.py dots -> derived/dots/<name>.{wav,txt}). Read-only.
- ${DOTS_VOICES_HOST_DIR:-/home/lkraven/voice-corpus/derived/dots}:/voices:ro
healthcheck:
# /health returns {"status":"ok",...} only once the model has loaded.
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; b=urllib.request.urlopen('http://127.0.0.1:8198/health',timeout=5).read(); sys.exit(0 if b'\\\"status\\\":\\\"ok\\\"' in b.replace(b' ',b'') else 1)\""]
interval: 30s
timeout: 10s
retries: 3
# Model load + optimize=True CUDA-graph warmup measured ~70s; generous.
start_period: 180s
labels:
- homepage.group=AI - Speech (TTS)
- homepage.name=dots.tts
- homepage.icon=mdi-account-voice
- homepage.description=Continuous-AR 48kHz zero-shot voice clone (irv-ml1)
- homepage.href=http://10.100.79.3:${DOTS_PORT:-8198}
-73
View File
@@ -1,73 +0,0 @@
"""Unit tests for _sanitize — the text pre-fold dots.tts sees before synth.
app.py imports dots_tts.runtime at module load (heavy, GPU-only), so we stub it
before import; _sanitize itself is pure and needs no model.
Behavior under test (v3 clause-pause mapping):
* dots runs flat on ; : and em-dash (measured ~+0.03s vs none); a period gives
a believable ~0.3s clause pause. So map those clause breaks -> period.
* Guard number contexts: times (3:45) and ratios (2:1) keep their colon;
en-dash ranges (10-20) must NOT become "10.20". En-dash -> hyphen (as v2).
* Curly-apostrophe fix (the v2 reason this map exists) stays intact.
* A genuine ellipsis keeps its strong pause (-> "...").
"""
import sys
import types
# Stub the GPU-only runtime import so app.py loads on a CPU test box.
_stub = types.ModuleType("dots_tts.runtime")
_stub.DotsTtsRuntime = object # type: ignore[attr-defined]
sys.modules.setdefault("dots_tts", types.ModuleType("dots_tts"))
sys.modules["dots_tts.runtime"] = _stub
import app # noqa: E402
s = app._sanitize
def test_semicolon_becomes_period():
assert s("I waited; you left") == "I waited. you left"
def test_clause_colon_becomes_period():
assert s("the truth: nobody knew") == "the truth. nobody knew"
def test_time_colon_preserved():
# 3:45 must not become 3.45 ("three point four five")
assert "3:45" in s("meet me at 3:45 sharp")
def test_ratio_colon_preserved():
assert "2:1" in s("the odds were 2:1 against")
def test_em_dash_becomes_period_spaced():
assert s("you came — how touching") == "you came. how touching"
def test_em_dash_becomes_period_unspaced():
assert s("you came—how touching") == "you came. how touching"
def test_en_dash_range_preserved_not_period():
out = s("wait 10–20 minutes")
assert "10.20" not in out # the corruption we're guarding against
assert "10-20" in out # en-dash folds to hyphen (v2 behavior)
def test_curly_apostrophe_folds():
# the original v2 bug: curly ' made "Donut's" -> "donut ess"
assert s("Donut’s treat") == "Donut's treat"
def test_curly_quotes_fold():
assert s("“hi” there") == '"hi" there'
def test_ellipsis_keeps_strong_pause():
assert s("wait… now") == "wait... now"
def test_no_doubled_spaces_introduced():
assert " " not in s("a ; b : c — d")