feat(dots-tts): ship OpenAI-compatible dots.tts TTS stack on irv-ml1:8198
Thin FastAPI wrapper over DotsTtsRuntime (soar, optimize=True, RTF ~0.22), serialized single-consumer; OpenAI /v1/audio/speech (stream + non-stream), voices from the voices/ corpus derived set. Live + healthy alongside chatterbox-fast on the 3090; nothing repointed. Dockerfile needs build-essential (torch.compile/inductor JITs via gcc at runtime) + persisted inductor cache. Remaining Phase-2: ratatoskr client cutover.
This commit is contained in:
@@ -119,6 +119,8 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i
|
||||
|
||||
- **EVIDENCE HOLD (partial):** WT #394 index-row half LIFTED + swept (kb 8,230→2,876); the **FILE half STILL STANDS** — do NOT delete the on-disk generation dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`). Also un-actioned worldtree-side: the `reconcile --repair` rc=1 git-staging bug (phantom `docs/specs/saga-taxonomy-redesign-plan.md`) — flagged, their code.
|
||||
|
||||
- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover** — draft the ask to repoint ratatoskr's RP TTS to irv-ml1:8198 `/v1/audio/speech` (Phase-2 tail; dots container LIVE, see Recent decisions). (2) **LFM2.5-2.6b bake-off alias for brokkr-smithy-dev** — arch confirmed (Lfm2ForCausalLM, vLLM 0.24.0 on ana-ml2 supports it, no bump), replied w/ alias `lfm2.5-2.6b` + plan (non-prod, vendor sampling temp0.1/topk50/reppen1.1, reasoning-channel content-non-empty smoke, LFM Open License = eval-only pending operator ruling); **stand-up QUEUED** (thread `01KZNA4MKPDR…`, brokkr said no-rush).
|
||||
|
||||
- **OPEN FOLLOW-UPS:** chatterbox-fast **deployed flat-build-context vs package-repo divergence** (reconcile so a repo rebuild matches deploy — repo commit `6bc7bf0` has cap+norm_loudness in package layout; deployed is flat cap-only). Standing/parked: CI-flip runner-auth research, gitea `REQUIRE_SIGNIN_VIEW=false`, #363 research-wing ingest (no deadline), zonos-gateway CI-wire.
|
||||
|
||||
**althing monitor** ARMED (handle `infra-ops`, task `b1uat4njl`; herald up). ⚠️ Re-arm ONLY after a real FIRE (`<task-notification> completed rc0`), NEVER after a plain operator turn (bounces rc3); spawn `althing-wake-listener` as its OWN `run_in_background` task, **NEVER chained with `&`/`&&`** (orphans it to init — bit me this session; reclaim with `althing-cli stop-monitor` then re-arm cleanly).
|
||||
@@ -128,7 +130,7 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-08-09→10]` **dots.tts (rednote-hilab) TTS burn-in on irv-ml1 + canonical voice corpus built (`voices/`).** Operator-directed eval to potentially replace chatterbox-fast. **dots.tts VERIFIED real** (canonical HF ns `dots-studio/`, `rednote-hilab/dots.tts-*` redirects there; Apache-2.0; PyPI `dots.tts` 0.2.1; 2B continuous-AR = semantic enc + Qwen2.5-1.5B LLM + flow-matching acoustic head over 48kHz AudioVAE; zero-shot clone from wav+transcript). **Runs on Ampere 3090** (sm_86, bf16, no fp8 dep); **optimized RTF 0.22** at num_steps=10 (`from_pretrained(..., optimize=True)` CUDA graphs — raw unoptimized was 1.21), **~6GB VRAM**, 48kHz, streams (`generate_stream`). Venv+cache at `irv-ml1:/home/lkraven/dots-tts` (~10GB). **Operator design calls:** SGLang Omni serving (OpenAI `/v1/audio/speech`), transcribe-refs-first, `soar` variant. ⚠ Omni serves soar but its continuous-batching + streaming opts are **mf-only** (soar = single-request) — non-issue for ratatoskr's single-consumer RP surface. **KEY FINDING — dots is highly sensitive to an accurate AND sentence-bounded reference transcript:** mismatched transcript → 0.16s collapse; over-long/messy transcript → reference-audio BLEEDS as an output prefix; mid-clause trim → dangling-word leak (glados "we'll", emmie "And,"). RECIPE (baked into `voices/derive.py`): trim ref to a clean ~6–10s clip ending on a sentence boundary + accurate transcript of exactly that clip. **CANONICAL VOICE CORPUS** stood up in eshpfi `voices/` (operator idea): engine-agnostic `canonical/<v>.wav` + `transcripts/<v>.txt` → per-engine ref sets DERIVED by `derive.py` reading `engines.yaml` profiles (dots/chatterbox/zonos); canonical wavs git-tracked (small/curated), `derived/` gitignored. **4 voices optimized + verified CLEAN for dots: donut, glados, emmie, miranda** (glados canonical is low-SR 16kHz — flagged upgrade candidate). ⚠ GPU GOTCHA: irv-ml1 native CUDA orders **A6000=device0** (ComfyUI-full) — pin the 3090 with `CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0`; and `PYTORCH_CUDA_ALLOC_CONF=expandable_segments` CONFLICTS with `optimize=True` CUDA graphs (curr_block error). Booths: `dots-vs-chatterbox`, `dots-voices-optimized`. **PENDING: operator A/B ear-verdict → Phase 2** (containerize SGLang Omni serving dots.tts-soar on the 3090 alongside chatterbox; ratatoskr client cutover to `/v1/audio/speech`); wrapper-vs-Omni serving-layer choice deferred to Phase 2. **OPEN operator call:** corpus home = eshpfi `voices/` (my rec) vs spin-out `vh/voice-corpus`. [[reference_chatterbox_fast_repo]] [[reference_zonos_tts_stack]] [[reference_verify_hf_repo_ids_before_pull]]
|
||||
- `[2026-08-09→10]` **dots.tts (rednote-hilab) TTS burn-in on irv-ml1 + canonical voice corpus built (`voices/`).** Operator-directed eval to potentially replace chatterbox-fast. **dots.tts VERIFIED real** (canonical HF ns `dots-studio/`, `rednote-hilab/dots.tts-*` redirects there; Apache-2.0; PyPI `dots.tts` 0.2.1; 2B continuous-AR = semantic enc + Qwen2.5-1.5B LLM + flow-matching acoustic head over 48kHz AudioVAE; zero-shot clone from wav+transcript). **Runs on Ampere 3090** (sm_86, bf16, no fp8 dep); **optimized RTF 0.22** at num_steps=10 (`from_pretrained(..., optimize=True)` CUDA graphs — raw unoptimized was 1.21), **~6GB VRAM**, 48kHz, streams (`generate_stream`). Venv+cache at `irv-ml1:/home/lkraven/dots-tts` (~10GB). **Operator design calls:** SGLang Omni serving (OpenAI `/v1/audio/speech`), transcribe-refs-first, `soar` variant. ⚠ Omni serves soar but its continuous-batching + streaming opts are **mf-only** (soar = single-request) — non-issue for ratatoskr's single-consumer RP surface. **KEY FINDING — dots is highly sensitive to an accurate AND sentence-bounded reference transcript:** mismatched transcript → 0.16s collapse; over-long/messy transcript → reference-audio BLEEDS as an output prefix; mid-clause trim → dangling-word leak (glados "we'll", emmie "And,"). RECIPE (baked into `voices/derive.py`): trim ref to a clean ~6–10s clip ending on a sentence boundary + accurate transcript of exactly that clip. **CANONICAL VOICE CORPUS** stood up in eshpfi `voices/` (operator idea): engine-agnostic `canonical/<v>.wav` + `transcripts/<v>.txt` → per-engine ref sets DERIVED by `derive.py` reading `engines.yaml` profiles (dots/chatterbox/zonos); canonical wavs git-tracked (small/curated), `derived/` gitignored. **4 voices optimized + verified CLEAN for dots: donut, glados, emmie, miranda** (glados canonical is low-SR 16kHz — flagged upgrade candidate). ⚠ GPU GOTCHA: irv-ml1 native CUDA orders **A6000=device0** (ComfyUI-full) — pin the 3090 with `CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0`; and `PYTORCH_CUDA_ALLOC_CONF=expandable_segments` CONFLICTS with `optimize=True` CUDA graphs (curr_block error). Booths: `dots-vs-chatterbox`, `dots-voices-optimized`. **SHIPPED 2026-08-10:** operator A/B verdict "dots is very good" → containerized as a **thin FastAPI wrapper over DotsTtsRuntime** (chosen over SGLang Omni — Omni's batching is mf-only, unneeded for ratatoskr's single consumer; wrapper is SERIALIZED one-gen-at-a-time via a threading.Lock, Omni+mf = parked API-compatible escalation if multi-consumer ever lands). **LIVE on irv-ml1:8198** (`local/dots-tts:v1`, OpenAI `/v1/audio/speech` + `/health` + `/v1/voices`, container healthy, both stream + non-stream verified CLEAN, 4 voices donut/glados/emmie/miranda) alongside chatterbox :8197 (nothing repointed). Stack = `stacks/dots-tts/` (Dockerfile/app.py/compose/.env.example/README). ⚠ CONTAINER GOTCHA: `optimize=True` (torch.compile/inductor/triton) needs a **C compiler at RUNTIME** — slim image must `apt install build-essential` or model-load dies "Failed to find C compiler" (host venv had gcc ambient, masking it); persist `TORCHINDUCTOR_CACHE_DIR` to a mounted dir or every restart re-JITs ~5min. Corpus home = eshpfi `voices/` (operator ruled keep-here). **REMAINING: ratatoskr client cutover** to :8198 `/v1/audio/speech` (Phase-2 tail, peer-coupled — draft the ask). [[reference_chatterbox_fast_repo]] [[reference_zonos_tts_stack]] [[reference_verify_hf_repo_ids_before_pull]]
|
||||
|
||||
- `[2026-08-08]` **worldtree-dev #400 CLOSED → fiction-decomp snapshot cleared from nh3-dev.** worldtree-dev signaled #400 done (shipped v1.0.0b185; exact-lexical efficacy 79%→12% on ratatoskr's gate, brokkr no-harm bracket green both ends; the snapshot served 4 probe rounds — rank decomposition, promoted-vs-gold annotation, tie-set falsification, A0/A1/A2 mechanism probe). Cleared `~/snapshots/worldtree-400-fiction-decomp` (208M: chroma + manifest/provenance/stamp) — a read-only rsync copy of PERSONAL Worldtree's Chroma (source on corviduo-dev, so safe to remove). **LEFT INTACT:** `rex393-fiction-index`/`rex393-fiction-snapshot` (separate operator KEEP word, unchanged) + `r42-gate-*`. No config deltas rode this train. Only remaining non-blocking await = ratatoskr-dev's chatterbox-fast knob revert. Replied confirming (`01KZJ9GMCC…`).
|
||||
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# dots-tts stack tunables. Copy to `.env` on irv-ml1 before deploying.
|
||||
|
||||
# ── image ────────────────────────────────────────────────────────────
|
||||
DOTS_TAG=v1
|
||||
|
||||
# ── network ──────────────────────────────────────────────────────────
|
||||
DOTS_BIND=0.0.0.0
|
||||
DOTS_PORT=8198
|
||||
|
||||
# ── GPU ──────────────────────────────────────────────────────────────
|
||||
# 0 = 3090 in Docker (PCI order), co-resident with chatterbox-fast. soar needs
|
||||
# ~6GB; the 3090 has headroom with Zonos parked down.
|
||||
DOTS_GPU_DEVICES=0
|
||||
|
||||
# ── model / inference ────────────────────────────────────────────────
|
||||
DOTS_MODEL=dots-studio/dots.tts-soar
|
||||
DOTS_DEFAULT_VOICE=donut
|
||||
DOTS_NUM_STEPS=10 # 10 = full quality @ RTF ~0.22; lower = faster/rougher
|
||||
DOTS_GUIDANCE_SCALE=1.2
|
||||
|
||||
# ── host mounts ──────────────────────────────────────────────────────
|
||||
# HF cache holding the downloaded soar snapshot (~5GB). Reuse the burn-in cache.
|
||||
DOTS_HFCACHE_DIR=/home/lkraven/dots-tts/hf_cache
|
||||
# dots-derived voice references (derive.py dots -> derived/dots/<name>.{wav,txt}).
|
||||
# Burn-in points at the corpus output directly; for a durable deploy, copy the
|
||||
# derived set to /opt/docker/conf/dots-tts/voices and point here.
|
||||
DOTS_VOICES_HOST_DIR=/home/lkraven/voice-corpus/derived/dots
|
||||
@@ -0,0 +1,34 @@
|
||||
# dots.tts OpenAI-compatible TTS server (thin FastAPI over DotsTtsRuntime).
|
||||
# GPU access is via `runtime: nvidia` at run time (torch ships its own CUDA
|
||||
# runtime; no CUDA toolkit / nvcc needed to build — the model uses no custom
|
||||
# compiled kernels, confirmed on the irv-ml1 venv).
|
||||
FROM python:3.11-slim
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libsndfile1 ffmpeg git curl ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN pip install --no-cache-dir uv
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Pin torch + deps to dots.tts upstream recommended constraints (the same set the
|
||||
# irv-ml1 venv installed against). ADD caches on the URL contents.
|
||||
ADD https://raw.githubusercontent.com/rednote-hilab/dots.tts/main/constraints/recommended.txt /tmp/rec.txt
|
||||
RUN uv pip install --system -c /tmp/rec.txt \
|
||||
dots.tts soundfile fastapi "uvicorn[standard]"
|
||||
|
||||
# C compiler for the RUNTIME (not build): optimize=True drives torch.compile /
|
||||
# inductor / triton, which JIT-compile kernels via gcc on model load. Without it
|
||||
# the runtime dies with "Failed to find C compiler". Placed after the pip layer
|
||||
# so it doesn't invalidate the expensive torch install cache.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY app.py /app/app.py
|
||||
|
||||
# Persist the inductor compile cache on the mounted (rw) HF cache so kernel
|
||||
# JIT doesn't re-run on every container restart (~70s warmup otherwise).
|
||||
ENV HF_HOME=/hf_cache DOTS_PORT=8198 CC=gcc CXX=g++ TORCHINDUCTOR_CACHE_DIR=/hf_cache/inductor
|
||||
EXPOSE 8198
|
||||
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8198"]
|
||||
@@ -0,0 +1,64 @@
|
||||
# dots-tts
|
||||
|
||||
OpenAI-compatible zero-shot voice-clone TTS over **dots.tts** (rednote-hilab) —
|
||||
2B continuous-AR, native **48kHz**, `optimize=True` CUDA graphs → **RTF ~0.22** on
|
||||
the irv-ml1 3090. Thin FastAPI wrapper around `DotsTtsRuntime` (chosen over SGLang
|
||||
Omni: Omni's batching is MeanFlow-only and unneeded for a single consumer; the raw
|
||||
runtime already streams at the same RTF and is ~100 lines we control).
|
||||
|
||||
- **Host:** irv-ml1, port **8198** (chatterbox-fast is :8197 — they co-reside on the 3090)
|
||||
- **Model:** `dots-studio/dots.tts-soar`, bf16, num_steps=10
|
||||
- **Voices:** every `<name>.wav` (+ `<name>.txt` transcript) in the mounted voices dir,
|
||||
sourced from the [`voices/`](../../voices/) canonical corpus via `derive.py dots`.
|
||||
|
||||
## API
|
||||
|
||||
```
|
||||
GET /health -> {status, model, sample_rate, voices[]}
|
||||
GET /v1/voices -> {voices[]}
|
||||
POST /v1/audio/speech -> audio
|
||||
body: {input, voice, response_format?("wav"|"pcm"), stream?}
|
||||
```
|
||||
|
||||
`stream:true` returns a WAV stream (placeholder-header + PCM frames, 48kHz mono
|
||||
s16le) — the same shape the Zonos/chatterbox consumers already handle. Non-stream
|
||||
returns a complete WAV (or raw PCM with `response_format:"pcm"`).
|
||||
|
||||
```bash
|
||||
curl -X POST http://10.100.79.3:8198/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"input":"Well, look who finally showed up.","voice":"glados"}' \
|
||||
--output out.wav
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Reference sets come from the canonical corpus, not this stack — derive then point
|
||||
the mount at them:
|
||||
|
||||
```bash
|
||||
# 1. produce dots refs from the corpus (on a box with the whisper venv):
|
||||
python voices/derive.py dots # -> voices/derived/dots/*.{wav,txt}
|
||||
|
||||
# 2. build + run on irv-ml1 (cp .env.example .env first; adjust mounts):
|
||||
scripts/deploy-stack.sh irv-ml1 dots-tts # or, on the host:
|
||||
docker compose build && docker compose up -d
|
||||
```
|
||||
|
||||
The model (~5GB) is **not** baked — it's read from the mounted `HF_HOME`
|
||||
(`DOTS_HFCACHE_DIR`). First boot downloads it there if absent.
|
||||
|
||||
## Voice cloning gotcha
|
||||
|
||||
dots.tts clones from `(reference wav + its transcript)` and **leaks reference
|
||||
audio into the output** if the transcript is inaccurate or ends mid-clause. The
|
||||
`voices/` corpus + `derive.py` handle this (sentence-bounded trim + accurate
|
||||
transcript); don't hand this server a raw reference wav without a matching `.txt`.
|
||||
|
||||
## Notes
|
||||
|
||||
- **GPU:** `NVIDIA_VISIBLE_DEVICES=0` = the 3090 in Docker (PCI order). `optimize=True`
|
||||
is **incompatible with `PYTORCH_CUDA_ALLOC_CONF=expandable_segments`** (CUDA-graph
|
||||
capture error) — don't set it.
|
||||
- **Variants:** `dots.tts-mf` (MeanFlow, faster) is a drop-in via `DOTS_MODEL`; soar
|
||||
is the quality pick and single-consumer doesn't need mf's batching.
|
||||
@@ -0,0 +1,147 @@
|
||||
"""OpenAI-compatible /v1/audio/speech server over dots.tts (rednote-hilab).
|
||||
|
||||
Thin wrapper around DotsTtsRuntime — chosen over SGLang Omni because Omni's edge
|
||||
(continuous batching) is MeanFlow-only and unneeded for a single-consumer surface,
|
||||
while the raw runtime with optimize=True already streams at RTF ~0.22 on our 3090.
|
||||
|
||||
Voice registry: every <name>.wav (+ optional <name>.txt transcript) under
|
||||
DOTS_VOICES_DIR becomes a callable voice. dots.tts REQUIRES an accurate,
|
||||
sentence-bounded transcript to clone cleanly (see the voices/ corpus) — the .txt
|
||||
is that transcript; without it the model leaks reference audio into the output.
|
||||
"""
|
||||
import io
|
||||
import os
|
||||
import glob
|
||||
import struct
|
||||
import threading
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi.responses import Response, StreamingResponse
|
||||
from pydantic import BaseModel
|
||||
|
||||
from dots_tts.runtime import DotsTtsRuntime
|
||||
|
||||
MODEL = os.environ.get("DOTS_MODEL", "dots-studio/dots.tts-soar")
|
||||
VOICES_DIR = os.environ.get("DOTS_VOICES_DIR", "/voices")
|
||||
DEFAULT_VOICE = os.environ.get("DOTS_DEFAULT_VOICE", "donut")
|
||||
NUM_STEPS = int(os.environ.get("DOTS_NUM_STEPS", "10"))
|
||||
GUIDANCE = float(os.environ.get("DOTS_GUIDANCE_SCALE", "1.2"))
|
||||
SAMPLE_RATE = 48000 # dots.tts fixed native output
|
||||
|
||||
app = FastAPI(title="dots.tts")
|
||||
_rt = None
|
||||
_voices: dict = {}
|
||||
# One DotsTtsRuntime, and it is NOT safe to call concurrently (CUDA-graph capture
|
||||
# + shared state). uvicorn runs sync endpoints in a threadpool, so we must
|
||||
# serialize generation ourselves: requests queue and run one at a time. This is
|
||||
# the deliberate trade for the thin-wrapper design — no vLLM-style continuous
|
||||
# batching. If concurrency demand appears, swap the backend to SGLang Omni + the
|
||||
# mf variant behind this same API (see README).
|
||||
_gen_lock = threading.Lock()
|
||||
|
||||
|
||||
def _load_voices() -> dict:
|
||||
reg = {}
|
||||
for wav in sorted(glob.glob(os.path.join(VOICES_DIR, "*.wav"))):
|
||||
name = os.path.splitext(os.path.basename(wav))[0]
|
||||
txt = os.path.splitext(wav)[0] + ".txt"
|
||||
reg[name] = {
|
||||
"wav": wav,
|
||||
"text": open(txt).read().strip() if os.path.exists(txt) else "",
|
||||
}
|
||||
return reg
|
||||
|
||||
|
||||
@app.on_event("startup")
|
||||
def _startup():
|
||||
global _rt, _voices
|
||||
_voices = _load_voices()
|
||||
_rt = DotsTtsRuntime.from_pretrained(MODEL, precision="bfloat16", optimize=True)
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
def health():
|
||||
return {
|
||||
"status": "ok" if _rt is not None else "loading",
|
||||
"model": MODEL,
|
||||
"sample_rate": SAMPLE_RATE,
|
||||
"voices": sorted(_voices),
|
||||
}
|
||||
|
||||
|
||||
@app.get("/v1/voices")
|
||||
def list_voices():
|
||||
return {"voices": sorted(_voices)}
|
||||
|
||||
|
||||
class SpeechRequest(BaseModel):
|
||||
input: str
|
||||
voice: str = DEFAULT_VOICE
|
||||
model: str | None = None # accepted, ignored (single served model)
|
||||
response_format: str = "wav" # wav | pcm
|
||||
stream: bool = False
|
||||
|
||||
|
||||
def _to_pcm16(audio: np.ndarray) -> bytes:
|
||||
return np.round(np.clip(audio, -1.0, 1.0) * 32767.0).astype("<i2").tobytes()
|
||||
|
||||
|
||||
def _wav_bytes(pcm: bytes) -> bytes:
|
||||
buf = io.BytesIO()
|
||||
w = wave.open(buf, "wb")
|
||||
w.setnchannels(1)
|
||||
w.setsampwidth(2)
|
||||
w.setframerate(SAMPLE_RATE)
|
||||
w.writeframes(pcm)
|
||||
w.close()
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def _streaming_wav_header() -> bytes:
|
||||
"""WAV header with placeholder (max) sizes — lets a client start playing the
|
||||
stream before the total length is known (the pattern the Zonos/chatterbox
|
||||
consumers already expect)."""
|
||||
return (
|
||||
b"RIFF" + struct.pack("<I", 0xFFFFFFFF) + b"WAVE"
|
||||
+ b"fmt " + struct.pack("<IHHIIHH", 16, 1, 1, SAMPLE_RATE, SAMPLE_RATE * 2, 2, 16)
|
||||
+ b"data" + struct.pack("<I", 0xFFFFFFFF)
|
||||
)
|
||||
|
||||
|
||||
@app.post("/v1/audio/speech")
|
||||
def speech(req: SpeechRequest):
|
||||
rt = _rt
|
||||
if rt is None:
|
||||
raise HTTPException(503, "model still loading")
|
||||
if req.voice not in _voices:
|
||||
raise HTTPException(404, f"unknown voice '{req.voice}'; have {sorted(_voices)}")
|
||||
if not req.input.strip():
|
||||
raise HTTPException(400, "empty input")
|
||||
|
||||
v = _voices[req.voice]
|
||||
kw = dict(
|
||||
prompt_audio_path=v["wav"],
|
||||
prompt_text=v["text"],
|
||||
num_steps=NUM_STEPS,
|
||||
guidance_scale=GUIDANCE,
|
||||
normalize_text=True,
|
||||
)
|
||||
|
||||
if req.stream:
|
||||
def gen():
|
||||
# Hold the lock for the whole stream — a second generation on the
|
||||
# shared runtime mid-stream would corrupt both.
|
||||
with _gen_lock:
|
||||
yield _streaming_wav_header()
|
||||
for chunk in rt.generate_stream(text=req.input, **kw):
|
||||
yield _to_pcm16(chunk.float().cpu().squeeze().numpy())
|
||||
return StreamingResponse(gen(), media_type="audio/wav")
|
||||
|
||||
with _gen_lock:
|
||||
res = rt.generate(text=req.input, **kw)
|
||||
pcm = _to_pcm16(res["audio"].float().cpu().squeeze().numpy())
|
||||
if req.response_format == "pcm":
|
||||
return Response(pcm, media_type="audio/L16;rate=48000")
|
||||
return Response(_wav_bytes(pcm), media_type="audio/wav")
|
||||
@@ -0,0 +1,50 @@
|
||||
# dots.tts — OpenAI-compatible 48kHz zero-shot voice-clone TTS (rednote-hilab).
|
||||
# Deployed on irv-ml1 ALONGSIDE chatterbox-fast (:8197) for burn-in; both share
|
||||
# the 3090. Mirrors the chatterbox-fast sibling: GPU via `runtime: nvidia` +
|
||||
# NVIDIA_VISIBLE_DEVICES, host IP:port (no traefik). All tunables in .env.
|
||||
#
|
||||
# GPU note: in Docker, NVIDIA_VISIBLE_DEVICES=0 resolves to the 3090 (PCI order),
|
||||
# same as chatterbox-fast — NOT the A6000 that *native* CUDA calls device 0 on
|
||||
# this host. CUDA_DEVICE_ORDER=PCI_BUS_ID is set belt-and-suspenders.
|
||||
|
||||
services:
|
||||
dots-tts:
|
||||
image: local/dots-tts:${DOTS_TAG:-v1}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
container_name: dots-tts
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${DOTS_BIND:-0.0.0.0}:${DOTS_PORT:-8198}:8198"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${DOTS_GPU_DEVICES:-0}
|
||||
- CUDA_DEVICE_ORDER=PCI_BUS_ID
|
||||
- HF_HOME=/hf_cache
|
||||
- DOTS_MODEL=${DOTS_MODEL:-dots-studio/dots.tts-soar}
|
||||
- DOTS_VOICES_DIR=/voices
|
||||
- DOTS_DEFAULT_VOICE=${DOTS_DEFAULT_VOICE:-donut}
|
||||
- DOTS_NUM_STEPS=${DOTS_NUM_STEPS:-10}
|
||||
- DOTS_GUIDANCE_SCALE=${DOTS_GUIDANCE_SCALE:-1.2}
|
||||
- DOTS_PORT=8198
|
||||
volumes:
|
||||
# HF model cache — reuse the already-downloaded soar snapshot (~5GB).
|
||||
- ${DOTS_HFCACHE_DIR:-/home/lkraven/dots-tts/hf_cache}:/hf_cache
|
||||
# Voice references — the dots-derived set from the voices/ corpus
|
||||
# (derive.py dots -> derived/dots/<name>.{wav,txt}). Read-only.
|
||||
- ${DOTS_VOICES_HOST_DIR:-/home/lkraven/voice-corpus/derived/dots}:/voices:ro
|
||||
healthcheck:
|
||||
# /health returns {"status":"ok",...} only once the model has loaded.
|
||||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; b=urllib.request.urlopen('http://127.0.0.1:8198/health',timeout=5).read(); sys.exit(0 if b'\\\"status\\\":\\\"ok\\\"' in b.replace(b' ',b'') else 1)\""]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# Model load + optimize=True CUDA-graph warmup measured ~70s; generous.
|
||||
start_period: 180s
|
||||
labels:
|
||||
- homepage.group=AI - Speech (TTS)
|
||||
- homepage.name=dots.tts
|
||||
- homepage.icon=mdi-account-voice
|
||||
- homepage.description=Continuous-AR 48kHz zero-shot voice clone (irv-ml1)
|
||||
- homepage.href=http://10.100.79.3:${DOTS_PORT:-8198}
|
||||
Reference in New Issue
Block a user