feat(dots-tts): ship OpenAI-compatible dots.tts TTS stack on irv-ml1:8198
Thin FastAPI wrapper over DotsTtsRuntime (soar, optimize=True, RTF ~0.22), serialized single-consumer; OpenAI /v1/audio/speech (stream + non-stream), voices from the voices/ corpus derived set. Live + healthy alongside chatterbox-fast on the 3090; nothing repointed. Dockerfile needs build-essential (torch.compile/inductor JITs via gcc at runtime) + persisted inductor cache. Remaining Phase-2: ratatoskr client cutover.
This commit is contained in:
@@ -119,6 +119,8 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i
|
|||||||
|
|
||||||
- **EVIDENCE HOLD (partial):** WT #394 index-row half LIFTED + swept (kb 8,230→2,876); the **FILE half STILL STANDS** — do NOT delete the on-disk generation dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`). Also un-actioned worldtree-side: the `reconcile --repair` rc=1 git-staging bug (phantom `docs/specs/saga-taxonomy-redesign-plan.md`) — flagged, their code.
|
- **EVIDENCE HOLD (partial):** WT #394 index-row half LIFTED + swept (kb 8,230→2,876); the **FILE half STILL STANDS** — do NOT delete the on-disk generation dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`). Also un-actioned worldtree-side: the `reconcile --repair` rc=1 git-staging bug (phantom `docs/specs/saga-taxonomy-redesign-plan.md`) — flagged, their code.
|
||||||
|
|
||||||
|
- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover** — draft the ask to repoint ratatoskr's RP TTS to irv-ml1:8198 `/v1/audio/speech` (Phase-2 tail; dots container LIVE, see Recent decisions). (2) **LFM2.5-2.6b bake-off alias for brokkr-smithy-dev** — arch confirmed (Lfm2ForCausalLM, vLLM 0.24.0 on ana-ml2 supports it, no bump), replied w/ alias `lfm2.5-2.6b` + plan (non-prod, vendor sampling temp0.1/topk50/reppen1.1, reasoning-channel content-non-empty smoke, LFM Open License = eval-only pending operator ruling); **stand-up QUEUED** (thread `01KZNA4MKPDR…`, brokkr said no-rush).
|
||||||
|
|
||||||
- **OPEN FOLLOW-UPS:** chatterbox-fast **deployed flat-build-context vs package-repo divergence** (reconcile so a repo rebuild matches deploy — repo commit `6bc7bf0` has cap+norm_loudness in package layout; deployed is flat cap-only). Standing/parked: CI-flip runner-auth research, gitea `REQUIRE_SIGNIN_VIEW=false`, #363 research-wing ingest (no deadline), zonos-gateway CI-wire.
|
- **OPEN FOLLOW-UPS:** chatterbox-fast **deployed flat-build-context vs package-repo divergence** (reconcile so a repo rebuild matches deploy — repo commit `6bc7bf0` has cap+norm_loudness in package layout; deployed is flat cap-only). Standing/parked: CI-flip runner-auth research, gitea `REQUIRE_SIGNIN_VIEW=false`, #363 research-wing ingest (no deadline), zonos-gateway CI-wire.
|
||||||
|
|
||||||
**althing monitor** ARMED (handle `infra-ops`, task `b1uat4njl`; herald up). ⚠️ Re-arm ONLY after a real FIRE (`<task-notification> completed rc0`), NEVER after a plain operator turn (bounces rc3); spawn `althing-wake-listener` as its OWN `run_in_background` task, **NEVER chained with `&`/`&&`** (orphans it to init — bit me this session; reclaim with `althing-cli stop-monitor` then re-arm cleanly).
|
**althing monitor** ARMED (handle `infra-ops`, task `b1uat4njl`; herald up). ⚠️ Re-arm ONLY after a real FIRE (`<task-notification> completed rc0`), NEVER after a plain operator turn (bounces rc3); spawn `althing-wake-listener` as its OWN `run_in_background` task, **NEVER chained with `&`/`&&`** (orphans it to init — bit me this session; reclaim with `althing-cli stop-monitor` then re-arm cleanly).
|
||||||
@@ -128,7 +130,7 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i
|
|||||||
|
|
||||||
## Recent decisions
|
## Recent decisions
|
||||||
|
|
||||||
- `[2026-08-09→10]` **dots.tts (rednote-hilab) TTS burn-in on irv-ml1 + canonical voice corpus built (`voices/`).** Operator-directed eval to potentially replace chatterbox-fast. **dots.tts VERIFIED real** (canonical HF ns `dots-studio/`, `rednote-hilab/dots.tts-*` redirects there; Apache-2.0; PyPI `dots.tts` 0.2.1; 2B continuous-AR = semantic enc + Qwen2.5-1.5B LLM + flow-matching acoustic head over 48kHz AudioVAE; zero-shot clone from wav+transcript). **Runs on Ampere 3090** (sm_86, bf16, no fp8 dep); **optimized RTF 0.22** at num_steps=10 (`from_pretrained(..., optimize=True)` CUDA graphs — raw unoptimized was 1.21), **~6GB VRAM**, 48kHz, streams (`generate_stream`). Venv+cache at `irv-ml1:/home/lkraven/dots-tts` (~10GB). **Operator design calls:** SGLang Omni serving (OpenAI `/v1/audio/speech`), transcribe-refs-first, `soar` variant. ⚠ Omni serves soar but its continuous-batching + streaming opts are **mf-only** (soar = single-request) — non-issue for ratatoskr's single-consumer RP surface. **KEY FINDING — dots is highly sensitive to an accurate AND sentence-bounded reference transcript:** mismatched transcript → 0.16s collapse; over-long/messy transcript → reference-audio BLEEDS as an output prefix; mid-clause trim → dangling-word leak (glados "we'll", emmie "And,"). RECIPE (baked into `voices/derive.py`): trim ref to a clean ~6–10s clip ending on a sentence boundary + accurate transcript of exactly that clip. **CANONICAL VOICE CORPUS** stood up in eshpfi `voices/` (operator idea): engine-agnostic `canonical/<v>.wav` + `transcripts/<v>.txt` → per-engine ref sets DERIVED by `derive.py` reading `engines.yaml` profiles (dots/chatterbox/zonos); canonical wavs git-tracked (small/curated), `derived/` gitignored. **4 voices optimized + verified CLEAN for dots: donut, glados, emmie, miranda** (glados canonical is low-SR 16kHz — flagged upgrade candidate). ⚠ GPU GOTCHA: irv-ml1 native CUDA orders **A6000=device0** (ComfyUI-full) — pin the 3090 with `CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0`; and `PYTORCH_CUDA_ALLOC_CONF=expandable_segments` CONFLICTS with `optimize=True` CUDA graphs (curr_block error). Booths: `dots-vs-chatterbox`, `dots-voices-optimized`. **PENDING: operator A/B ear-verdict → Phase 2** (containerize SGLang Omni serving dots.tts-soar on the 3090 alongside chatterbox; ratatoskr client cutover to `/v1/audio/speech`); wrapper-vs-Omni serving-layer choice deferred to Phase 2. **OPEN operator call:** corpus home = eshpfi `voices/` (my rec) vs spin-out `vh/voice-corpus`. [[reference_chatterbox_fast_repo]] [[reference_zonos_tts_stack]] [[reference_verify_hf_repo_ids_before_pull]]
|
- `[2026-08-09→10]` **dots.tts (rednote-hilab) TTS burn-in on irv-ml1 + canonical voice corpus built (`voices/`).** Operator-directed eval to potentially replace chatterbox-fast. **dots.tts VERIFIED real** (canonical HF ns `dots-studio/`, `rednote-hilab/dots.tts-*` redirects there; Apache-2.0; PyPI `dots.tts` 0.2.1; 2B continuous-AR = semantic enc + Qwen2.5-1.5B LLM + flow-matching acoustic head over 48kHz AudioVAE; zero-shot clone from wav+transcript). **Runs on Ampere 3090** (sm_86, bf16, no fp8 dep); **optimized RTF 0.22** at num_steps=10 (`from_pretrained(..., optimize=True)` CUDA graphs — raw unoptimized was 1.21), **~6GB VRAM**, 48kHz, streams (`generate_stream`). Venv+cache at `irv-ml1:/home/lkraven/dots-tts` (~10GB). **Operator design calls:** SGLang Omni serving (OpenAI `/v1/audio/speech`), transcribe-refs-first, `soar` variant. ⚠ Omni serves soar but its continuous-batching + streaming opts are **mf-only** (soar = single-request) — non-issue for ratatoskr's single-consumer RP surface. **KEY FINDING — dots is highly sensitive to an accurate AND sentence-bounded reference transcript:** mismatched transcript → 0.16s collapse; over-long/messy transcript → reference-audio BLEEDS as an output prefix; mid-clause trim → dangling-word leak (glados "we'll", emmie "And,"). RECIPE (baked into `voices/derive.py`): trim ref to a clean ~6–10s clip ending on a sentence boundary + accurate transcript of exactly that clip. **CANONICAL VOICE CORPUS** stood up in eshpfi `voices/` (operator idea): engine-agnostic `canonical/<v>.wav` + `transcripts/<v>.txt` → per-engine ref sets DERIVED by `derive.py` reading `engines.yaml` profiles (dots/chatterbox/zonos); canonical wavs git-tracked (small/curated), `derived/` gitignored. **4 voices optimized + verified CLEAN for dots: donut, glados, emmie, miranda** (glados canonical is low-SR 16kHz — flagged upgrade candidate). ⚠ GPU GOTCHA: irv-ml1 native CUDA orders **A6000=device0** (ComfyUI-full) — pin the 3090 with `CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0`; and `PYTORCH_CUDA_ALLOC_CONF=expandable_segments` CONFLICTS with `optimize=True` CUDA graphs (curr_block error). Booths: `dots-vs-chatterbox`, `dots-voices-optimized`. **SHIPPED 2026-08-10:** operator A/B verdict "dots is very good" → containerized as a **thin FastAPI wrapper over DotsTtsRuntime** (chosen over SGLang Omni — Omni's batching is mf-only, unneeded for ratatoskr's single consumer; wrapper is SERIALIZED one-gen-at-a-time via a threading.Lock, Omni+mf = parked API-compatible escalation if multi-consumer ever lands). **LIVE on irv-ml1:8198** (`local/dots-tts:v1`, OpenAI `/v1/audio/speech` + `/health` + `/v1/voices`, container healthy, both stream + non-stream verified CLEAN, 4 voices donut/glados/emmie/miranda) alongside chatterbox :8197 (nothing repointed). Stack = `stacks/dots-tts/` (Dockerfile/app.py/compose/.env.example/README). ⚠ CONTAINER GOTCHA: `optimize=True` (torch.compile/inductor/triton) needs a **C compiler at RUNTIME** — slim image must `apt install build-essential` or model-load dies "Failed to find C compiler" (host venv had gcc ambient, masking it); persist `TORCHINDUCTOR_CACHE_DIR` to a mounted dir or every restart re-JITs ~5min. Corpus home = eshpfi `voices/` (operator ruled keep-here). **REMAINING: ratatoskr client cutover** to :8198 `/v1/audio/speech` (Phase-2 tail, peer-coupled — draft the ask). [[reference_chatterbox_fast_repo]] [[reference_zonos_tts_stack]] [[reference_verify_hf_repo_ids_before_pull]]
|
||||||
|
|
||||||
- `[2026-08-08]` **worldtree-dev #400 CLOSED → fiction-decomp snapshot cleared from nh3-dev.** worldtree-dev signaled #400 done (shipped v1.0.0b185; exact-lexical efficacy 79%→12% on ratatoskr's gate, brokkr no-harm bracket green both ends; the snapshot served 4 probe rounds — rank decomposition, promoted-vs-gold annotation, tie-set falsification, A0/A1/A2 mechanism probe). Cleared `~/snapshots/worldtree-400-fiction-decomp` (208M: chroma + manifest/provenance/stamp) — a read-only rsync copy of PERSONAL Worldtree's Chroma (source on corviduo-dev, so safe to remove). **LEFT INTACT:** `rex393-fiction-index`/`rex393-fiction-snapshot` (separate operator KEEP word, unchanged) + `r42-gate-*`. No config deltas rode this train. Only remaining non-blocking await = ratatoskr-dev's chatterbox-fast knob revert. Replied confirming (`01KZJ9GMCC…`).
|
- `[2026-08-08]` **worldtree-dev #400 CLOSED → fiction-decomp snapshot cleared from nh3-dev.** worldtree-dev signaled #400 done (shipped v1.0.0b185; exact-lexical efficacy 79%→12% on ratatoskr's gate, brokkr no-harm bracket green both ends; the snapshot served 4 probe rounds — rank decomposition, promoted-vs-gold annotation, tie-set falsification, A0/A1/A2 mechanism probe). Cleared `~/snapshots/worldtree-400-fiction-decomp` (208M: chroma + manifest/provenance/stamp) — a read-only rsync copy of PERSONAL Worldtree's Chroma (source on corviduo-dev, so safe to remove). **LEFT INTACT:** `rex393-fiction-index`/`rex393-fiction-snapshot` (separate operator KEEP word, unchanged) + `r42-gate-*`. No config deltas rode this train. Only remaining non-blocking await = ratatoskr-dev's chatterbox-fast knob revert. Replied confirming (`01KZJ9GMCC…`).
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
# dots-tts stack tunables. Copy to `.env` on irv-ml1 before deploying.
|
||||||
|
|
||||||
|
# ── image ────────────────────────────────────────────────────────────
|
||||||
|
DOTS_TAG=v1
|
||||||
|
|
||||||
|
# ── network ──────────────────────────────────────────────────────────
|
||||||
|
DOTS_BIND=0.0.0.0
|
||||||
|
DOTS_PORT=8198
|
||||||
|
|
||||||
|
# ── GPU ──────────────────────────────────────────────────────────────
|
||||||
|
# 0 = 3090 in Docker (PCI order), co-resident with chatterbox-fast. soar needs
|
||||||
|
# ~6GB; the 3090 has headroom with Zonos parked down.
|
||||||
|
DOTS_GPU_DEVICES=0
|
||||||
|
|
||||||
|
# ── model / inference ────────────────────────────────────────────────
|
||||||
|
DOTS_MODEL=dots-studio/dots.tts-soar
|
||||||
|
DOTS_DEFAULT_VOICE=donut
|
||||||
|
DOTS_NUM_STEPS=10 # 10 = full quality @ RTF ~0.22; lower = faster/rougher
|
||||||
|
DOTS_GUIDANCE_SCALE=1.2
|
||||||
|
|
||||||
|
# ── host mounts ──────────────────────────────────────────────────────
|
||||||
|
# HF cache holding the downloaded soar snapshot (~5GB). Reuse the burn-in cache.
|
||||||
|
DOTS_HFCACHE_DIR=/home/lkraven/dots-tts/hf_cache
|
||||||
|
# dots-derived voice references (derive.py dots -> derived/dots/<name>.{wav,txt}).
|
||||||
|
# Burn-in points at the corpus output directly; for a durable deploy, copy the
|
||||||
|
# derived set to /opt/docker/conf/dots-tts/voices and point here.
|
||||||
|
DOTS_VOICES_HOST_DIR=/home/lkraven/voice-corpus/derived/dots
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# dots.tts OpenAI-compatible TTS server (thin FastAPI over DotsTtsRuntime).
|
||||||
|
# GPU access is via `runtime: nvidia` at run time (torch ships its own CUDA
|
||||||
|
# runtime; no CUDA toolkit / nvcc needed to build — the model uses no custom
|
||||||
|
# compiled kernels, confirmed on the irv-ml1 venv).
|
||||||
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
libsndfile1 ffmpeg git curl ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN pip install --no-cache-dir uv
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Pin torch + deps to dots.tts upstream recommended constraints (the same set the
|
||||||
|
# irv-ml1 venv installed against). ADD caches on the URL contents.
|
||||||
|
ADD https://raw.githubusercontent.com/rednote-hilab/dots.tts/main/constraints/recommended.txt /tmp/rec.txt
|
||||||
|
RUN uv pip install --system -c /tmp/rec.txt \
|
||||||
|
dots.tts soundfile fastapi "uvicorn[standard]"
|
||||||
|
|
||||||
|
# C compiler for the RUNTIME (not build): optimize=True drives torch.compile /
|
||||||
|
# inductor / triton, which JIT-compile kernels via gcc on model load. Without it
|
||||||
|
# the runtime dies with "Failed to find C compiler". Placed after the pip layer
|
||||||
|
# so it doesn't invalidate the expensive torch install cache.
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY app.py /app/app.py
|
||||||
|
|
||||||
|
# Persist the inductor compile cache on the mounted (rw) HF cache so kernel
|
||||||
|
# JIT doesn't re-run on every container restart (~70s warmup otherwise).
|
||||||
|
ENV HF_HOME=/hf_cache DOTS_PORT=8198 CC=gcc CXX=g++ TORCHINDUCTOR_CACHE_DIR=/hf_cache/inductor
|
||||||
|
EXPOSE 8198
|
||||||
|
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8198"]
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# dots-tts
|
||||||
|
|
||||||
|
OpenAI-compatible zero-shot voice-clone TTS over **dots.tts** (rednote-hilab) —
|
||||||
|
2B continuous-AR, native **48kHz**, `optimize=True` CUDA graphs → **RTF ~0.22** on
|
||||||
|
the irv-ml1 3090. Thin FastAPI wrapper around `DotsTtsRuntime` (chosen over SGLang
|
||||||
|
Omni: Omni's batching is MeanFlow-only and unneeded for a single consumer; the raw
|
||||||
|
runtime already streams at the same RTF and is ~100 lines we control).
|
||||||
|
|
||||||
|
- **Host:** irv-ml1, port **8198** (chatterbox-fast is :8197 — they co-reside on the 3090)
|
||||||
|
- **Model:** `dots-studio/dots.tts-soar`, bf16, num_steps=10
|
||||||
|
- **Voices:** every `<name>.wav` (+ `<name>.txt` transcript) in the mounted voices dir,
|
||||||
|
sourced from the [`voices/`](../../voices/) canonical corpus via `derive.py dots`.
|
||||||
|
|
||||||
|
## API
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /health -> {status, model, sample_rate, voices[]}
|
||||||
|
GET /v1/voices -> {voices[]}
|
||||||
|
POST /v1/audio/speech -> audio
|
||||||
|
body: {input, voice, response_format?("wav"|"pcm"), stream?}
|
||||||
|
```
|
||||||
|
|
||||||
|
`stream:true` returns a WAV stream (placeholder-header + PCM frames, 48kHz mono
|
||||||
|
s16le) — the same shape the Zonos/chatterbox consumers already handle. Non-stream
|
||||||
|
returns a complete WAV (or raw PCM with `response_format:"pcm"`).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://10.100.79.3:8198/v1/audio/speech \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"input":"Well, look who finally showed up.","voice":"glados"}' \
|
||||||
|
--output out.wav
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deploy
|
||||||
|
|
||||||
|
Reference sets come from the canonical corpus, not this stack — derive then point
|
||||||
|
the mount at them:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. produce dots refs from the corpus (on a box with the whisper venv):
|
||||||
|
python voices/derive.py dots # -> voices/derived/dots/*.{wav,txt}
|
||||||
|
|
||||||
|
# 2. build + run on irv-ml1 (cp .env.example .env first; adjust mounts):
|
||||||
|
scripts/deploy-stack.sh irv-ml1 dots-tts # or, on the host:
|
||||||
|
docker compose build && docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
The model (~5GB) is **not** baked — it's read from the mounted `HF_HOME`
|
||||||
|
(`DOTS_HFCACHE_DIR`). First boot downloads it there if absent.
|
||||||
|
|
||||||
|
## Voice cloning gotcha
|
||||||
|
|
||||||
|
dots.tts clones from `(reference wav + its transcript)` and **leaks reference
|
||||||
|
audio into the output** if the transcript is inaccurate or ends mid-clause. The
|
||||||
|
`voices/` corpus + `derive.py` handle this (sentence-bounded trim + accurate
|
||||||
|
transcript); don't hand this server a raw reference wav without a matching `.txt`.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- **GPU:** `NVIDIA_VISIBLE_DEVICES=0` = the 3090 in Docker (PCI order). `optimize=True`
|
||||||
|
is **incompatible with `PYTORCH_CUDA_ALLOC_CONF=expandable_segments`** (CUDA-graph
|
||||||
|
capture error) — don't set it.
|
||||||
|
- **Variants:** `dots.tts-mf` (MeanFlow, faster) is a drop-in via `DOTS_MODEL`; soar
|
||||||
|
is the quality pick and single-consumer doesn't need mf's batching.
|
||||||
@@ -0,0 +1,147 @@
|
|||||||
|
"""OpenAI-compatible /v1/audio/speech server over dots.tts (rednote-hilab).
|
||||||
|
|
||||||
|
Thin wrapper around DotsTtsRuntime — chosen over SGLang Omni because Omni's edge
|
||||||
|
(continuous batching) is MeanFlow-only and unneeded for a single-consumer surface,
|
||||||
|
while the raw runtime with optimize=True already streams at RTF ~0.22 on our 3090.
|
||||||
|
|
||||||
|
Voice registry: every <name>.wav (+ optional <name>.txt transcript) under
|
||||||
|
DOTS_VOICES_DIR becomes a callable voice. dots.tts REQUIRES an accurate,
|
||||||
|
sentence-bounded transcript to clone cleanly (see the voices/ corpus) — the .txt
|
||||||
|
is that transcript; without it the model leaks reference audio into the output.
|
||||||
|
"""
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
import glob
|
||||||
|
import struct
|
||||||
|
import threading
|
||||||
|
import wave
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
from fastapi import FastAPI, HTTPException
|
||||||
|
from fastapi.responses import Response, StreamingResponse
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from dots_tts.runtime import DotsTtsRuntime
|
||||||
|
|
||||||
|
MODEL = os.environ.get("DOTS_MODEL", "dots-studio/dots.tts-soar")
|
||||||
|
VOICES_DIR = os.environ.get("DOTS_VOICES_DIR", "/voices")
|
||||||
|
DEFAULT_VOICE = os.environ.get("DOTS_DEFAULT_VOICE", "donut")
|
||||||
|
NUM_STEPS = int(os.environ.get("DOTS_NUM_STEPS", "10"))
|
||||||
|
GUIDANCE = float(os.environ.get("DOTS_GUIDANCE_SCALE", "1.2"))
|
||||||
|
SAMPLE_RATE = 48000 # dots.tts fixed native output
|
||||||
|
|
||||||
|
app = FastAPI(title="dots.tts")
|
||||||
|
_rt = None
|
||||||
|
_voices: dict = {}
|
||||||
|
# One DotsTtsRuntime, and it is NOT safe to call concurrently (CUDA-graph capture
|
||||||
|
# + shared state). uvicorn runs sync endpoints in a threadpool, so we must
|
||||||
|
# serialize generation ourselves: requests queue and run one at a time. This is
|
||||||
|
# the deliberate trade for the thin-wrapper design — no vLLM-style continuous
|
||||||
|
# batching. If concurrency demand appears, swap the backend to SGLang Omni + the
|
||||||
|
# mf variant behind this same API (see README).
|
||||||
|
_gen_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def _load_voices() -> dict:
|
||||||
|
reg = {}
|
||||||
|
for wav in sorted(glob.glob(os.path.join(VOICES_DIR, "*.wav"))):
|
||||||
|
name = os.path.splitext(os.path.basename(wav))[0]
|
||||||
|
txt = os.path.splitext(wav)[0] + ".txt"
|
||||||
|
reg[name] = {
|
||||||
|
"wav": wav,
|
||||||
|
"text": open(txt).read().strip() if os.path.exists(txt) else "",
|
||||||
|
}
|
||||||
|
return reg
|
||||||
|
|
||||||
|
|
||||||
|
@app.on_event("startup")
|
||||||
|
def _startup():
|
||||||
|
global _rt, _voices
|
||||||
|
_voices = _load_voices()
|
||||||
|
_rt = DotsTtsRuntime.from_pretrained(MODEL, precision="bfloat16", optimize=True)
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/health")
|
||||||
|
def health():
|
||||||
|
return {
|
||||||
|
"status": "ok" if _rt is not None else "loading",
|
||||||
|
"model": MODEL,
|
||||||
|
"sample_rate": SAMPLE_RATE,
|
||||||
|
"voices": sorted(_voices),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/v1/voices")
|
||||||
|
def list_voices():
|
||||||
|
return {"voices": sorted(_voices)}
|
||||||
|
|
||||||
|
|
||||||
|
class SpeechRequest(BaseModel):
|
||||||
|
input: str
|
||||||
|
voice: str = DEFAULT_VOICE
|
||||||
|
model: str | None = None # accepted, ignored (single served model)
|
||||||
|
response_format: str = "wav" # wav | pcm
|
||||||
|
stream: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
def _to_pcm16(audio: np.ndarray) -> bytes:
|
||||||
|
return np.round(np.clip(audio, -1.0, 1.0) * 32767.0).astype("<i2").tobytes()
|
||||||
|
|
||||||
|
|
||||||
|
def _wav_bytes(pcm: bytes) -> bytes:
|
||||||
|
buf = io.BytesIO()
|
||||||
|
w = wave.open(buf, "wb")
|
||||||
|
w.setnchannels(1)
|
||||||
|
w.setsampwidth(2)
|
||||||
|
w.setframerate(SAMPLE_RATE)
|
||||||
|
w.writeframes(pcm)
|
||||||
|
w.close()
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
def _streaming_wav_header() -> bytes:
|
||||||
|
"""WAV header with placeholder (max) sizes — lets a client start playing the
|
||||||
|
stream before the total length is known (the pattern the Zonos/chatterbox
|
||||||
|
consumers already expect)."""
|
||||||
|
return (
|
||||||
|
b"RIFF" + struct.pack("<I", 0xFFFFFFFF) + b"WAVE"
|
||||||
|
+ b"fmt " + struct.pack("<IHHIIHH", 16, 1, 1, SAMPLE_RATE, SAMPLE_RATE * 2, 2, 16)
|
||||||
|
+ b"data" + struct.pack("<I", 0xFFFFFFFF)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/v1/audio/speech")
|
||||||
|
def speech(req: SpeechRequest):
|
||||||
|
rt = _rt
|
||||||
|
if rt is None:
|
||||||
|
raise HTTPException(503, "model still loading")
|
||||||
|
if req.voice not in _voices:
|
||||||
|
raise HTTPException(404, f"unknown voice '{req.voice}'; have {sorted(_voices)}")
|
||||||
|
if not req.input.strip():
|
||||||
|
raise HTTPException(400, "empty input")
|
||||||
|
|
||||||
|
v = _voices[req.voice]
|
||||||
|
kw = dict(
|
||||||
|
prompt_audio_path=v["wav"],
|
||||||
|
prompt_text=v["text"],
|
||||||
|
num_steps=NUM_STEPS,
|
||||||
|
guidance_scale=GUIDANCE,
|
||||||
|
normalize_text=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
if req.stream:
|
||||||
|
def gen():
|
||||||
|
# Hold the lock for the whole stream — a second generation on the
|
||||||
|
# shared runtime mid-stream would corrupt both.
|
||||||
|
with _gen_lock:
|
||||||
|
yield _streaming_wav_header()
|
||||||
|
for chunk in rt.generate_stream(text=req.input, **kw):
|
||||||
|
yield _to_pcm16(chunk.float().cpu().squeeze().numpy())
|
||||||
|
return StreamingResponse(gen(), media_type="audio/wav")
|
||||||
|
|
||||||
|
with _gen_lock:
|
||||||
|
res = rt.generate(text=req.input, **kw)
|
||||||
|
pcm = _to_pcm16(res["audio"].float().cpu().squeeze().numpy())
|
||||||
|
if req.response_format == "pcm":
|
||||||
|
return Response(pcm, media_type="audio/L16;rate=48000")
|
||||||
|
return Response(_wav_bytes(pcm), media_type="audio/wav")
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# dots.tts — OpenAI-compatible 48kHz zero-shot voice-clone TTS (rednote-hilab).
|
||||||
|
# Deployed on irv-ml1 ALONGSIDE chatterbox-fast (:8197) for burn-in; both share
|
||||||
|
# the 3090. Mirrors the chatterbox-fast sibling: GPU via `runtime: nvidia` +
|
||||||
|
# NVIDIA_VISIBLE_DEVICES, host IP:port (no traefik). All tunables in .env.
|
||||||
|
#
|
||||||
|
# GPU note: in Docker, NVIDIA_VISIBLE_DEVICES=0 resolves to the 3090 (PCI order),
|
||||||
|
# same as chatterbox-fast — NOT the A6000 that *native* CUDA calls device 0 on
|
||||||
|
# this host. CUDA_DEVICE_ORDER=PCI_BUS_ID is set belt-and-suspenders.
|
||||||
|
|
||||||
|
services:
|
||||||
|
dots-tts:
|
||||||
|
image: local/dots-tts:${DOTS_TAG:-v1}
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: dots-tts
|
||||||
|
restart: unless-stopped
|
||||||
|
runtime: nvidia
|
||||||
|
ports:
|
||||||
|
- "${DOTS_BIND:-0.0.0.0}:${DOTS_PORT:-8198}:8198"
|
||||||
|
environment:
|
||||||
|
- NVIDIA_VISIBLE_DEVICES=${DOTS_GPU_DEVICES:-0}
|
||||||
|
- CUDA_DEVICE_ORDER=PCI_BUS_ID
|
||||||
|
- HF_HOME=/hf_cache
|
||||||
|
- DOTS_MODEL=${DOTS_MODEL:-dots-studio/dots.tts-soar}
|
||||||
|
- DOTS_VOICES_DIR=/voices
|
||||||
|
- DOTS_DEFAULT_VOICE=${DOTS_DEFAULT_VOICE:-donut}
|
||||||
|
- DOTS_NUM_STEPS=${DOTS_NUM_STEPS:-10}
|
||||||
|
- DOTS_GUIDANCE_SCALE=${DOTS_GUIDANCE_SCALE:-1.2}
|
||||||
|
- DOTS_PORT=8198
|
||||||
|
volumes:
|
||||||
|
# HF model cache — reuse the already-downloaded soar snapshot (~5GB).
|
||||||
|
- ${DOTS_HFCACHE_DIR:-/home/lkraven/dots-tts/hf_cache}:/hf_cache
|
||||||
|
# Voice references — the dots-derived set from the voices/ corpus
|
||||||
|
# (derive.py dots -> derived/dots/<name>.{wav,txt}). Read-only.
|
||||||
|
- ${DOTS_VOICES_HOST_DIR:-/home/lkraven/voice-corpus/derived/dots}:/voices:ro
|
||||||
|
healthcheck:
|
||||||
|
# /health returns {"status":"ok",...} only once the model has loaded.
|
||||||
|
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; b=urllib.request.urlopen('http://127.0.0.1:8198/health',timeout=5).read(); sys.exit(0 if b'\\\"status\\\":\\\"ok\\\"' in b.replace(b' ',b'') else 1)\""]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
# Model load + optimize=True CUDA-graph warmup measured ~70s; generous.
|
||||||
|
start_period: 180s
|
||||||
|
labels:
|
||||||
|
- homepage.group=AI - Speech (TTS)
|
||||||
|
- homepage.name=dots.tts
|
||||||
|
- homepage.icon=mdi-account-voice
|
||||||
|
- homepage.description=Continuous-AR 48kHz zero-shot voice clone (irv-ml1)
|
||||||
|
- homepage.href=http://10.100.79.3:${DOTS_PORT:-8198}
|
||||||
Reference in New Issue
Block a user