feat(omnivoice): new TTS stack — k2-fsa/OmniVoice on irv-ml1 3090

Zero-shot, massively-multilingual (600+ language) voice-cloning + voice-design TTS (diffusion-LM, Apache-2.0). No official image, so a thin CUDA container around the pip package running upstream's own Gradio demo (no FastAPI wrapper). Pinned to GPU 0 (3090) — the A6000 is ComfyUI-exclusive — port 8199. Built + verified live on irv-ml1 (Gradio 200, container healthy). Surface is the Gradio UI + Gradio API, NOT OpenAI-compat /v1/audio/speech (wrap later if asset-engine should consume it). deploy-omnivoice.yaml builds local + verifies.
2026-06-18 22:25:54 -07:00
parent 715a68bee7
commit 984b72757f
6 changed files with 319 additions and 0 deletions
@@ -0,0 +1,58 @@
+# OmniVoice
+
+[k2-fsa/OmniVoice](https://github.com/k2-fsa/OmniVoice) — zero-shot,
+massively-multilingual (**600+ languages**) voice-cloning + voice-design
+TTS from the Next-gen Kaldi / k2-fsa team. Diffusion-LM architecture,
+RTF as low as ~0.025 (≈40× real-time). **Apache-2.0** — commercially clean
+(unlike Voxtral's CC BY-NC).
+
+## What it does
+
+| Capability | Notes |
+|---|---|
+| Zero-shot voice cloning | Clone from a short reference clip |
+| Voice **design** | Synthesize a voice from attributes (gender, age, pitch, accent, whisper, …) — no reference needed |
+| 600+ languages | Broadest coverage of any zero-shot TTS |
+| Fine control | Non-verbal symbols + pronunciation correction |
+
+## How it's served
+
+Upstream ships **its own Gradio demo** (`omnivoice-demo`), so this stack
+just runs that — no custom wrapper. That means the surface is the **Gradio
+UI + Gradio API**, *not* an OpenAI-compatible `/v1/audio/speech` endpoint.
+
+- UI: `http://10.100.79.3:8199/`
+- Programmatic: the Gradio API under `/gradio_api/` (or `/config` to
+  introspect). If you later want OpenAI-compat for asset-engine, add a thin
+  FastAPI wrapper like [`stacks/index-tts/app.py`](../index-tts/app.py).
+
+## Placement
+
+- **irv-ml1, GPU 0 (RTX 3090)** — pinned via `OMNIVOICE_GPU_DEVICES=0`.
+  The A6000 (device 1) is ComfyUI-exclusive after the 2026-06-18 VRAM
+  consolidation. OmniVoice fits in <5 GB; the 3090 had ~18 GB free.
+- Port **8199** (8001 inside the container).
+
+## Deploy
+
+```bash
+scripts/elway irv-ml1 --playbook playbooks/deploy-omnivoice.yaml
+```
+
+Builds the image locally (CUDA 12.8 + torch 2.8.0 + `omnivoice` from PyPI),
+stages the build context under `/opt/docker/compose/omnivoice/`, brings it
+up, and waits for the Gradio UI on `:8199`. First boot is slow: ~5-10 min
+docker build + a one-time HF weight pre-warm (`k2-fsa/OmniVoice`, entrypoint
+pre-download into `${OMNIVOICE_CACHE_DIR}`).
+
+## Tunables
+
+All in `.env` (see `.env.example`): `OMNIVOICE_PORT`, `OMNIVOICE_GPU_DEVICES`,
+`OMNIVOICE_TAG`, `OMNIVOICE_VERSION` (optional PyPI pin),
+`OMNIVOICE_CACHE_DIR`, `OMNIVOICE_VOICES_DIR`. Drop reference WAV/FLAC into
+`/worktank/omnivoice/voices/` to stage cloning sources.
+
+## Footprint
+
+- **Disk**: HF weight cache under `/worktank/omnivoice/hf_cache`.
+- **VRAM**: <5 GB (docs cite 4 GB+ GPUs).