vh
984b72757f
Zero-shot, massively-multilingual (600+ language) voice-cloning + voice-design TTS (diffusion-LM, Apache-2.0). No official image, so a thin CUDA container around the pip package running upstream's own Gradio demo (no FastAPI wrapper). Pinned to GPU 0 (3090) — the A6000 is ComfyUI-exclusive — port 8199. Built + verified live on irv-ml1 (Gradio 200, container healthy). Surface is the Gradio UI + Gradio API, NOT OpenAI-compat /v1/audio/speech (wrap later if asset-engine should consume it). deploy-omnivoice.yaml builds local + verifies.
59 lines
2.3 KiB
Markdown
59 lines
2.3 KiB
Markdown
# OmniVoice
|
||
|
||
[k2-fsa/OmniVoice](https://github.com/k2-fsa/OmniVoice) — zero-shot,
|
||
massively-multilingual (**600+ languages**) voice-cloning + voice-design
|
||
TTS from the Next-gen Kaldi / k2-fsa team. Diffusion-LM architecture,
|
||
RTF as low as ~0.025 (≈40× real-time). **Apache-2.0** — commercially clean
|
||
(unlike Voxtral's CC BY-NC).
|
||
|
||
## What it does
|
||
|
||
| Capability | Notes |
|
||
|---|---|
|
||
| Zero-shot voice cloning | Clone from a short reference clip |
|
||
| Voice **design** | Synthesize a voice from attributes (gender, age, pitch, accent, whisper, …) — no reference needed |
|
||
| 600+ languages | Broadest coverage of any zero-shot TTS |
|
||
| Fine control | Non-verbal symbols + pronunciation correction |
|
||
|
||
## How it's served
|
||
|
||
Upstream ships **its own Gradio demo** (`omnivoice-demo`), so this stack
|
||
just runs that — no custom wrapper. That means the surface is the **Gradio
|
||
UI + Gradio API**, *not* an OpenAI-compatible `/v1/audio/speech` endpoint.
|
||
|
||
- UI: `http://10.100.79.3:8199/`
|
||
- Programmatic: the Gradio API under `/gradio_api/` (or `/config` to
|
||
introspect). If you later want OpenAI-compat for asset-engine, add a thin
|
||
FastAPI wrapper like [`stacks/index-tts/app.py`](../index-tts/app.py).
|
||
|
||
## Placement
|
||
|
||
- **irv-ml1, GPU 0 (RTX 3090)** — pinned via `OMNIVOICE_GPU_DEVICES=0`.
|
||
The A6000 (device 1) is ComfyUI-exclusive after the 2026-06-18 VRAM
|
||
consolidation. OmniVoice fits in <5 GB; the 3090 had ~18 GB free.
|
||
- Port **8199** (8001 inside the container).
|
||
|
||
## Deploy
|
||
|
||
```bash
|
||
scripts/elway irv-ml1 --playbook playbooks/deploy-omnivoice.yaml
|
||
```
|
||
|
||
Builds the image locally (CUDA 12.8 + torch 2.8.0 + `omnivoice` from PyPI),
|
||
stages the build context under `/opt/docker/compose/omnivoice/`, brings it
|
||
up, and waits for the Gradio UI on `:8199`. First boot is slow: ~5-10 min
|
||
docker build + a one-time HF weight pre-warm (`k2-fsa/OmniVoice`, entrypoint
|
||
pre-download into `${OMNIVOICE_CACHE_DIR}`).
|
||
|
||
## Tunables
|
||
|
||
All in `.env` (see `.env.example`): `OMNIVOICE_PORT`, `OMNIVOICE_GPU_DEVICES`,
|
||
`OMNIVOICE_TAG`, `OMNIVOICE_VERSION` (optional PyPI pin),
|
||
`OMNIVOICE_CACHE_DIR`, `OMNIVOICE_VOICES_DIR`. Drop reference WAV/FLAC into
|
||
`/worktank/omnivoice/voices/` to stage cloning sources.
|
||
|
||
## Footprint
|
||
|
||
- **Disk**: HF weight cache under `/worktank/omnivoice/hf_cache`.
|
||
- **VRAM**: <5 GB (docs cite 4 GB+ GPUs).
|