# OmniVoice

[k2-fsa/OmniVoice](https://github.com/k2-fsa/OmniVoice) — zero-shot,
massively-multilingual (**600+ languages**) voice-cloning + voice-design
TTS from the Next-gen Kaldi / k2-fsa team. Diffusion-LM architecture,
RTF as low as ~0.025 (≈40× real-time). **Apache-2.0** — commercially clean
(unlike Voxtral's CC BY-NC).

## What it does

| Capability | Notes |
|---|---|
| Zero-shot voice cloning | Clone from a short reference clip |
| Voice **design** | Synthesize a voice from attributes (gender, age, pitch, accent, whisper, …) — no reference needed |
| 600+ languages | Broadest coverage of any zero-shot TTS |
| Fine control | Non-verbal symbols + pronunciation correction |

## How it's served

Upstream ships **its own Gradio demo** (`omnivoice-demo`), so this stack
just runs that — no custom wrapper. That means the surface is the **Gradio
UI + Gradio API**, *not* an OpenAI-compatible `/v1/audio/speech` endpoint.

- UI: `http://10.100.79.3:8199/`
- Programmatic: the Gradio API under `/gradio_api/` (or `/config` to
  introspect). If you later want OpenAI-compat for asset-engine, add a thin
  FastAPI wrapper like [`stacks/index-tts/app.py`](../index-tts/app.py).

## Placement

- **irv-ml1, GPU 0 (RTX 3090)** — pinned via `OMNIVOICE_GPU_DEVICES=0`.
  The A6000 (device 1) is ComfyUI-exclusive after the 2026-06-18 VRAM
  consolidation. OmniVoice fits in <5 GB; the 3090 had ~18 GB free.
- Port **8199** (8001 inside the container).

## Deploy

```bash
scripts/elway irv-ml1 --playbook playbooks/deploy-omnivoice.yaml
```

Builds the image locally (CUDA 12.8 + torch 2.8.0 + `omnivoice` from PyPI),
stages the build context under `/opt/docker/compose/omnivoice/`, brings it
up, and waits for the Gradio UI on `:8199`. First boot is slow: ~5-10 min
docker build + a one-time HF weight pre-warm (`k2-fsa/OmniVoice`, entrypoint
pre-download into `${OMNIVOICE_CACHE_DIR}`).

## Tunables

All in `.env` (see `.env.example`): `OMNIVOICE_PORT`, `OMNIVOICE_GPU_DEVICES`,
`OMNIVOICE_TAG`, `OMNIVOICE_VERSION` (optional PyPI pin),
`OMNIVOICE_CACHE_DIR`, `OMNIVOICE_VOICES_DIR`. Drop reference WAV/FLAC into
`/worktank/omnivoice/voices/` to stage cloning sources.

## Footprint

- **Disk**: HF weight cache under `/worktank/omnivoice/hf_cache`.
- **VRAM**: <5 GB (docs cite 4 GB+ GPUs).