Files

T

vh 984b72757f feat(omnivoice): new TTS stack — k2-fsa/OmniVoice on irv-ml1 3090

Zero-shot, massively-multilingual (600+ language) voice-cloning + voice-design
TTS (diffusion-LM, Apache-2.0). No official image, so a thin CUDA container
around the pip package running upstream's own Gradio demo (no FastAPI wrapper).
Pinned to GPU 0 (3090) — the A6000 is ComfyUI-exclusive — port 8199. Built +
verified live on irv-ml1 (Gradio 200, container healthy). Surface is the Gradio
UI + Gradio API, NOT OpenAI-compat /v1/audio/speech (wrap later if asset-engine
should consume it). deploy-omnivoice.yaml builds local + verifies.

2026-06-18 22:25:54 -07:00

2.3 KiB

Raw Blame History

OmniVoice

k2-fsa/OmniVoice — zero-shot, massively-multilingual (600+ languages) voice-cloning + voice-design TTS from the Next-gen Kaldi / k2-fsa team. Diffusion-LM architecture, RTF as low as ~0.025 (≈40× real-time). Apache-2.0 — commercially clean (unlike Voxtral's CC BY-NC).

What it does

Capability	Notes
Zero-shot voice cloning	Clone from a short reference clip
Voice design	Synthesize a voice from attributes (gender, age, pitch, accent, whisper, …) — no reference needed
600+ languages	Broadest coverage of any zero-shot TTS
Fine control	Non-verbal symbols + pronunciation correction

How it's served

Upstream ships its own Gradio demo (omnivoice-demo), so this stack just runs that — no custom wrapper. That means the surface is the Gradio UI + Gradio API, not an OpenAI-compatible /v1/audio/speech endpoint.

UI: http://10.100.79.3:8199/
Programmatic: the Gradio API under /gradio_api/ (or /config to introspect). If you later want OpenAI-compat for asset-engine, add a thin FastAPI wrapper like stacks/index-tts/app.py.

Placement

irv-ml1, GPU 0 (RTX 3090) — pinned via OMNIVOICE_GPU_DEVICES=0. The A6000 (device 1) is ComfyUI-exclusive after the 2026-06-18 VRAM consolidation. OmniVoice fits in <5 GB; the 3090 had ~18 GB free.
Port 8199 (8001 inside the container).

Deploy

scripts/elway irv-ml1 --playbook playbooks/deploy-omnivoice.yaml

Builds the image locally (CUDA 12.8 + torch 2.8.0 + omnivoice from PyPI), stages the build context under /opt/docker/compose/omnivoice/, brings it up, and waits for the Gradio UI on :8199. First boot is slow: ~5-10 min docker build + a one-time HF weight pre-warm (k2-fsa/OmniVoice, entrypoint pre-download into ${OMNIVOICE_CACHE_DIR}).

Tunables

All in .env (see .env.example): OMNIVOICE_PORT, OMNIVOICE_GPU_DEVICES, OMNIVOICE_TAG, OMNIVOICE_VERSION (optional PyPI pin), OMNIVOICE_CACHE_DIR, OMNIVOICE_VOICES_DIR. Drop reference WAV/FLAC into /worktank/omnivoice/voices/ to stage cloning sources.

Footprint

Disk: HF weight cache under /worktank/omnivoice/hf_cache.
VRAM: <5 GB (docs cite 4 GB+ GPUs).

2.3 KiB Raw Blame History Unescape Escape