feat(omnivoice): new TTS stack — k2-fsa/OmniVoice on irv-ml1 3090
Zero-shot, massively-multilingual (600+ language) voice-cloning + voice-design TTS (diffusion-LM, Apache-2.0). No official image, so a thin CUDA container around the pip package running upstream's own Gradio demo (no FastAPI wrapper). Pinned to GPU 0 (3090) — the A6000 is ComfyUI-exclusive — port 8199. Built + verified live on irv-ml1 (Gradio 200, container healthy). Surface is the Gradio UI + Gradio API, NOT OpenAI-compat /v1/audio/speech (wrap later if asset-engine should consume it). deploy-omnivoice.yaml builds local + verifies.
This commit is contained in:
@@ -0,0 +1,20 @@
|
||||
# OmniVoice (k2-fsa/OmniVoice) — irv-ml1 stack tunables.
|
||||
# Copy to .env on the host (/opt/docker/compose/omnivoice/.env). The deploy
|
||||
# playbook seeds .env from this template on first run only.
|
||||
|
||||
# Host port (container always listens on 8001). 8199 is free in the irv-ml1
|
||||
# audio range (8190-8198 + 8765/919x taken; 8201 reserved for voxtral).
|
||||
OMNIVOICE_PORT=8199
|
||||
OMNIVOICE_BIND=0.0.0.0
|
||||
|
||||
# GPU: device 0 = RTX 3090 on irv-ml1 (device 1 / A6000 is ComfyUI-exclusive).
|
||||
# OmniVoice runs in <5 GB; the 3090 had ~18 GB free.
|
||||
OMNIVOICE_GPU_DEVICES=0
|
||||
|
||||
# Image tag + optional upstream pin (empty = latest omnivoice on PyPI).
|
||||
OMNIVOICE_TAG=latest
|
||||
OMNIVOICE_VERSION=
|
||||
|
||||
# Persistent HF weight cache + reference-voice staging on /worktank.
|
||||
OMNIVOICE_CACHE_DIR=/worktank/omnivoice/hf_cache
|
||||
OMNIVOICE_VOICES_DIR=/worktank/omnivoice/voices
|
||||
@@ -0,0 +1,54 @@
|
||||
# syntax=docker/dockerfile:1.6
|
||||
#
|
||||
# OmniVoice (k2-fsa/OmniVoice) — zero-shot, massively-multilingual (600+
|
||||
# languages) voice-cloning + voice-design TTS, diffusion-LM architecture,
|
||||
# Apache-2.0. Upstream ships a pip package + its own Gradio demo
|
||||
# (`omnivoice-demo`); there's no official image, so we build a thin CUDA
|
||||
# container around the pip package and run its Gradio server directly.
|
||||
# Unlike index-tts we DON'T need a FastAPI wrapper — OmniVoice serves itself.
|
||||
|
||||
ARG CUDA_BASE=nvidia/cuda:12.8.0-cudnn-runtime-ubuntu22.04
|
||||
FROM ${CUDA_BASE}
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive \
|
||||
PIP_ROOT_USER_ACTION=ignore \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PATH="/opt/venv/bin:${PATH}"
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
python3.10 python3.10-venv python3-pip \
|
||||
git ffmpeg libsndfile1 \
|
||||
ca-certificates wget \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& python3.10 -m venv /opt/venv
|
||||
|
||||
# CUDA 12.8 torch wheels (per OmniVoice's documented install line).
|
||||
RUN pip install --no-cache-dir \
|
||||
torch==2.8.0 torchaudio==2.8.0 \
|
||||
--index-url https://download.pytorch.org/whl/cu128
|
||||
|
||||
# OmniVoice from PyPI (+ huggingface_hub for the entrypoint weight pre-warm).
|
||||
# Optional reproducible pin via the OMNIVOICE_VERSION build arg (empty=latest).
|
||||
ARG OMNIVOICE_VERSION=
|
||||
RUN pip install --no-cache-dir "omnivoice${OMNIVOICE_VERSION:+==${OMNIVOICE_VERSION}}" huggingface_hub
|
||||
|
||||
# Fail the build loudly if the console script name isn't what we expect,
|
||||
# rather than crash-loop at runtime. Logs the actual omni* entrypoints.
|
||||
RUN echo "omni console scripts:" && (ls /opt/venv/bin | grep -i omni || true) \
|
||||
&& command -v omnivoice-demo >/dev/null \
|
||||
|| { echo "ERROR: omnivoice-demo CLI not found after install"; exit 1; }
|
||||
|
||||
WORKDIR /app
|
||||
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
|
||||
RUN chmod +x /usr/local/bin/entrypoint.sh
|
||||
|
||||
EXPOSE 8001
|
||||
|
||||
# Gradio serves HTML at / — 200 once the UI is up (weights load lazily on
|
||||
# first synth; the entrypoint pre-warms them). Generous start period.
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=900s --retries=3 \
|
||||
CMD wget -q -O /dev/null http://127.0.0.1:8001/ || exit 1
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
|
||||
CMD ["omnivoice-demo", "--ip", "0.0.0.0", "--port", "8001"]
|
||||
@@ -0,0 +1,58 @@
|
||||
# OmniVoice
|
||||
|
||||
[k2-fsa/OmniVoice](https://github.com/k2-fsa/OmniVoice) — zero-shot,
|
||||
massively-multilingual (**600+ languages**) voice-cloning + voice-design
|
||||
TTS from the Next-gen Kaldi / k2-fsa team. Diffusion-LM architecture,
|
||||
RTF as low as ~0.025 (≈40× real-time). **Apache-2.0** — commercially clean
|
||||
(unlike Voxtral's CC BY-NC).
|
||||
|
||||
## What it does
|
||||
|
||||
| Capability | Notes |
|
||||
|---|---|
|
||||
| Zero-shot voice cloning | Clone from a short reference clip |
|
||||
| Voice **design** | Synthesize a voice from attributes (gender, age, pitch, accent, whisper, …) — no reference needed |
|
||||
| 600+ languages | Broadest coverage of any zero-shot TTS |
|
||||
| Fine control | Non-verbal symbols + pronunciation correction |
|
||||
|
||||
## How it's served
|
||||
|
||||
Upstream ships **its own Gradio demo** (`omnivoice-demo`), so this stack
|
||||
just runs that — no custom wrapper. That means the surface is the **Gradio
|
||||
UI + Gradio API**, *not* an OpenAI-compatible `/v1/audio/speech` endpoint.
|
||||
|
||||
- UI: `http://10.100.79.3:8199/`
|
||||
- Programmatic: the Gradio API under `/gradio_api/` (or `/config` to
|
||||
introspect). If you later want OpenAI-compat for asset-engine, add a thin
|
||||
FastAPI wrapper like [`stacks/index-tts/app.py`](../index-tts/app.py).
|
||||
|
||||
## Placement
|
||||
|
||||
- **irv-ml1, GPU 0 (RTX 3090)** — pinned via `OMNIVOICE_GPU_DEVICES=0`.
|
||||
The A6000 (device 1) is ComfyUI-exclusive after the 2026-06-18 VRAM
|
||||
consolidation. OmniVoice fits in <5 GB; the 3090 had ~18 GB free.
|
||||
- Port **8199** (8001 inside the container).
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
scripts/elway irv-ml1 --playbook playbooks/deploy-omnivoice.yaml
|
||||
```
|
||||
|
||||
Builds the image locally (CUDA 12.8 + torch 2.8.0 + `omnivoice` from PyPI),
|
||||
stages the build context under `/opt/docker/compose/omnivoice/`, brings it
|
||||
up, and waits for the Gradio UI on `:8199`. First boot is slow: ~5-10 min
|
||||
docker build + a one-time HF weight pre-warm (`k2-fsa/OmniVoice`, entrypoint
|
||||
pre-download into `${OMNIVOICE_CACHE_DIR}`).
|
||||
|
||||
## Tunables
|
||||
|
||||
All in `.env` (see `.env.example`): `OMNIVOICE_PORT`, `OMNIVOICE_GPU_DEVICES`,
|
||||
`OMNIVOICE_TAG`, `OMNIVOICE_VERSION` (optional PyPI pin),
|
||||
`OMNIVOICE_CACHE_DIR`, `OMNIVOICE_VOICES_DIR`. Drop reference WAV/FLAC into
|
||||
`/worktank/omnivoice/voices/` to stage cloning sources.
|
||||
|
||||
## Footprint
|
||||
|
||||
- **Disk**: HF weight cache under `/worktank/omnivoice/hf_cache`.
|
||||
- **VRAM**: <5 GB (docs cite 4 GB+ GPUs).
|
||||
@@ -0,0 +1,48 @@
|
||||
# OmniVoice (k2-fsa/OmniVoice) — zero-shot, massively-multilingual (600+
|
||||
# language) voice-cloning + voice-design TTS, diffusion-LM, Apache-2.0.
|
||||
# Served via upstream's own Gradio demo. NOTE: this exposes the Gradio UI
|
||||
# + Gradio API, NOT an OpenAI-compatible /v1/audio/speech endpoint — wrap
|
||||
# it later (à la stacks/index-tts/app.py) if asset-engine integration is
|
||||
# wanted. For now it's a "stand it up and try it" UI.
|
||||
#
|
||||
# Build: local image from the Dockerfile in this dir. Weights download
|
||||
# from HF (k2-fsa/OmniVoice) on first boot into ${OMNIVOICE_CACHE_DIR}.
|
||||
#
|
||||
# Pinned to the 3090 (device 0) on irv-ml1 — the A6000 (device 1) is
|
||||
# ComfyUI-exclusive after the 2026-06-18 VRAM consolidation. OmniVoice
|
||||
# runs in well under 5 GB; the 3090 had ~18 GB free.
|
||||
#
|
||||
# All tunables live in .env — edit that, not this file.
|
||||
|
||||
services:
|
||||
omnivoice:
|
||||
image: local/omnivoice:${OMNIVOICE_TAG:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
OMNIVOICE_VERSION: ${OMNIVOICE_VERSION:-}
|
||||
container_name: omnivoice
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${OMNIVOICE_BIND:-0.0.0.0}:${OMNIVOICE_PORT}:8001"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${OMNIVOICE_GPU_DEVICES:-0}
|
||||
- HF_HOME=/app/hf_cache
|
||||
volumes:
|
||||
- ${OMNIVOICE_CACHE_DIR}:/app/hf_cache
|
||||
- ${OMNIVOICE_VOICES_DIR}:/app/voices
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -q -O /dev/null http://localhost:8001/ || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# First boot: weight pre-warm download (entrypoint) + CUDA warmup.
|
||||
start_period: 900s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=OmniVoice
|
||||
- homepage.icon=mdi-account-voice
|
||||
- homepage.description=Zero-shot multilingual voice-cloning TTS (irv-ml1, 3090)
|
||||
- homepage.href=http://10.100.79.3:${OMNIVOICE_PORT}
|
||||
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env bash
|
||||
# entrypoint.sh — pre-warm OmniVoice weights (k2-fsa/OmniVoice) into the
|
||||
# persistent HF cache on first run, then exec the Gradio demo.
|
||||
#
|
||||
# The demo also auto-downloads on first synth, so the pre-warm is
|
||||
# best-effort (NON-FATAL): it just makes the first generation fast +
|
||||
# deterministic and lets the healthcheck come up against a ready model.
|
||||
set -e
|
||||
|
||||
: "${HF_HOME:=/app/hf_cache}"
|
||||
export HF_HOME
|
||||
mkdir -p "${HF_HOME}"
|
||||
|
||||
MARKER="${HF_HOME}/.omnivoice-prewarmed"
|
||||
if [ ! -f "${MARKER}" ]; then
|
||||
echo "[omnivoice] pre-warming k2-fsa/OmniVoice weights into ${HF_HOME} (one-time)…"
|
||||
if python3 - <<'EOF'
|
||||
from huggingface_hub import snapshot_download
|
||||
snapshot_download(repo_id="k2-fsa/OmniVoice")
|
||||
print("[omnivoice] pre-warm complete")
|
||||
EOF
|
||||
then
|
||||
touch "${MARKER}"
|
||||
else
|
||||
echo "[omnivoice] pre-warm failed (non-fatal) — demo will download on first synth"
|
||||
fi
|
||||
fi
|
||||
|
||||
exec "$@"
|
||||
Reference in New Issue
Block a user