stacks: add Kokoro, VibeVoice 1.5B, Chatterbox Turbo (TTS slate fill-in)

Three TTS additions to round out coverage on irv-ml1, each filling a distinct niche the existing slate doesn't own. Final coverage matrix (all on irv-ml1): Kokoro — low-latency English, fixed voice library, ~300ms TTFA Chatterbox Turbo — low-latency English w/ voice cloning + paralinguistic tags IndexTTS-2 — English voice cloning + emotion vector / text control Qwen3-TTS-1.7B-Base — high-quality English voice cloning CosyVoice 3 — multilingual (Chinese-leaning) VibeVoice 1.5B — long-form / multi-speaker dialogue stacks/kokoro: - port 8193, GPU device 0 (3090) - pulls ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master (no Dockerfile, no first-run model download — models baked in) - 60+ built-in voices, OpenAI-compat with stream=true over chunked HTTP - Apache-2.0 weights + code, ~1 GB VRAM stacks/vibevoice: - port 8194, GPU device 1 (A6000 — for 7B headroom) - builds groxaxo/VibeVoice-FastAPI1 (more current fork of ncoder-ai) pinned to 7614c469a145 - default model microsoft/VibeVoice-1.5B (~7 GB bf16 VRAM); env var swap to rsxdalv/VibeVoice-Large (7B) or FabioSarracino/VibeVoice-Large-Q8 - multi-speaker dialogue via /v1/vibevoice/generate with Speaker N: format - long-form niche only — not low-latency stacks/chatterbox: - port 8196, GPU device 0 (3090) - builds devnen/Chatterbox-TTS-Server (most active Turbo-supporting wrapper) - default model ResembleAI/chatterbox-turbo (~2.5 GB fp16, ~75ms latency) - paralinguistic tags inline ([laugh] [whisper] etc) — different shape from IndexTTS-2's emotion vector; fills the speed+cloning niche Kokoro/IndexTTS don't cover together - mandatory PerTh watermark on outputs (Resemble policy) Three matching playbooks under playbooks/deploy-{kokoro,vibevoice, chatterbox}.yaml. All idempotent, creates-/when-gated. Cold-deploy disk on /worktank/: ~7 GB Kokoro + ~19 GB VibeVoice 1.5B + ~12 GB Chatterbox = ~38 GB total. VRAM concurrent: ~10-11 GB across both GPUs. Skipped from the original four-stack proposal: VibeVoice Realtime (overlaps Kokoro's niche; Kokoro wins on latency, license, and not needing a build).
2026-04-25 16:18:37 -07:00
parent 54fef0e9d8
commit 4549d241a7
12 changed files with 930 additions and 0 deletions
@@ -0,0 +1,76 @@
+# VibeVoice 1.5B (long-form) stack tunables.
+# Copy to `.env` on irv-ml1 before deploying.
+
+# ── build pin ────────────────────────────────────────────────────────
+# SHA of groxaxo/VibeVoice-FastAPI1 (a more current fork of
+# ncoder-ai/VibeVoice-FastAPI). Bump + rebuild when you want upstream
+# wrapper updates.
+VIBEVOICE_SHA=7614c469a145
+
+# Local image tag — bump when you change build context to force a
+# fresh layer build.
+VIBEVOICE_TAG=v1
+
+# ── network ──────────────────────────────────────────────────────────
+# Host port. Container listens on 8001 internally.
+# Reserved on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS,
+# 8192 IndexTTS-2, 8193 Kokoro, 8765 Parakeet. 8194 picked here.
+VIBEVOICE_PORT=8194
+
+# Bind address. 0.0.0.0 exposes on all interfaces (incl. WG tunnel
+# interface 10.100.79.3); 127.0.0.1 restricts to local-only.
+VIBEVOICE_BIND=0.0.0.0
+
+# ── runtime / GPU ────────────────────────────────────────────────────
+# Devices visible inside the container. "1" pins to the RTX A6000 —
+# 1.5B fits easily on the 3090 too, but pinning to the bigger card
+# leaves headroom if you later flip VIBEVOICE_MODEL to the 7B variant.
+VIBEVOICE_GPU_DEVICES=1
+
+# Model. Options (per groxaxo/ncoder-ai docs):
+#   microsoft/VibeVoice-1.5B        — flagship, ~7 GB bf16 VRAM
+#   rsxdalv/VibeVoice-Large         — 7B variant, ~18 GB bf16 VRAM
+#                                     (need device_ids="1" / A6000)
+#   FabioSarracino/VibeVoice-Large-Q8 — 7B int8 quantized, ~10 GB
+VIBEVOICE_MODEL=microsoft/VibeVoice-1.5B
+
+# Number of denoising inference steps. Default 10 is a good
+# quality/speed tradeoff. Lower = faster but lower quality.
+VIBEVOICE_INFERENCE_STEPS=10
+
+# Compute dtype. bfloat16 default (best speed/quality on Ampere+).
+# Use float16 for older GPUs without bf16 support.
+VIBEVOICE_DTYPE=bfloat16
+
+# Attention impl. flash_attention_2 is fastest if installed (bundled
+# in the upstream image build). Fall back to "sdpa" if it errors.
+VIBEVOICE_ATTN=flash_attention_2
+
+# Quantization. Empty = none. "int8_torchao" saves ~40% VRAM at a
+# small quality cost — useful if you want to run 7B on a smaller GPU.
+VIBEVOICE_QUANT=
+
+# torch.compile. Bumps cold-start by ~3-5 min the first time but
+# trims per-generation latency. Disable if you're iterating quickly.
+VIBEVOICE_TORCH_COMPILE=false
+VIBEVOICE_TORCH_COMPILE_MODE=default
+
+# CFG (classifier-free guidance) scale. Default 1.8 from upstream;
+# higher = stronger adherence to text/voice, lower = more free.
+VIBEVOICE_CFG_SCALE=1.8
+
+# Max generation length in audio frames. 5400 = ~6 minutes at the
+# native rate. Bump for longer podcasts (each frame takes work).
+VIBEVOICE_MAX_GEN_LEN=5400
+
+# ── persistent storage on the host ───────────────────────────────────
+# Voice library — flat dir of .wav/.mp3/.flac/.m4a files. Mounted
+# read-only into the container. Drop a file in, restart container,
+# voice is available. (Restart needed because the upstream wrapper
+# scans on init, not per-request.)
+VIBEVOICE_VOICES_DIR=/worktank/vibevoice/voices
+
+# HuggingFace cache. Holds VibeVoice weights + any aux models pulled.
+# Bind-mounted so model state survives container recreate. Excluded
+# from restic (regenerable from HF).
+VIBEVOICE_CACHE_DIR=/worktank/vibevoice/cache
@@ -0,0 +1,115 @@
+# VibeVoice
+
+Microsoft's diffusion-based long-form TTS, served via
+[groxaxo/VibeVoice-FastAPI1](https://github.com/groxaxo/VibeVoice-FastAPI1)
+(a recent fork of [ncoder-ai/VibeVoice-FastAPI](https://github.com/ncoder-ai/VibeVoice-FastAPI)
+which moves faster than upstream).
+
+Model: [microsoft/VibeVoice-1.5B](https://huggingface.co/microsoft/VibeVoice-1.5B)
+by default. Switch to the 7B variant via `.env` if you want the
+bigger checkpoint.
+
+## Why this stack exists
+
+Long-form / podcast-quality TTS with native multi-speaker dialogue
+support. Designed for one-shot generation of multi-minute scripts
+where conversation flow matters. **Not** for low-latency single-line
+synthesis — for that use Kokoro or Chatterbox Turbo.
+
+| | use case |
+|---|---|
+| **VibeVoice 1.5B** | long-form, multi-speaker dialogue (this stack) |
+| Kokoro | low-latency English, fixed voice library |
+| Chatterbox Turbo | low-latency English w/ voice cloning |
+| IndexTTS-2 | English voice cloning + emotion control |
+| Qwen3-TTS-1.7B-Base | high-quality English voice cloning |
+| CosyVoice 3 | multilingual (Chinese-leaning) |
+
+## API
+
+OpenAI-compat at `http://10.100.79.3:8194`:
+
+```bash
+# Single-speaker (OpenAI-style).
+curl -fsS -X POST http://10.100.79.3:8194/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"vibevoice","input":"Hello there.","voice":"voice-name","response_format":"wav"}' \
+  > out.wav
+
+# Multi-speaker dialogue — the headline feature. Format the input
+# as a script with `Speaker N:` prefixes (0-indexed). The wrapper's
+# extended /v1/vibevoice/generate endpoint handles voice switching.
+curl -fsS -X POST http://10.100.79.3:8194/v1/vibevoice/generate \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "script":"Speaker 0: Welcome to the show.\nSpeaker 1: Glad to be here.\nSpeaker 0: Today we discuss…",
+    "voices":["voice-host","voice-guest"],
+    "stream":true
+  }' > podcast.wav
+
+# List available voices.
+curl http://10.100.79.3:8194/v1/audio/voices
+```
+
+OpenAPI / docs at `/docs`. Healthcheck at `/health`.
+
+`stream=true` is honored on the multi-speaker endpoint; the
+single-shot OpenAI endpoint returns the full file in one go.
+
+## Voices
+
+Drop `.wav` / `.mp3` / `.flac` / `.m4a` into
+`/worktank/vibevoice/voices/` on the host (mounted read-only into
+the container). Restart the container after adding; the wrapper
+scans the dir at init, not per-request:
+
+```bash
+ssh irv-ml1 'cd /opt/docker/compose/vibevoice && docker compose restart'
+```
+
+VibeVoice also has built-in voice presets (Carter, Davis, Emma,
+Frank, Grace, Mike, Samuel) accessible by name. Microsoft has not
+released the cloning tooling so you can't add new "trained" voices
+— but the bundled ones already cover most podcast use cases.
+
+## Deploy
+
+```bash
+scripts/elway irv-ml1 --playbook playbooks/deploy-vibevoice.yaml
+```
+
+Cold deploy budget:
+- ~12 GB image build (CUDA 12.8 + torch 2.8 + flash-attn)
+- ~7 GB model download (VibeVoice-1.5B) on first start
+- **Total: ~19 GB on /worktank/vibevoice/**
+
+First build: ~12 min. First generation: ~30-60 s warmup.
+
+## Switching to the 7B variant
+
+```bash
+ssh irv-ml1 '
+  cd /opt/docker/compose/vibevoice
+  sed -i "s|^VIBEVOICE_MODEL=.*|VIBEVOICE_MODEL=rsxdalv/VibeVoice-Large|" .env
+  docker compose up -d
+'
+```
+
+The 7B model auto-downloads on next start (~18 GB). VRAM jumps from
+~7 GB to ~18 GB bf16 — keep `VIBEVOICE_GPU_DEVICES=1` (A6000) for it.
+For lower VRAM at slight quality cost, set `VIBEVOICE_QUANT=int8_torchao`
+which brings 7B down to ~10 GB.
+
+## Gotchas
+
+- **Not streaming-friendly for single-line use.** The diffusion head
+  has to denoise the whole latent before vocoding. Streaming on
+  `/v1/vibevoice/generate` works at script-segment granularity
+  (paragraph-ish), not token-by-token.
+- **Voice cloning isn't published.** Microsoft released the inference
+  models but not the training pipeline. Use the built-in voices, or
+  pick another stack (IndexTTS-2 / Qwen3-TTS / Chatterbox Turbo).
+- **`flash_attention_2`** is the upstream default; if your GPU/torch
+  combo doesn't have it built, set `VIBEVOICE_ATTN=sdpa` in `.env`
+  to fall back to PyTorch's scaled-dot-product attention.
+- **License**: VibeVoice MIT (Microsoft); wrapper MIT.
@@ -0,0 +1,68 @@
+# VibeVoice 1.5B long-form TTS via groxaxo/VibeVoice-FastAPI1
+# (fork of ncoder-ai/VibeVoice-FastAPI). Multi-speaker dialogue support
+# via the extended /v1/vibevoice/generate endpoint with a Speaker N:
+# script format. OpenAI-compat /v1/audio/speech also exposed.
+#
+# Why this stack exists alongside the other TTS:
+#   * Long-form / podcast-quality slot — VibeVoice is Microsoft's
+#     diffusion-based long-form TTS designed for multi-speaker output.
+#   * Dialogue mode: feed `Speaker 0: ... \n Speaker 1: ...` and the
+#     model handles voice switching natively.
+#   * Trade-off: NOT streaming-friendly — generation is single-shot
+#     latent denoising over the whole sequence, then vocode. For
+#     low-latency English, use Kokoro or Chatterbox Turbo instead.
+#
+# Image is built locally from the upstream Dockerfile via docker
+# buildx git-context (no source vendored on the host). Pinned to a
+# SHA in .env so rebuilds are reproducible.
+#
+# Default model is VibeVoice-1.5B (~7 GB bf16 VRAM). Switch to
+# rsxdalv/VibeVoice-Large for the 7B variant (~18 GB) — pin to A6000
+# in that case.
+#
+# All tunables live in .env — edit that, not this file.
+
+services:
+  vibevoice:
+    image: local/vibevoice:${VIBEVOICE_TAG}
+    build:
+      context: https://github.com/groxaxo/VibeVoice-FastAPI1.git#${VIBEVOICE_SHA}
+      dockerfile: Dockerfile
+    container_name: vibevoice
+    restart: unless-stopped
+    runtime: nvidia
+    ports:
+      - "${VIBEVOICE_BIND:-0.0.0.0}:${VIBEVOICE_PORT}:8001"
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=${VIBEVOICE_GPU_DEVICES:-1}
+      - VIBEVOICE_MODEL_PATH=${VIBEVOICE_MODEL:-microsoft/VibeVoice-1.5B}
+      - VIBEVOICE_DEVICE=cuda
+      - VIBEVOICE_INFERENCE_STEPS=${VIBEVOICE_INFERENCE_STEPS:-10}
+      - VIBEVOICE_DTYPE=${VIBEVOICE_DTYPE:-bfloat16}
+      - VIBEVOICE_ATTN_IMPLEMENTATION=${VIBEVOICE_ATTN:-flash_attention_2}
+      - VIBEVOICE_QUANTIZATION=${VIBEVOICE_QUANT:-}
+      - TORCH_COMPILE=${VIBEVOICE_TORCH_COMPILE:-false}
+      - TORCH_COMPILE_MODE=${VIBEVOICE_TORCH_COMPILE_MODE:-default}
+      - VOICES_DIR=/app/voices
+      - DEFAULT_CFG_SCALE=${VIBEVOICE_CFG_SCALE:-1.8}
+      - MAX_GENERATION_LENGTH=${VIBEVOICE_MAX_GEN_LEN:-5400}
+      - HF_HOME=/root/.cache/huggingface
+    volumes:
+      - ${VIBEVOICE_VOICES_DIR}:/app/voices:ro
+      - ${VIBEVOICE_CACHE_DIR}:/root/.cache/huggingface
+    healthcheck:
+      # Upstream Dockerfile exposes /health.
+      test: ["CMD-SHELL", "curl -fsS -o /dev/null http://localhost:8001/health || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      # First boot pulls VibeVoice-1.5B (~7 GB) on a cold cache and
+      # may also build flash-attn / torch.compile JIT cache on the
+      # first inference. Generous deadline.
+      start_period: 900s
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=VibeVoice
+      - homepage.icon=mdi-podcast
+      - homepage.description=Long-form / multi-speaker dialogue TTS (irv-ml1)
+      - homepage.href=http://10.100.79.3:${VIBEVOICE_PORT}