vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(zonos): OpenAI-ish REST adapter for asset-engine routing

Browse Source 
Upstream Zonos ships only Gradio + Python SDK — no REST surface — so
asset-engine (which routes a clean JSON POST to /v1/audio/speech) can't
target it directly. Add a thin FastAPI adapter (stacks/zonos/adapter/):
POST /v1/audio/speech in front of the Zonos SDK, built FROM local/zonos
to reuse torch/CUDA/SDK. Returns a JSON envelope {audio, audio_format,
seed} — the seed rides back so asset-engine regenerate/fork can pin it
(Zonos is the fleet's first genuinely seedable TTS). compose gains a
zonos-api service on 8201; .env.example gains the port + voices dir.
This commit is contained in:
Vuong Hoang
2026-05-31 13:59:10 -07:00
parent 71df6f7474
commit 81efa8da96
7 changed files with 283 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/zonos/.env.example
+15 -1
View File
@@ -16,9 +16,14 @@ ZONOS_TAG=v1
# Reserved on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS,
# 8192 IndexTTS-2, 8193 Kokoro, 8194 VibeVoice, 8195 Fish-S2,
# 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai, 8200 Dia, 8765 Parakeet.
# 8199 picked here.
# 8199 picked here for the Gradio eval UI.
ZONOS_PORT=8199
# Host port for the OpenAI-ish REST adapter (zonos-api service) that
# asset-engine routes to. Container listens on 8000 internally. 8201 is
# the next free slot above the Gradio port.
ZONOS_API_PORT=8201
# Bind address. 0.0.0.0 exposes on all interfaces (incl. WG tunnel
# interface 10.100.79.3); 127.0.0.1 restricts to local-only.
ZONOS_BIND=0.0.0.0
@@ -42,3 +47,12 @@ ZONOS_GPU_DEVICES=0
# Bind-mounted so they survive container recreate. Excluded from restic
# (regenerable from HF).
ZONOS_CACHE_DIR=/worktank/zonos/cache
# Reference voice clips for zero-shot cloning (mounted read-only into the
# adapter at /app/voices). Drop 10–30s clean WAVs here; the catalog's
# `voice` field selects one by filename. Empty/absent → default voice.
ZONOS_VOICES_DIR=/worktank/zonos/voices
# Model the adapter loads at startup. Only the transformer variant is
# available in the current image (mamba-ssm absent → no hybrid).
ZONOS_MODEL=Zyphra/Zonos-v0.1-transformer
stacks/zonos/README.md
+29 -9
View File
@@ -13,7 +13,7 @@ Models ([`Zonos.from_pretrained()`](https://github.com/Zyphra/Zonos)):
| hybrid | [Zyphra/Zonos-v0.1-hybrid](https://huggingface.co/Zyphra/Zonos-v0.1-hybrid) | Mamba-SSM; needs Ampere+ GPU + extra build deps |
**Server:** irv-ml1 (Irvine, WireGuard-only)
**Port:** 8199 (container Gradio listens on 7860)
**Ports:** 8199 Gradio eval UI (container 7860) · 8201 REST adapter (container 8000)
**GPUs:** pins to device 0 (RTX 3090) by default; ~6 GB VRAM
**Image:** `local/zonos:v1` — built locally from a pinned git SHA of the
upstream repo via docker buildx's git URL context
@@ -29,17 +29,37 @@ natural-language paralinguistic tags. Worth A/B-ing by ear against
Chatterbox-Turbo (the research that prompted this stack explicitly said
"benchmark Zonos against Chatterbox before choosing").
## ⚠️ Audition surface, not skaldsong-pluggable (yet)
## Two surfaces: Gradio eval (8199) + REST adapter (8201)
The official repo ships a **Gradio WebUI + Python SDK only** — there is
**no OpenAI-compatible `/v1/audio/speech` endpoint**. So this stack is
for *auditioning quality*, not for wiring into skaldsong's engine
router as-is. To promote Zonos to a real engine slot we'd need either:
The official repo ships a **Gradio WebUI + Python SDK only** — no REST
endpoint. So this stack runs two services:
- the community FastAPI fork ([Zyphra/Zonos PR #73](https://github.com/Zyphra/Zonos/pull/73), adds REST + basic streaming), or
- a thin OpenAI-compat adapter in front of the Python SDK.
- **`zonos`** (8199) — upstream Gradio UI, for *auditioning quality by ear*.
- **`zonos-api`** (8201) — a thin OpenAI-ish adapter we built
(`adapter/server.py`) exposing `POST /v1/audio/speech` so **asset-engine**
can route to Zonos like every other TTS in the catalog. Returns a JSON
envelope `{audio: <base64>, audio_format, seed}` — the `seed` rides back
so regenerate/fork can pin it (Zonos is the fleet's first seedable TTS).
Both are a follow-up if Zonos earns a slot in the ear test.
The adapter is built `FROM local/zonos:<tag>` (reuses torch/CUDA/SDK) and
loads its own copy of the model (~6 GB on top of the Gradio service). Once
Zonos earns a permanent slot, drop the Gradio service and keep the adapter.
The community FastAPI fork ([PR #73](https://github.com/Zyphra/Zonos/pull/73))
was the alternative; we chose the self-owned adapter over pinning to an
unmerged fork.
### Adapter request (example)
```bash
curl -sS http://10.100.79.3:8201/v1/audio/speech \
-H 'content-type: application/json' \
-d '{"input":"Hello from Zonos.","language":"en-us","seed":420}' \
| python3 -c 'import sys,json,base64; d=json.load(sys.stdin); open("out.wav","wb").write(base64.b64decode(d["audio"])); print("seed",d["seed"])'
```
Cloning: drop a 10–30s WAV in `/worktank/zonos/voices/` and pass its
filename as `"voice"`. `GET /v1/audio/voices` lists what's available.
## Deploy
stacks/zonos/adapter/Dockerfile
+22
View File
@@ -0,0 +1,22 @@
# Thin OpenAI-ish REST adapter, layered on top of the already-built
# upstream Zonos image (local/zonos:<tag>) so we reuse its torch + CUDA +
# espeak-ng + Zonos SDK install rather than rebuilding the heavy base.
#
# Build arg ZONOS_BASE lets compose pin which base tag to layer on.
ARG ZONOS_BASE=local/zonos:v1
FROM ${ZONOS_BASE}
# libsndfile1 backs soundfile's wav/flac encode (the base image lacks it).
RUN apt-get update \
&& apt-get install -y --no-install-recommends libsndfile1 \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt
WORKDIR /app
COPY server.py /app/server.py
# Container listens on 8000 internally; compose maps it to the host port.
EXPOSE 8000
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
stacks/zonos/adapter/__pycache__/server.cpython-311.pyc
BIN
View File
Binary file not shown.
stacks/zonos/adapter/requirements.txt
+6
View File
@@ -0,0 +1,6 @@
# Adapter-only deps. torch / torchaudio / the zonos SDK already live in
# the base image (local/zonos) — do NOT reinstall them here or the CUDA
# build gets clobbered.
fastapi==0.115.6
uvicorn[standard]==0.34.0
soundfile==0.12.1
stacks/zonos/adapter/server.py
+174
View File
@@ -0,0 +1,174 @@
"""OpenAI-ish /v1/audio/speech adapter in front of the Zonos Python SDK.
Why this exists: the upstream Zyphra/Zonos repo ships only a Gradio WebUI
+ Python SDK — no REST endpoint. asset-engine routes generation over a
clean JSON POST (every other TTS in the catalog speaks /v1/audio/speech),
so this thin FastAPI layer adapts the SDK's generate path to that wire.
Wire shape (request): POST /v1/audio/speech application/json
Wire shape (response): JSON envelope {audio: <base64>, audio_format:
"audio/wav", seed: <int>} — mirrors the
kokoro-captioned envelope, plus `seed` so
asset-engine's regenerate/fork can pin it
(Zonos is the first genuinely seedable TTS in the
fleet: catalog reproducibility.seed_field=seed).
The model is loaded once at startup. Generation is serialized behind an
asyncio lock — the SDK model and the speaker-embedding cache are not
concurrency-safe, and there is one GPU.
"""
from __future__ import annotations
import asyncio
import base64
import io
import os
from typing import Optional
import soundfile as sf
import torch
import torchaudio
from fastapi import FastAPI, HTTPException
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from zonos.conditioning import make_cond_dict, supported_language_codes
from zonos.model import Zonos
from zonos.utils import DEFAULT_DEVICE as DEVICE
MODEL_ID = os.getenv("ZONOS_MODEL", "Zyphra/Zonos-v0.1-transformer")
VOICES_DIR = os.getenv("ZONOS_VOICES_DIR", "/app/voices")
# Blessed defaults lifted verbatim from upstream gradio_interface.py — the
# model authors picked these for human-facing UX (CATALOG-CONTRACT source
# precedence: Gradio UI for blessed defaults/ranges).
SAMPLER = dict(top_p=0.0, top_k=0, min_p=0.0, linear=0.5, conf=0.40, quad=0.00)
MAX_NEW_TOKENS = 86 * 30 # ~30s ceiling, as upstream
app = FastAPI(title="zonos-adapter", version="0.1.0")
_lock = asyncio.Lock()
_model: Optional[Zonos] = None
# (voice_path -> embedding) cache; recomputing the speaker embedding per
# request is the dominant avoidable cost for repeated clones.
_spk_cache: dict[str, torch.Tensor] = {}
class SpeechRequest(BaseModel):
input: str = Field(..., description="Text to synthesize")
model: str = Field(MODEL_ID, description="Only the transformer variant is loaded")
voice: Optional[str] = Field(
None, description="Reference clip name under VOICES_DIR (cloning); omit for default voice"
)
language: str = Field("en-us", description="eSpeak language code")
response_format: str = Field("wav", description="wav | flac")
seed: Optional[int] = Field(
None, description="Omit for a random seed; the seed actually used is returned"
)
# Conditioning knobs (ranges mirror the Gradio sliders).
speaking_rate: float = Field(15.0, ge=5.0, le=30.0)
pitch_std: float = Field(45.0, ge=0.0, le=300.0)
fmax: float = Field(24000.0, ge=0.0, le=24000.0)
cfg_scale: float = Field(2.0, ge=1.0, le=5.0)
# 8-float emotion vector [happy, sad, disgust, fear, surprise, anger,
# other, neutral]; omit to leave emotion unconditional (Gradio default).
emotion: Optional[list[float]] = Field(None, min_length=8, max_length=8)
def _get_model() -> Zonos:
global _model
if _model is None:
_model = Zonos.from_pretrained(MODEL_ID, device=DEVICE)
_model.requires_grad_(False).eval()
return _model
def _speaker_embedding(model: Zonos, voice: str) -> torch.Tensor:
path = voice if os.path.isabs(voice) else os.path.join(VOICES_DIR, voice)
if not os.path.isfile(path):
raise HTTPException(404, f"voice not found: {voice}")
if path not in _spk_cache:
wav, sr = torchaudio.load(path)
emb = model.make_speaker_embedding(wav, sr).to(DEVICE, dtype=torch.bfloat16)
_spk_cache[path] = emb
return _spk_cache[path]
_FORMATS = {"wav": "audio/wav", "flac": "audio/flac"}
@app.get("/health")
async def health() -> dict:
return {"status": "ok", "model": MODEL_ID, "loaded": _model is not None}
@app.get("/v1/audio/voices")
async def voices() -> dict:
try:
names = sorted(f for f in os.listdir(VOICES_DIR) if f.lower().endswith((".wav", ".flac", ".mp3")))
except FileNotFoundError:
names = []
return {"voices": names}
@app.post("/v1/audio/speech")
async def speech(req: SpeechRequest) -> JSONResponse:
if req.response_format not in _FORMATS:
raise HTTPException(400, f"unsupported response_format: {req.response_format}")
if req.language not in supported_language_codes:
raise HTTPException(400, f"unsupported language: {req.language}")
async with _lock:
model = _get_model()
# Resolve seed: explicit pins it; otherwise draw one and report it
# back so asset-engine can regenerate/fork deterministically.
seed = req.seed if req.seed is not None else int(torch.randint(0, 2**32 - 1, (1,)).item())
torch.manual_seed(seed)
unconditional_keys: list[str] = []
speaker = None
if req.voice:
speaker = _speaker_embedding(model, req.voice)
else:
unconditional_keys.append("speaker")
if req.emotion is not None:
emotion = torch.tensor([float(x) for x in req.emotion], device=DEVICE)
else:
emotion = None
unconditional_keys.append("emotion")
cond = make_cond_dict(
text=req.input,
language=req.language,
speaker=speaker,
emotion=emotion,
fmax=float(req.fmax),
pitch_std=float(req.pitch_std),
speaking_rate=float(req.speaking_rate),
device=DEVICE,
unconditional_keys=unconditional_keys,
)
conditioning = model.prepare_conditioning(cond)
with torch.inference_mode():
codes = model.generate(
prefix_conditioning=conditioning,
max_new_tokens=MAX_NEW_TOKENS,
cfg_scale=float(req.cfg_scale),
batch_size=1,
sampling_params=SAMPLER,
)
wav = model.autoencoder.decode(codes).cpu().detach()
sr = model.autoencoder.sampling_rate
if wav.dim() == 2 and wav.size(0) > 1:
wav = wav[0:1, :]
samples = wav.squeeze().to(torch.float32).numpy()
buf = io.BytesIO()
sf.write(buf, samples, sr, format=req.response_format.upper())
audio_b64 = base64.b64encode(buf.getvalue()).decode("ascii")
return JSONResponse(
{"audio": audio_b64, "audio_format": _FORMATS[req.response_format], "seed": seed}
)
stacks/zonos/compose.yaml
+37
View File
@@ -63,3 +63,40 @@ services:
- homepage.icon=mdi-waveform
- homepage.description=Expressive multilingual TTS + cloning, 44kHz (Gradio eval, irv-ml1)
- homepage.href=http://10.100.79.3:${ZONOS_PORT}
# OpenAI-ish REST adapter (POST /v1/audio/speech) that asset-engine
# routes to — upstream Zonos has no REST surface, only Gradio + SDK.
# Built FROM the gradio image above (reuses torch/CUDA/SDK); loads its
# own copy of the model, so it adds ~6 GB VRAM on top of the gradio
# service. Drop the gradio service once Zonos earns a permanent slot.
zonos-api:
image: local/zonos-api:${ZONOS_TAG}
build:
context: ./adapter
dockerfile: Dockerfile
args:
ZONOS_BASE: local/zonos:${ZONOS_TAG}
container_name: zonos-api
restart: unless-stopped
runtime: nvidia
depends_on:
- zonos
ports:
- "${ZONOS_BIND:-0.0.0.0}:${ZONOS_API_PORT}:8000"
environment:
- NVIDIA_VISIBLE_DEVICES=${ZONOS_GPU_DEVICES:-0}
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
- HF_HOME=/app/hf_cache
- ZONOS_MODEL=${ZONOS_MODEL:-Zyphra/Zonos-v0.1-transformer}
- ZONOS_VOICES_DIR=/app/voices
volumes:
- ${ZONOS_CACHE_DIR}:/app/hf_cache
# Reference clips for zero-shot cloning, selected by the `voice`
# request field (filename under this dir).
- ${ZONOS_VOICES_DIR}:/app/voices:ro
healthcheck:
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5); sys.exit(0)\""]
interval: 30s
timeout: 10s
retries: 3
start_period: 600s