vh
2941158c70
Current state rewritten to the active build task: containerize the production
Zonos2 :1920 TTS engine as a self-contained image (config captured in 14a0004)
and prototype the operator's prosody-priming hypothesis (prime→generate-one-
utterance→parakeet-clip→deliver, adapter-level, engine stays stock). New detail
file + Recent-decisions pointer capture the full plan + the crux (does AR prosody
carry the sentence boundary — A/B the join). /tmp handoff written for the fresh session.
4.2 KiB
[2026-07-17]Zonos2:1920engine → self-contained container (stays on 3090); prosody-priming is a SERVING-LAYER change (engine stays stock).
Context. The production Zonos TTS engine (irv-ml1 :1920, feeds asset-engine + gateway-chat via zonos-gateway :8890) was a bare native process — its real launch config existed ONLY in the running process argv (the committed ~/tts-audition/harness/zonos_server.sh was STALE: said A6000/:1919/no perf flags; live is 3090/:1920 with --cuda-graph-max-bs 1 --num-pages 16384 --max-running-requests 2 --memory-ratio 0.3). Captured to eshpfi stacks/zonos-engine/ (README + corrected zonos2-server.sh + .env.example), commit 14a0004 (UNPUSHED as of the snapshot).
Decision 1 — containerize as a SELF-CONTAINED image (not systemd — operator rejected; not a thin bind-mount wrapper — I walked that back: bind-mounting the host's CUDA-compiled .venv couples to the host's exact CUDA/glibc and is fragile + not reproducible). Shape: FROM a CUDA 12.8 base → uv sync against the repo's committed uv.lock (deterministic env) → mount the ~15 GB HF weights (~/.cache/huggingface/hub/models--Zyphra--ZONOS2, do NOT bake) → pin the 3090 (NVIDIA_VISIBLE_DEVICES=0) → restart: unless-stopped → CMD = the captured invocation. Engine stays STOCK Zyphra/Zonos2 @ commit 194c0a3 (no fork — the zonos2 package ships its own server). Build risk: heavy compiled-CUDA deps (flashinfer / sgl_kernel / cutlass-dsl / apache-tvm-ffi / pynini) on torch 2.9.1+cu128 — mostly prebuilt wheels + the uv.lock make it tractable, expect a couple build iterations. Cutover (in place on the 3090): stop the native process (frees ~17 GB) → docker compose up -d (re-allocates ~17 GB, same footprint) → repoint zonos-gateway's ZONOS_URL at the container (or keep the :1920 host-port publish). One brief prod-TTS blip.
GPU = 3090 (operator 2026-07-17). Keep it OFF the A6000 — the A6000 already OOMs under ComfyUI load (idle ~19 GB but spikes far higher during gen), so it can't host Zonos too. The 3090 already runs Zonos, so the containerize-in-place cutover changes nothing about placement.
Decision 2 — the prosody-priming hypothesis (operator's test; the reason for building fresh). PRIME the autoregressive engine with an emotional sentence, then TRUNCATE it from delivery: prepend a primer → generate "primer + real text" as ONE continuous utterance (the AR model carries prosody forward across the boundary) → ASR-timestamp the primer's end (parakeet, already up on irv-ml1 :8765, word timestamps) → clip the primer in the inter-sentence silence gap (+ ~15 ms fade-in, no click) → deliver only the real text, now wearing the primed prosody. Examples: primer "I'm so EXCITED about this." → "This will be a lot of fun!" spoken excited; primer "I'm whispering this to you right now." → "I'm so glad to see you baby." whispered. This is PURE serving-layer orchestration — the engine is untouched; it lives in the gateway adapter stacks/zonos/adapter/server.py. Only fork the engine if the black-box approach fails.
THE CRUX the test resolves: does AR prosody actually carry across the sentence boundary, or does Zonos reset at the period? → the harness A/Bs the JOIN punctuation: period (operator's examples) vs comma vs ellipsis vs none ("…excited about this, this will be…"). Everything else is plumbing.
Plan / design recs. (a) Build the stock engine image (parallel track). (b) Stand up a priming TEST HARNESS against the NATIVE engine (fast iteration, seconds) + parakeet ASR: prime→generate→timestamp→gap-clip→out; compare primed-clipped vs plain on the two cases (subjective + a cheap objective proxy: pitch/energy variance for "excited", spectral-tilt/low-energy for "whisper"). Iterate on the join, then bake the winner into the gateway adapter. Primer source: caller-supplied for the harness (test arbitrary primers) → a curated emotion→primer library (excited/whisper/…) + optional caller override for production. ASR: parakeet primary; WhisperX forced-align fallback if parakeet word timestamps are coarse.
See eshpfi stacks/zonos-engine/README.md + stacks/zonos/ (the gateway adapter).