The 2026-09-06 headscale cutover retired irv-ml1's wg0 tunnel IP 10.100.79.3
(now 10.6.110.50). Repointed all LIVE canonical refs to the DNS NAME so the next
move can't re-break them: homepage.href/siteMonitor labels across 25 stack
composes, load-bearing env defaults (asset-engine INFERENCE_HOST, open-webui
AUDIO_TTS_OPENAI_API_BASE_URL, skaldsong SKALDSONG_TTS_BASE_URL, zonos-gateway
ZONOS_URL, dia), homepage services.yaml manual cards (Voice Design Studio,
IRV-ML1), and servers/irv-ml1/ssh-target. Updated the stale 'WG tunnel' comment
to the mesh reality.
Left as-is: README curl-examples and .env.example comments (docs), and historical
mentions in CLAUDE.md/persistent-memory. NOTE: applying the label repoints to the
RUNNING irv-ml1 containers needs a recreate per service (labels read at creation);
deployed .env values are separate from these canonical defaults.
Three upstream gaps surfaced once /generate was actually exercised:
1. infer-api.py builds an 18-arg positional tuple but the pipeline
expects 24 — first missing arg is `format`, so audio_duration
shifts into format's slot and the pipeline calls len() on an
int. Ship a patched copy of infer-api.py and COPY over upstream's
in the Dockerfile. Also handle empty lora_name_or_path -> "none"
(empty string trips HF Hub's repo-id validator).
2. torchcodec + ffmpeg are required by the WAV save path but neither
is in upstream requirements.txt. Without them every /generate
runs to completion and then 500s at write-time.
3. ACE-Step caches checkpoints at /root/.cache/ace-step/checkpoints
(HARDCODED, not honored by HF_HOME). Mount our persistent dir
there so the ~7 GB model survives container recreates.
Bench on A6000 (cached model, lo-fi hip hop, 60-step euler/apg):
10s @ 27 steps -> 9.4s (0.94x)
30s @ 60 steps -> 11.2s (0.37x, ~2.7x realtime)
60s @ 60 steps -> 14.8s (0.24x, ~4x realtime)
Two new audio-generation stacks alongside the TTS slate:
ace-step :8210 — Apache 2.0 music generation foundation model
(hybrid diffusion + LLM). Lyric-aware multi-minute songs. ~10-12 GB
VRAM during inference, A6000-pinned. Custom Dockerfile patches
upstream's torch/cu126 resolution bug (--extra-index-url cu126 was
falling back to pypi-default cu13 wheels, mismatching torchvision).
stable-audio-open :8211 — Stability AI 1.21B latent-diffusion SFX +
ambience. Up to 47s clips at 44.1 kHz. ~6 GB VRAM in fp16,
A6000-pinned. Custom FastAPI shim around diffusers' StableAudioPipeline
(no upstream HTTP server). Dockerfile pins torchsde explicitly —
diffusers doesn't pull it as a hard dep but
CosineDPMSolverMultistepScheduler needs it.
vh