ace-step + stable-audio-open: deploy music + SFX generation to irv-ml1

Two new audio-generation stacks alongside the TTS slate: ace-step :8210 — Apache 2.0 music generation foundation model (hybrid diffusion + LLM). Lyric-aware multi-minute songs. ~10-12 GB VRAM during inference, A6000-pinned. Custom Dockerfile patches upstream's torch/cu126 resolution bug (--extra-index-url cu126 was falling back to pypi-default cu13 wheels, mismatching torchvision). stable-audio-open :8211 — Stability AI 1.21B latent-diffusion SFX + ambience. Up to 47s clips at 44.1 kHz. ~6 GB VRAM in fp16, A6000-pinned. Custom FastAPI shim around diffusers' StableAudioPipeline (no upstream HTTP server). Dockerfile pins torchsde explicitly — diffusers doesn't pull it as a hard dep but CosineDPMSolverMultistepScheduler needs it.
2026-04-28 09:11:23 -07:00
parent 0ba41e02ea
commit 4a4c09177f
11 changed files with 732 additions and 0 deletions
@@ -0,0 +1,83 @@
+# Deploy ACE-Step 1.5 (Apache 2.0 music generation foundation model)
+# to irv-ml1.
+#
+# Builds the image locally from ace-step/ACE-Step via docker buildx
+# git URL context. ~10-15 min cold build (CUDA 12.6 runtime + torch +
+# transformers + spacy + audio deps). First /generate triggers the
+# model download (~5-10 GB) into the bind-mounted HF cache + warmup.
+#
+# Pre-req (user runs once):
+#   ssh -t irv-ml1 'sudo mkdir -p /worktank/ace-step/{checkpoints,outputs,logs,hf_cache} \
+#       /opt/docker/compose/ace-step && \
+#       sudo chown -R lkraven:lkraven /worktank/ace-step /opt/docker/compose/ace-step'
+#
+# Usage:
+#   scripts/elway irv-ml1 --playbook playbooks/deploy-ace-step.yaml
+#
+# Idempotent — every step is creates-/when-gated; rerun is safe.
+
+vars:
+  compose_dir: /opt/docker/compose/ace-step
+  worktank_root: /worktank/ace-step
+  host_port: "8210"
+
+steps:
+  # ── sanity checks (dirs were created by user-side sudo prep) ────────
+
+  - name: Verify /worktank/ace-step exists and is writable
+    shell: test -w {{ worktank_root }}
+    changed_when: "false"
+
+  - name: Verify compose dir exists and is writable
+    shell: test -w {{ compose_dir }}
+    changed_when: "false"
+
+  # ── deploy compose + env ────────────────────────────────────────────
+
+  - name: Upload compose.yaml
+    upload:
+      src: stacks/ace-step/compose.yaml
+      dest: "{{ compose_dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Upload Dockerfile (patched for cu126 torch resolution)
+    upload:
+      src: stacks/ace-step/Dockerfile
+      dest: "{{ compose_dir }}/Dockerfile"
+      mode: "0644"
+
+  - name: Seed .env from template (only if absent)
+    upload:
+      src: stacks/ace-step/.env.example
+      dest: "{{ compose_dir }}/.env"
+      mode: "0644"
+    when: "[ ! -f {{ compose_dir }}/.env ]"
+
+  # ── build + bring up ────────────────────────────────────────────────
+
+  - name: docker compose build (~10-15 min first time; cached after)
+    shell: |
+      set -o pipefail
+      cd {{ compose_dir }} && docker compose build 2>&1 \
+        | grep -vE '^#[0-9]+ |^ => |^=> |Collecting|Downloading|Requirement|Using cached|Installing collected|Successfully (installed|built)|━'
+
+  - name: docker compose up -d
+    shell: cd {{ compose_dir }} && docker compose up -d
+
+  - name: Wait for /health to respond (allow ~10 min for first model download + warmup)
+    shell: |
+      for i in $(seq 1 120); do
+        curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/health && exit 0
+        sleep 5
+      done
+      exit 1
+    changed_when: "false"
+
+verify:
+  - name: /health returns 200
+    shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/health
+    changed_when: "false"
+
+  - name: Container is running
+    shell: docker inspect ace-step --format '{{.State.Status}}' | grep -q running
+    changed_when: "false"