feat(morpheus): permanent mOrpheus TTS stack (vLLM bf16 + SNAC/FastAPI wrapper) on irv-ml1

Two-container stack serving MrDragonFox/mOrpheus (uncensored Orpheus TTS, Llama-3.2-3B
-> SNAC 24kHz). vllm-morpheus (GPU/3090) emits Orpheus audio tokens; morpheus-tts (CPU)
SNAC-decodes them to WAV and exposes POST /tts (baddy voice + zero-shot cloning). Deployed
+ tested end-to-end (28/28 valid frames, valid WAV, reachable over WG).

Hard-won config, all encoded in compose/README:
- bf16 REQUIRED: --quantization fp8 destroys audio-token generation (0 valid SNAC frames
  even at greedy). Footprint ~7.9GB.
- Image PINNED to v0.23.0: 'latest' ships Blackwell oink/aiter kernels that crash on Ampere
  import.
- 3090 (not the comfy-contended A6000); --enforce-eager to fit the shared card.
- RTF ~1.0 end-to-end (gen ~98 tok/s / RTF 0.84 + CPU decode + HTTP).

INTERNAL RESEARCH ONLY (CC-BY-NC-4.0); do not expose externally.
This commit is contained in:
vh
2026-07-09 00:49:25 -07:00
parent 99a4a1721f
commit 01eedd8d27
6 changed files with 315 additions and 0 deletions
+91
View File
@@ -0,0 +1,91 @@
# mOrpheus — uncensored Orpheus TTS, permanent stack on irv-ml1.
#
# Two services:
# vllm-morpheus (GPU, 3090, FP8) — serves the mOrpheus 3B LLM; emits Orpheus audio
# tokens. Real-time: ~165 tok/s single-stream => RTF ~0.50 (2x real-time), measured.
# morpheus-tts (CPU) — SNAC-decodes those tokens to 24kHz WAV; the public /tts endpoint
# (baddy voice + optional zero-shot cloning). Kept off the GPU to avoid contention.
#
# PLACEMENT / GOTCHAS (irv-ml1, INTERNAL RESEARCH — CC-BY-NC, do NOT expose externally):
# - GPU = the 3090 (audio-zoo GPU, stabler); the A6000 is comfy's and spikes to ~41GB
# without warning (it OOM'd two launch attempts). Pin by UUID via MORPHEUS_GPU_UUID.
# - Image PINNED to vllm/vllm-openai:v0.23.0 — `latest` ships Blackwell-only kernels
# (oink/aiter) that crash on Ampere import. Do NOT bump to latest on this Ampere box.
# - bf16 REQUIRED, NOT FP8. FP8 (--quantization fp8) destroys audio-token generation on
# this model — 0 valid SNAC frames even at greedy (quant-breaks-TTS). bf16 weights are
# ~6.6GB; with --enforce-eager (no CUDA-graph capture) it fits the shared 3090 (~8GB
# free after nothing else vLLM). util 0.32 (~7.7GB of the 24GB 3090).
# All tunables in .env — edit that, not this file.
name: morpheus
services:
vllm-morpheus:
image: vllm/vllm-openai:v0.23.0
container_name: vllm-morpheus
restart: unless-stopped
runtime: nvidia
ipc: host
environment:
- NVIDIA_VISIBLE_DEVICES=${MORPHEUS_GPU_UUID:-GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72}
- CUDA_DEVICE_ORDER=PCI_BUS_ID
volumes:
- ${MORPHEUS_MODEL_DIR:-/home/lkraven/morpheus/models/mOrpheus}:/model:ro
command:
- /model
- --served-model-name
- morpheus
- --dtype
- bfloat16
# NO --quantization: FP8 breaks audio-token generation on this model (see header).
- --enforce-eager
- --max-model-len
- "${MORPHEUS_MAX_LEN:-4096}"
- --gpu-memory-utilization
- "${MORPHEUS_GPU_UTIL:-0.32}"
- --max-num-seqs
- "${MORPHEUS_MAX_SEQS:-8}"
healthcheck:
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 30s
timeout: 10s
retries: 3
start_period: 300s
networks: [morpheus-net]
morpheus-tts:
build: ./tts
image: morpheus-tts:local
container_name: morpheus-tts
restart: unless-stopped
depends_on:
vllm-morpheus:
condition: service_healthy
environment:
- VLLM_URL=http://vllm-morpheus:8000/v1/completions
- VLLM_MODEL=morpheus
- SNAC_DEVICE=cpu
- MORPHEUS_DEFAULT_VOICE=${MORPHEUS_DEFAULT_VOICE:-baddy}
- MORPHEUS_VOICES=${MORPHEUS_VOICES:-baddy}
volumes:
- ${MORPHEUS_MODEL_DIR:-/home/lkraven/morpheus/models/mOrpheus}:/model:ro
- ${SNAC_DIR:-/home/lkraven/morpheus/models/snac_24khz}:/snac:ro
ports:
- "${MORPHEUS_TTS_PORT:-8299}:8000"
healthcheck:
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
labels:
- homepage.group=AI Systems
- homepage.name=mOrpheus TTS
- homepage.icon=mdi-account-voice
- homepage.description=Uncensored Orpheus TTS (baddy + zero-shot clone), real-time, irv-ml1
- homepage.href=http://10.100.79.3:${MORPHEUS_TTS_PORT:-8299}/docs
networks: [morpheus-net]
networks:
morpheus-net:
name: morpheus-net