feat(morpheus): permanent mOrpheus TTS stack (vLLM bf16 + SNAC/FastAPI wrapper) on irv-ml1
Two-container stack serving MrDragonFox/mOrpheus (uncensored Orpheus TTS, Llama-3.2-3B -> SNAC 24kHz). vllm-morpheus (GPU/3090) emits Orpheus audio tokens; morpheus-tts (CPU) SNAC-decodes them to WAV and exposes POST /tts (baddy voice + zero-shot cloning). Deployed + tested end-to-end (28/28 valid frames, valid WAV, reachable over WG). Hard-won config, all encoded in compose/README: - bf16 REQUIRED: --quantization fp8 destroys audio-token generation (0 valid SNAC frames even at greedy). Footprint ~7.9GB. - Image PINNED to v0.23.0: 'latest' ships Blackwell oink/aiter kernels that crash on Ampere import. - 3090 (not the comfy-contended A6000); --enforce-eager to fit the shared card. - RTF ~1.0 end-to-end (gen ~98 tok/s / RTF 0.84 + CPU decode + HTTP). INTERNAL RESEARCH ONLY (CC-BY-NC-4.0); do not expose externally.
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
# mOrpheus — uncensored Orpheus TTS, permanent stack on irv-ml1.
|
||||
#
|
||||
# Two services:
|
||||
# vllm-morpheus (GPU, 3090, FP8) — serves the mOrpheus 3B LLM; emits Orpheus audio
|
||||
# tokens. Real-time: ~165 tok/s single-stream => RTF ~0.50 (2x real-time), measured.
|
||||
# morpheus-tts (CPU) — SNAC-decodes those tokens to 24kHz WAV; the public /tts endpoint
|
||||
# (baddy voice + optional zero-shot cloning). Kept off the GPU to avoid contention.
|
||||
#
|
||||
# PLACEMENT / GOTCHAS (irv-ml1, INTERNAL RESEARCH — CC-BY-NC, do NOT expose externally):
|
||||
# - GPU = the 3090 (audio-zoo GPU, stabler); the A6000 is comfy's and spikes to ~41GB
|
||||
# without warning (it OOM'd two launch attempts). Pin by UUID via MORPHEUS_GPU_UUID.
|
||||
# - Image PINNED to vllm/vllm-openai:v0.23.0 — `latest` ships Blackwell-only kernels
|
||||
# (oink/aiter) that crash on Ampere import. Do NOT bump to latest on this Ampere box.
|
||||
# - bf16 REQUIRED, NOT FP8. FP8 (--quantization fp8) destroys audio-token generation on
|
||||
# this model — 0 valid SNAC frames even at greedy (quant-breaks-TTS). bf16 weights are
|
||||
# ~6.6GB; with --enforce-eager (no CUDA-graph capture) it fits the shared 3090 (~8GB
|
||||
# free after nothing else vLLM). util 0.32 (~7.7GB of the 24GB 3090).
|
||||
# All tunables in .env — edit that, not this file.
|
||||
|
||||
name: morpheus
|
||||
|
||||
services:
|
||||
vllm-morpheus:
|
||||
image: vllm/vllm-openai:v0.23.0
|
||||
container_name: vllm-morpheus
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ipc: host
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${MORPHEUS_GPU_UUID:-GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72}
|
||||
- CUDA_DEVICE_ORDER=PCI_BUS_ID
|
||||
volumes:
|
||||
- ${MORPHEUS_MODEL_DIR:-/home/lkraven/morpheus/models/mOrpheus}:/model:ro
|
||||
command:
|
||||
- /model
|
||||
- --served-model-name
|
||||
- morpheus
|
||||
- --dtype
|
||||
- bfloat16
|
||||
# NO --quantization: FP8 breaks audio-token generation on this model (see header).
|
||||
- --enforce-eager
|
||||
- --max-model-len
|
||||
- "${MORPHEUS_MAX_LEN:-4096}"
|
||||
- --gpu-memory-utilization
|
||||
- "${MORPHEUS_GPU_UTIL:-0.32}"
|
||||
- --max-num-seqs
|
||||
- "${MORPHEUS_MAX_SEQS:-8}"
|
||||
healthcheck:
|
||||
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 300s
|
||||
networks: [morpheus-net]
|
||||
|
||||
morpheus-tts:
|
||||
build: ./tts
|
||||
image: morpheus-tts:local
|
||||
container_name: morpheus-tts
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
vllm-morpheus:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
- VLLM_URL=http://vllm-morpheus:8000/v1/completions
|
||||
- VLLM_MODEL=morpheus
|
||||
- SNAC_DEVICE=cpu
|
||||
- MORPHEUS_DEFAULT_VOICE=${MORPHEUS_DEFAULT_VOICE:-baddy}
|
||||
- MORPHEUS_VOICES=${MORPHEUS_VOICES:-baddy}
|
||||
volumes:
|
||||
- ${MORPHEUS_MODEL_DIR:-/home/lkraven/morpheus/models/mOrpheus}:/model:ro
|
||||
- ${SNAC_DIR:-/home/lkraven/morpheus/models/snac_24khz}:/snac:ro
|
||||
ports:
|
||||
- "${MORPHEUS_TTS_PORT:-8299}:8000"
|
||||
healthcheck:
|
||||
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=mOrpheus TTS
|
||||
- homepage.icon=mdi-account-voice
|
||||
- homepage.description=Uncensored Orpheus TTS (baddy + zero-shot clone), real-time, irv-ml1
|
||||
- homepage.href=http://10.100.79.3:${MORPHEUS_TTS_PORT:-8299}/docs
|
||||
networks: [morpheus-net]
|
||||
|
||||
networks:
|
||||
morpheus-net:
|
||||
name: morpheus-net
|
||||
Reference in New Issue
Block a user