feat(morpheus): permanent mOrpheus TTS stack (vLLM bf16 + SNAC/FastAPI wrapper) on irv-ml1
Two-container stack serving MrDragonFox/mOrpheus (uncensored Orpheus TTS, Llama-3.2-3B -> SNAC 24kHz). vllm-morpheus (GPU/3090) emits Orpheus audio tokens; morpheus-tts (CPU) SNAC-decodes them to WAV and exposes POST /tts (baddy voice + zero-shot cloning). Deployed + tested end-to-end (28/28 valid frames, valid WAV, reachable over WG). Hard-won config, all encoded in compose/README: - bf16 REQUIRED: --quantization fp8 destroys audio-token generation (0 valid SNAC frames even at greedy). Footprint ~7.9GB. - Image PINNED to v0.23.0: 'latest' ships Blackwell oink/aiter kernels that crash on Ampere import. - 3090 (not the comfy-contended A6000); --enforce-eager to fit the shared card. - RTF ~1.0 end-to-end (gen ~98 tok/s / RTF 0.84 + CPU decode + HTTP). INTERNAL RESEARCH ONLY (CC-BY-NC-4.0); do not expose externally.
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# mOrpheus stack tunables (irv-ml1). Copy to .env on the host; edit values there.
|
||||
|
||||
# GPU pin (UUID). Default = the 3090 (the stabler audio-zoo GPU; A6000 is comfy's + spiky).
|
||||
# irv-ml1: 3090 = GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72
|
||||
# A6000 = GPU-9672f0d5-3caf-b15c-898a-cd9d653c654b
|
||||
MORPHEUS_GPU_UUID=GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72
|
||||
|
||||
# Weights (stage once; not in git). mOrpheus = LlamaForCausalLM 3.3B bf16 (6.6GB).
|
||||
MORPHEUS_MODEL_DIR=/home/lkraven/morpheus/models/mOrpheus
|
||||
SNAC_DIR=/home/lkraven/morpheus/models/snac_24khz
|
||||
|
||||
# Engine. bf16 REQUIRED — FP8 destroys audio-token generation on this model (0 valid SNAC
|
||||
# frames even at greedy). bf16 weights ~6.6GB; --enforce-eager (in compose) skips CUDA-
|
||||
# graph capture so it fits the shared 3090. util 0.32 (~7.7GB of the 24GB card); raise if
|
||||
# you free the 3090, lower only if you can (bf16 weights alone are 6.6GB => >=~0.30 floor).
|
||||
MORPHEUS_GPU_UTIL=0.32
|
||||
MORPHEUS_MAX_LEN=4096
|
||||
MORPHEUS_MAX_SEQS=8
|
||||
|
||||
# TTS endpoint (host port). Public /tts + /docs live here.
|
||||
MORPHEUS_TTS_PORT=8299
|
||||
MORPHEUS_DEFAULT_VOICE=baddy
|
||||
MORPHEUS_VOICES=baddy
|
||||
Reference in New Issue
Block a user