vh
01eedd8d27
Two-container stack serving MrDragonFox/mOrpheus (uncensored Orpheus TTS, Llama-3.2-3B -> SNAC 24kHz). vllm-morpheus (GPU/3090) emits Orpheus audio tokens; morpheus-tts (CPU) SNAC-decodes them to WAV and exposes POST /tts (baddy voice + zero-shot cloning). Deployed + tested end-to-end (28/28 valid frames, valid WAV, reachable over WG). Hard-won config, all encoded in compose/README: - bf16 REQUIRED: --quantization fp8 destroys audio-token generation (0 valid SNAC frames even at greedy). Footprint ~7.9GB. - Image PINNED to v0.23.0: 'latest' ships Blackwell oink/aiter kernels that crash on Ampere import. - 3090 (not the comfy-contended A6000); --enforce-eager to fit the shared card. - RTF ~1.0 end-to-end (gen ~98 tok/s / RTF 0.84 + CPU decode + HTTP). INTERNAL RESEARCH ONLY (CC-BY-NC-4.0); do not expose externally.
24 lines
1.1 KiB
Bash
24 lines
1.1 KiB
Bash
# mOrpheus stack tunables (irv-ml1). Copy to .env on the host; edit values there.
|
|
|
|
# GPU pin (UUID). Default = the 3090 (the stabler audio-zoo GPU; A6000 is comfy's + spiky).
|
|
# irv-ml1: 3090 = GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72
|
|
# A6000 = GPU-9672f0d5-3caf-b15c-898a-cd9d653c654b
|
|
MORPHEUS_GPU_UUID=GPU-12c15b3c-0827-0bab-b0cd-31aaa37a8e72
|
|
|
|
# Weights (stage once; not in git). mOrpheus = LlamaForCausalLM 3.3B bf16 (6.6GB).
|
|
MORPHEUS_MODEL_DIR=/home/lkraven/morpheus/models/mOrpheus
|
|
SNAC_DIR=/home/lkraven/morpheus/models/snac_24khz
|
|
|
|
# Engine. bf16 REQUIRED — FP8 destroys audio-token generation on this model (0 valid SNAC
|
|
# frames even at greedy). bf16 weights ~6.6GB; --enforce-eager (in compose) skips CUDA-
|
|
# graph capture so it fits the shared 3090. util 0.32 (~7.7GB of the 24GB card); raise if
|
|
# you free the 3090, lower only if you can (bf16 weights alone are 6.6GB => >=~0.30 floor).
|
|
MORPHEUS_GPU_UTIL=0.32
|
|
MORPHEUS_MAX_LEN=4096
|
|
MORPHEUS_MAX_SEQS=8
|
|
|
|
# TTS endpoint (host port). Public /tts + /docs live here.
|
|
MORPHEUS_TTS_PORT=8299
|
|
MORPHEUS_DEFAULT_VOICE=baddy
|
|
MORPHEUS_VOICES=baddy
|