vh
569e1af9ca
Move the ~22-service flat "AI Systems" group off the Main tab into a new four-tab layout (Main / AI / Infrastructure / Toolchain). The AI tab sorts the inference fleet by function into seven groups: AI - Inference gen, char-rp, char-rp-reasoning, Granite summarizer AI - Eval & Retrieval Selene, Skywork Reward, Qwen3 rerank/embed, image-bench AI - Gateways & Chat LiteLLM, Asset Engine, Gateway Chat, Open WebUI, ... AI - Speech (TTS) Chatterbox Fast, Kokoro, mOrpheus AI - Audio Tools Parakeet ASR, YT Voice Clipper AI - Image & Media ComfyUI, Arbo AI - Dormant stopped rollback seats + retired auditions Relabel each stack's homepage.group so canonical stacks/ matches the live containers on ana-ml2, ana-docker, and irv-ml1. Dormant stacks were refreshed with `docker compose up --no-start` so they carry the new label while staying stopped (compose-start rollback preserved). settings.yaml drives tab/order/ columns; services.yaml and README updated to the new scheme.
mOrpheus — uncensored Orpheus TTS (irv-ml1)
Permanent serving stack for MrDragonFox/mOrpheus_3B-1Base_early_preview-v1-25000 — an
uncensored Orpheus TTS finetune (Llama-3.2-3B LLM → SNAC 24 kHz audio). Trained speaker
"baddy"; supports zero-shot voice cloning from a reference clip.
INTERNAL RESEARCH ONLY. License is CC-BY-NC-4.0 (non-commercial). Do not expose this endpoint externally or use it in any commercial-facing product.
Shape
Two containers (see compose.yaml):
| service | where | role |
|---|---|---|
vllm-morpheus |
GPU (3090), FP8 | serves the mOrpheus LLM; emits Orpheus audio tokens |
morpheus-tts |
CPU | SNAC-decodes tokens → 24 kHz WAV; the public /tts endpoint |
Real-time: ~165 tok/s single-stream on the 3090 (FP8) ⇒ RTF ≈ 0.50 (2× real-time), measured. A ~4 s clip generates in ~2 s. (Whole-clip decode in v1; chunked streaming for lower time-to-first-audio is a future enhancement.)
Endpoints (http://10.100.79.3:8299)
POST /tts→audio/wav. Body:{"text": "...", "voice": "baddy", "temperature": 0.6, "max_tokens": 1200, "repetition_penalty": 1.1}.- Zero-shot clone (ad-hoc): add
"reference_audio_b64": "<base64 WAV>"+"reference_text": "<its transcript>". Keeprepetition_penalty <= 1.1for cloning (higher penalizes the in-context reference audio tokens and breaks generation). - Staged clone voices: drop
<name>.wav+<name>.txt(its transcript) into the voices dir (/home/lkraven/morpheus/voices/); each is encoded to its reference block once at startup, sovoice: "<name>"zero-shot clones it (e.g.beatrice).GET /voiceslists them. max_tokensdefaults to 3500 (~42 s), auto-clamped so prompt + gen never exceeds the 4096 context (a cloning reference block is ~1,100 tokens).repetition_penalty1.1 is load-bearing — at 1.0 the model never emits end-of-speech and rambles to the cap.
- Zero-shot clone (ad-hoc): add
GET /voices,GET /health,GET /docs(OpenAPI UI).
Expressive tags (baddy is trained for these): <sigh> <gasp> <laugh> <chuckle> <pant> <groan> <moan> etc. Use real carrier sentences with sparse, sentence-boundary tags —
stacking many tags with little text sends this early checkpoint into a repeat-loop.
Deploy (irv-ml1, as lkraven — docker-group, no sudo)
# one-time: stage weights (from the audition dir or a fresh pull-hf-repo) + copy the stack
mkdir -p /home/lkraven/morpheus/models
mv /home/lkraven/orpheus-audition/models/mOrpheus /home/lkraven/morpheus/models/
mv /home/lkraven/orpheus-audition/models/snac_24khz /home/lkraven/morpheus/models/
# copy compose.yaml + tts/ to /home/lkraven/morpheus/, cp .env.example .env
cd /home/lkraven/morpheus && docker compose build && docker compose up -d
Gotchas
- Pin
vllm/vllm-openai:v0.23.0—latestships Blackwell-only kernels (oink/aiter) that crash on Ampere import. Do not bump tolateston this box. - GPU = 3090, not the A6000 — the A6000 is comfy's and spikes to ~41 GB without warning (OOM'd two launches). FP8's ~5 GB footprint coexists with the 3090 audio zoo.
- FP8 on Ampere is a VRAM save (upcast), no compute speedup — real-time comes from vLLM.
- Canonical copy lives here; deployed copy is
/home/lkraven/morpheus/on irv-ml1.