feat(morpheus): staged clone voices + max_tokens 3500 (context-clamped)

- max_tokens default 2400->3500 (~42s) in wrapper + gateway-chat client, with a _cap()
  clamp so prompt+gen never exceeds MAX_CTX (4096) — a cloning ref block is ~1100 tokens,
  so an unclamped 3500 would overflow context on the clone path.
- Staged clone voices: /voices dir of <name>.wav + <name>.txt, each encoded to its Orpheus
  reference block at startup; voice="<name>" zero-shot clones it. Beatrice (a chatterbox
  reference) staged as the first normal-voice clone. GET /voices lists baddy + clones.
- compose: mount voices dir + pass MORPHEUS_MAX_LEN to the wrapper (clamp must match engine).

vLLM concurrency (measured, --max-num-seqs 8, 250-tok reqs): near-linear batching — 8
concurrent finish in the same ~2.8s as 1 (707 tok/s, 8.1x single, flat per-req latency).
Chunked-sentence production can fan out for ~8x throughput; CPU SNAC decode is the scale
bottleneck, not generation.
This commit is contained in:
vh
2026-07-09 01:35:14 -07:00
parent f363fe6c84
commit 0655a37bf6
5 changed files with 41 additions and 8 deletions
+3
View File
@@ -67,9 +67,12 @@ services:
- SNAC_DEVICE=cpu
- MORPHEUS_DEFAULT_VOICE=${MORPHEUS_DEFAULT_VOICE:-baddy}
- MORPHEUS_VOICES=${MORPHEUS_VOICES:-baddy}
- MORPHEUS_VOICES_DIR=/voices # <name>.wav + <name>.txt => voice="<name>" clones it
- MORPHEUS_MAX_LEN=${MORPHEUS_MAX_LEN:-4096} # must match engine --max-model-len (max_tokens clamp)
volumes:
- ${MORPHEUS_MODEL_DIR:-/home/lkraven/morpheus/models/mOrpheus}:/model:ro
- ${SNAC_DIR:-/home/lkraven/morpheus/models/snac_24khz}:/snac:ro
- ${MORPHEUS_VOICES_DIR:-/home/lkraven/morpheus/voices}:/voices:ro
ports:
- "${MORPHEUS_TTS_PORT:-8299}:8000"
healthcheck: