feat(morpheus): staged clone voices + max_tokens 3500 (context-clamped)
- max_tokens default 2400->3500 (~42s) in wrapper + gateway-chat client, with a _cap() clamp so prompt+gen never exceeds MAX_CTX (4096) — a cloning ref block is ~1100 tokens, so an unclamped 3500 would overflow context on the clone path. - Staged clone voices: /voices dir of <name>.wav + <name>.txt, each encoded to its Orpheus reference block at startup; voice="<name>" zero-shot clones it. Beatrice (a chatterbox reference) staged as the first normal-voice clone. GET /voices lists baddy + clones. - compose: mount voices dir + pass MORPHEUS_MAX_LEN to the wrapper (clamp must match engine). vLLM concurrency (measured, --max-num-seqs 8, 250-tok reqs): near-linear batching — 8 concurrent finish in the same ~2.8s as 1 (707 tok/s, 8.1x single, flat per-req latency). Chunked-sentence production can fan out for ~8x throughput; CPU SNAC decode is the scale bottleneck, not generation.
This commit is contained in:
@@ -160,7 +160,7 @@ async function streamSpeak(speech, tag){
|
||||
const url = $('ttsUrl').value.trim().replace(/\/+$/,'').replace(/\/tts$/,'') + '/tts/stream';
|
||||
const res = await fetch(url, {
|
||||
method:'POST', headers:{ 'Content-Type':'application/json' },
|
||||
body: JSON.stringify({ text: speech, voice: ($('ttsVoice').value.trim() || 'baddy'), max_tokens: 2400 })
|
||||
body: JSON.stringify({ text: speech, voice: ($('ttsVoice').value.trim() || 'baddy'), max_tokens: 3500 })
|
||||
});
|
||||
if (!res.ok || !res.body){ tag.textContent = ' 🔇' + res.status; return; }
|
||||
const reader = res.body.getReader();
|
||||
|
||||
Reference in New Issue
Block a user