docs/asset-engine: kokoro v2 + new kokoro-captioned entry
Per a request from the asset_engine consumer (althing thread
01KRCF4W66X3N24B01FF2Y7V3D), and verified against the live kokoro
OpenAPI + exercised endpoints:
* kokoro: version 1 → 2; adds three fields surfaced by the upstream
schema but not previously declared:
- speed (slider 0.25–4.0, default 1.0)
- volume_multiplier (slider 0.5–2.0, default 1.0; UI-bounded
since upstream is unbounded — noted in description)
- lang_code (text, optional override of the voice-name-derived
language hint)
* kokoro-captioned: new service entry wrapping
/dev/captioned_speech. Same model + image as kokoro proper but
separate catalog entry because the response shape is structured
JSON (audio inline as base64 + word-level timestamps), not raw
audio bytes. Verified shape captured in reproducibility.notes
so future consumers don't have to re-discover it. response.type
= json (consumer renders custom: player + subtitle overlay).
* reproducibility_audit: row added for kokoro-captioned.
Deferred (separate from this commit):
- kokoro-blend-voice. /v1/audio/voices/combine returns 403 on the
default config (allow_local_voice_saving=False); even with the
flag flipped it writes to a temp dir, not /worktank/kokoro/user_voices.
The persistent blend mechanism in this fleet is
playbooks/blend-kokoro-voice.yaml. Ad-hoc blending already works
through /v1/audio/speech via the inline syntax voice="a(w)+b(w)";
consumer can surface that as a UI affordance without any
catalog change.
catalog_version stays at 1 (no field-type vocabulary changes).
JSON Schema regeneration produced byte-identical output.
This commit is contained in:
@@ -26,7 +26,7 @@ services:
|
|||||||
60+ built-in voices in 8 languages. Voice mixing via inline weights.
|
60+ built-in voices in 8 languages. Voice mixing via inline weights.
|
||||||
OpenAI-compatible /v1/audio/speech.
|
OpenAI-compatible /v1/audio/speech.
|
||||||
category: tts
|
category: tts
|
||||||
version: 1
|
version: 2
|
||||||
host: irv-ml1
|
host: irv-ml1
|
||||||
endpoint: http://10.100.79.3:8193/v1/audio/speech
|
endpoint: http://10.100.79.3:8193/v1/audio/speech
|
||||||
method: POST
|
method: POST
|
||||||
@@ -54,6 +54,35 @@ services:
|
|||||||
type: select
|
type: select
|
||||||
options: [wav, mp3, opus, flac, pcm]
|
options: [wav, mp3, opus, flac, pcm]
|
||||||
default: wav
|
default: wav
|
||||||
|
- name: speed
|
||||||
|
type: slider
|
||||||
|
label: Speed
|
||||||
|
required: false
|
||||||
|
min: 0.25
|
||||||
|
max: 4.0
|
||||||
|
step: 0.05
|
||||||
|
default: 1.0
|
||||||
|
description: Playback speed multiplier. 1.0 is natural.
|
||||||
|
- name: volume_multiplier
|
||||||
|
type: slider
|
||||||
|
label: Volume
|
||||||
|
required: false
|
||||||
|
min: 0.5
|
||||||
|
max: 2.0
|
||||||
|
step: 0.05
|
||||||
|
default: 1.0
|
||||||
|
description: >
|
||||||
|
Output gain. Useful for level-matching against music beds.
|
||||||
|
Upstream is unbounded; UI bounds (0.5–2.0) are a sensible default —
|
||||||
|
override the slider client-side if you need extremes.
|
||||||
|
- name: lang_code
|
||||||
|
type: text
|
||||||
|
label: Language code
|
||||||
|
required: false
|
||||||
|
description: >
|
||||||
|
Optional language hint (e.g. "a" for American English). When unset,
|
||||||
|
kokoro infers from the first letter of the voice name. Set when text
|
||||||
|
contains code-switching kokoro guesses wrong on.
|
||||||
- name: stream
|
- name: stream
|
||||||
type: bool
|
type: bool
|
||||||
default: false
|
default: false
|
||||||
@@ -75,6 +104,81 @@ services:
|
|||||||
notes: |
|
notes: |
|
||||||
Voice mixing: voice="name1(w1)+name2(w2)+..." normalizes weights.
|
Voice mixing: voice="name1(w1)+name2(w2)+..." normalizes weights.
|
||||||
Custom voices persist at /worktank/kokoro/user_voices (bind-mounted).
|
Custom voices persist at /worktank/kokoro/user_voices (bind-mounted).
|
||||||
|
v2 (2026-05-11): added speed, volume_multiplier, lang_code per
|
||||||
|
kokoro's OpenAPI surface.
|
||||||
|
|
||||||
|
- id: kokoro-captioned
|
||||||
|
name: Kokoro Captioned Speech
|
||||||
|
description: >
|
||||||
|
Kokoro TTS with word-level timestamps returned alongside the audio.
|
||||||
|
For subtitle generation and video sync. Same model as `kokoro`; this
|
||||||
|
is a separate catalog entry because the response shape is structured
|
||||||
|
JSON (audio inline + timestamps), not raw audio bytes.
|
||||||
|
category: tts
|
||||||
|
version: 1
|
||||||
|
host: irv-ml1
|
||||||
|
endpoint: http://10.100.79.3:8193/dev/captioned_speech
|
||||||
|
method: POST
|
||||||
|
content_type: application/json
|
||||||
|
model:
|
||||||
|
id: hexgrad/Kokoro-82M
|
||||||
|
revision: null
|
||||||
|
image: ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master
|
||||||
|
fields:
|
||||||
|
- name: input
|
||||||
|
type: textarea
|
||||||
|
label: Text
|
||||||
|
required: true
|
||||||
|
max_length: 5000
|
||||||
|
- name: voice
|
||||||
|
type: select
|
||||||
|
label: Voice
|
||||||
|
source_url: http://10.100.79.3:8193/v1/audio/voices
|
||||||
|
source_jsonpath: $.voices[*]
|
||||||
|
default: af_bella
|
||||||
|
- name: response_format
|
||||||
|
type: select
|
||||||
|
options: [wav, mp3, opus, flac, pcm]
|
||||||
|
default: wav
|
||||||
|
- name: speed
|
||||||
|
type: slider
|
||||||
|
label: Speed
|
||||||
|
required: false
|
||||||
|
min: 0.25
|
||||||
|
max: 4.0
|
||||||
|
step: 0.05
|
||||||
|
default: 1.0
|
||||||
|
- name: lang_code
|
||||||
|
type: text
|
||||||
|
label: Language code
|
||||||
|
required: false
|
||||||
|
response:
|
||||||
|
type: json
|
||||||
|
mime: application/json
|
||||||
|
reproducibility:
|
||||||
|
seedable: false
|
||||||
|
deterministic: true
|
||||||
|
notes: >
|
||||||
|
Same determinism story as kokoro proper. Response shape (verified
|
||||||
|
2026-05-11 against live API):
|
||||||
|
{
|
||||||
|
"audio": "<base64-encoded bytes in response_format>",
|
||||||
|
"audio_format": "audio/wav" (or matching response_format),
|
||||||
|
"timestamps": [{"word": str, "start_time": float, "end_time": float}, ...]
|
||||||
|
}
|
||||||
|
Consumer must base64-decode `audio` to play; `timestamps` drives
|
||||||
|
subtitle/karaoke UI. Audio_format string in the payload is
|
||||||
|
authoritative for the decoded bytes.
|
||||||
|
estimated_latency:
|
||||||
|
cold_start_s: 2
|
||||||
|
warm_per_unit: "~same as kokoro proper, plus minor overhead for timestamp emission"
|
||||||
|
license: Apache-2.0
|
||||||
|
notes: |
|
||||||
|
`return_timestamps` and `stream` upstream params are deliberately
|
||||||
|
omitted from the catalog: timestamps must be on for this endpoint
|
||||||
|
to be meaningful, and streaming + JSON-with-base64 don't compose.
|
||||||
|
`download_format` / `return_download_link` skipped — same reasoning
|
||||||
|
as kokoro proper.
|
||||||
|
|
||||||
- id: chatterbox
|
- id: chatterbox
|
||||||
name: Chatterbox Turbo TTS
|
name: Chatterbox Turbo TTS
|
||||||
@@ -820,6 +924,11 @@ reproducibility_audit:
|
|||||||
model_deterministic: true
|
model_deterministic: true
|
||||||
image_tag_mutable: true
|
image_tag_mutable: true
|
||||||
notes: ":v0.2.4-master is mutable; pin to digest for v3."
|
notes: ":v0.2.4-master is mutable; pin to digest for v3."
|
||||||
|
- service: kokoro-captioned
|
||||||
|
seedable: false
|
||||||
|
model_deterministic: true
|
||||||
|
image_tag_mutable: true
|
||||||
|
notes: "Same image as kokoro proper; same mutability story. Response carries timestamps."
|
||||||
- service: chatterbox
|
- service: chatterbox
|
||||||
seedable: false
|
seedable: false
|
||||||
model_deterministic: true
|
model_deterministic: true
|
||||||
|
|||||||
Reference in New Issue
Block a user