docs/asset-engine: kokoro v2 + new kokoro-captioned entry
Per a request from the asset_engine consumer (althing thread
01KRCF4W66X3N24B01FF2Y7V3D), and verified against the live kokoro
OpenAPI + exercised endpoints:
* kokoro: version 1 → 2; adds three fields surfaced by the upstream
schema but not previously declared:
- speed (slider 0.25–4.0, default 1.0)
- volume_multiplier (slider 0.5–2.0, default 1.0; UI-bounded
since upstream is unbounded — noted in description)
- lang_code (text, optional override of the voice-name-derived
language hint)
* kokoro-captioned: new service entry wrapping
/dev/captioned_speech. Same model + image as kokoro proper but
separate catalog entry because the response shape is structured
JSON (audio inline as base64 + word-level timestamps), not raw
audio bytes. Verified shape captured in reproducibility.notes
so future consumers don't have to re-discover it. response.type
= json (consumer renders custom: player + subtitle overlay).
* reproducibility_audit: row added for kokoro-captioned.
Deferred (separate from this commit):
- kokoro-blend-voice. /v1/audio/voices/combine returns 403 on the
default config (allow_local_voice_saving=False); even with the
flag flipped it writes to a temp dir, not /worktank/kokoro/user_voices.
The persistent blend mechanism in this fleet is
playbooks/blend-kokoro-voice.yaml. Ad-hoc blending already works
through /v1/audio/speech via the inline syntax voice="a(w)+b(w)";
consumer can surface that as a UI affordance without any
catalog change.
catalog_version stays at 1 (no field-type vocabulary changes).
JSON Schema regeneration produced byte-identical output.
This commit is contained in:
@@ -26,7 +26,7 @@ services:
|
||||
60+ built-in voices in 8 languages. Voice mixing via inline weights.
|
||||
OpenAI-compatible /v1/audio/speech.
|
||||
category: tts
|
||||
version: 1
|
||||
version: 2
|
||||
host: irv-ml1
|
||||
endpoint: http://10.100.79.3:8193/v1/audio/speech
|
||||
method: POST
|
||||
@@ -54,6 +54,35 @@ services:
|
||||
type: select
|
||||
options: [wav, mp3, opus, flac, pcm]
|
||||
default: wav
|
||||
- name: speed
|
||||
type: slider
|
||||
label: Speed
|
||||
required: false
|
||||
min: 0.25
|
||||
max: 4.0
|
||||
step: 0.05
|
||||
default: 1.0
|
||||
description: Playback speed multiplier. 1.0 is natural.
|
||||
- name: volume_multiplier
|
||||
type: slider
|
||||
label: Volume
|
||||
required: false
|
||||
min: 0.5
|
||||
max: 2.0
|
||||
step: 0.05
|
||||
default: 1.0
|
||||
description: >
|
||||
Output gain. Useful for level-matching against music beds.
|
||||
Upstream is unbounded; UI bounds (0.5–2.0) are a sensible default —
|
||||
override the slider client-side if you need extremes.
|
||||
- name: lang_code
|
||||
type: text
|
||||
label: Language code
|
||||
required: false
|
||||
description: >
|
||||
Optional language hint (e.g. "a" for American English). When unset,
|
||||
kokoro infers from the first letter of the voice name. Set when text
|
||||
contains code-switching kokoro guesses wrong on.
|
||||
- name: stream
|
||||
type: bool
|
||||
default: false
|
||||
@@ -75,6 +104,81 @@ services:
|
||||
notes: |
|
||||
Voice mixing: voice="name1(w1)+name2(w2)+..." normalizes weights.
|
||||
Custom voices persist at /worktank/kokoro/user_voices (bind-mounted).
|
||||
v2 (2026-05-11): added speed, volume_multiplier, lang_code per
|
||||
kokoro's OpenAPI surface.
|
||||
|
||||
- id: kokoro-captioned
|
||||
name: Kokoro Captioned Speech
|
||||
description: >
|
||||
Kokoro TTS with word-level timestamps returned alongside the audio.
|
||||
For subtitle generation and video sync. Same model as `kokoro`; this
|
||||
is a separate catalog entry because the response shape is structured
|
||||
JSON (audio inline + timestamps), not raw audio bytes.
|
||||
category: tts
|
||||
version: 1
|
||||
host: irv-ml1
|
||||
endpoint: http://10.100.79.3:8193/dev/captioned_speech
|
||||
method: POST
|
||||
content_type: application/json
|
||||
model:
|
||||
id: hexgrad/Kokoro-82M
|
||||
revision: null
|
||||
image: ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master
|
||||
fields:
|
||||
- name: input
|
||||
type: textarea
|
||||
label: Text
|
||||
required: true
|
||||
max_length: 5000
|
||||
- name: voice
|
||||
type: select
|
||||
label: Voice
|
||||
source_url: http://10.100.79.3:8193/v1/audio/voices
|
||||
source_jsonpath: $.voices[*]
|
||||
default: af_bella
|
||||
- name: response_format
|
||||
type: select
|
||||
options: [wav, mp3, opus, flac, pcm]
|
||||
default: wav
|
||||
- name: speed
|
||||
type: slider
|
||||
label: Speed
|
||||
required: false
|
||||
min: 0.25
|
||||
max: 4.0
|
||||
step: 0.05
|
||||
default: 1.0
|
||||
- name: lang_code
|
||||
type: text
|
||||
label: Language code
|
||||
required: false
|
||||
response:
|
||||
type: json
|
||||
mime: application/json
|
||||
reproducibility:
|
||||
seedable: false
|
||||
deterministic: true
|
||||
notes: >
|
||||
Same determinism story as kokoro proper. Response shape (verified
|
||||
2026-05-11 against live API):
|
||||
{
|
||||
"audio": "<base64-encoded bytes in response_format>",
|
||||
"audio_format": "audio/wav" (or matching response_format),
|
||||
"timestamps": [{"word": str, "start_time": float, "end_time": float}, ...]
|
||||
}
|
||||
Consumer must base64-decode `audio` to play; `timestamps` drives
|
||||
subtitle/karaoke UI. Audio_format string in the payload is
|
||||
authoritative for the decoded bytes.
|
||||
estimated_latency:
|
||||
cold_start_s: 2
|
||||
warm_per_unit: "~same as kokoro proper, plus minor overhead for timestamp emission"
|
||||
license: Apache-2.0
|
||||
notes: |
|
||||
`return_timestamps` and `stream` upstream params are deliberately
|
||||
omitted from the catalog: timestamps must be on for this endpoint
|
||||
to be meaningful, and streaming + JSON-with-base64 don't compose.
|
||||
`download_format` / `return_download_link` skipped — same reasoning
|
||||
as kokoro proper.
|
||||
|
||||
- id: chatterbox
|
||||
name: Chatterbox Turbo TTS
|
||||
@@ -820,6 +924,11 @@ reproducibility_audit:
|
||||
model_deterministic: true
|
||||
image_tag_mutable: true
|
||||
notes: ":v0.2.4-master is mutable; pin to digest for v3."
|
||||
- service: kokoro-captioned
|
||||
seedable: false
|
||||
model_deterministic: true
|
||||
image_tag_mutable: true
|
||||
notes: "Same image as kokoro proper; same mutability story. Response carries timestamps."
|
||||
- service: chatterbox
|
||||
seedable: false
|
||||
model_deterministic: true
|
||||
|
||||
Reference in New Issue
Block a user