docs/asset-engine: kokoro v2 + new kokoro-captioned entry

Per a request from the asset_engine consumer (althing thread
01KRCF4W66X3N24B01FF2Y7V3D), and verified against the live kokoro
OpenAPI + exercised endpoints:

* kokoro: version 1 → 2; adds three fields surfaced by the upstream
  schema but not previously declared:
    - speed (slider 0.25–4.0, default 1.0)
    - volume_multiplier (slider 0.5–2.0, default 1.0; UI-bounded
      since upstream is unbounded — noted in description)
    - lang_code (text, optional override of the voice-name-derived
      language hint)

* kokoro-captioned: new service entry wrapping
  /dev/captioned_speech. Same model + image as kokoro proper but
  separate catalog entry because the response shape is structured
  JSON (audio inline as base64 + word-level timestamps), not raw
  audio bytes. Verified shape captured in reproducibility.notes
  so future consumers don't have to re-discover it. response.type
  = json (consumer renders custom: player + subtitle overlay).

* reproducibility_audit: row added for kokoro-captioned.

Deferred (separate from this commit):
- kokoro-blend-voice. /v1/audio/voices/combine returns 403 on the
  default config (allow_local_voice_saving=False); even with the
  flag flipped it writes to a temp dir, not /worktank/kokoro/user_voices.
  The persistent blend mechanism in this fleet is
  playbooks/blend-kokoro-voice.yaml. Ad-hoc blending already works
  through /v1/audio/speech via the inline syntax voice="a(w)+b(w)";
  consumer can surface that as a UI affordance without any
  catalog change.

catalog_version stays at 1 (no field-type vocabulary changes).
JSON Schema regeneration produced byte-identical output.
This commit is contained in:
2026-05-11 14:38:05 -07:00
parent 0157066d6e
commit 44c565ac77
+110 -1
View File
@@ -26,7 +26,7 @@ services:
60+ built-in voices in 8 languages. Voice mixing via inline weights.
OpenAI-compatible /v1/audio/speech.
category: tts
version: 1
version: 2
host: irv-ml1
endpoint: http://10.100.79.3:8193/v1/audio/speech
method: POST
@@ -54,6 +54,35 @@ services:
type: select
options: [wav, mp3, opus, flac, pcm]
default: wav
- name: speed
type: slider
label: Speed
required: false
min: 0.25
max: 4.0
step: 0.05
default: 1.0
description: Playback speed multiplier. 1.0 is natural.
- name: volume_multiplier
type: slider
label: Volume
required: false
min: 0.5
max: 2.0
step: 0.05
default: 1.0
description: >
Output gain. Useful for level-matching against music beds.
Upstream is unbounded; UI bounds (0.52.0) are a sensible default —
override the slider client-side if you need extremes.
- name: lang_code
type: text
label: Language code
required: false
description: >
Optional language hint (e.g. "a" for American English). When unset,
kokoro infers from the first letter of the voice name. Set when text
contains code-switching kokoro guesses wrong on.
- name: stream
type: bool
default: false
@@ -75,6 +104,81 @@ services:
notes: |
Voice mixing: voice="name1(w1)+name2(w2)+..." normalizes weights.
Custom voices persist at /worktank/kokoro/user_voices (bind-mounted).
v2 (2026-05-11): added speed, volume_multiplier, lang_code per
kokoro's OpenAPI surface.
- id: kokoro-captioned
name: Kokoro Captioned Speech
description: >
Kokoro TTS with word-level timestamps returned alongside the audio.
For subtitle generation and video sync. Same model as `kokoro`; this
is a separate catalog entry because the response shape is structured
JSON (audio inline + timestamps), not raw audio bytes.
category: tts
version: 1
host: irv-ml1
endpoint: http://10.100.79.3:8193/dev/captioned_speech
method: POST
content_type: application/json
model:
id: hexgrad/Kokoro-82M
revision: null
image: ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.4-master
fields:
- name: input
type: textarea
label: Text
required: true
max_length: 5000
- name: voice
type: select
label: Voice
source_url: http://10.100.79.3:8193/v1/audio/voices
source_jsonpath: $.voices[*]
default: af_bella
- name: response_format
type: select
options: [wav, mp3, opus, flac, pcm]
default: wav
- name: speed
type: slider
label: Speed
required: false
min: 0.25
max: 4.0
step: 0.05
default: 1.0
- name: lang_code
type: text
label: Language code
required: false
response:
type: json
mime: application/json
reproducibility:
seedable: false
deterministic: true
notes: >
Same determinism story as kokoro proper. Response shape (verified
2026-05-11 against live API):
{
"audio": "<base64-encoded bytes in response_format>",
"audio_format": "audio/wav" (or matching response_format),
"timestamps": [{"word": str, "start_time": float, "end_time": float}, ...]
}
Consumer must base64-decode `audio` to play; `timestamps` drives
subtitle/karaoke UI. Audio_format string in the payload is
authoritative for the decoded bytes.
estimated_latency:
cold_start_s: 2
warm_per_unit: "~same as kokoro proper, plus minor overhead for timestamp emission"
license: Apache-2.0
notes: |
`return_timestamps` and `stream` upstream params are deliberately
omitted from the catalog: timestamps must be on for this endpoint
to be meaningful, and streaming + JSON-with-base64 don't compose.
`download_format` / `return_download_link` skipped — same reasoning
as kokoro proper.
- id: chatterbox
name: Chatterbox Turbo TTS
@@ -820,6 +924,11 @@ reproducibility_audit:
model_deterministic: true
image_tag_mutable: true
notes: ":v0.2.4-master is mutable; pin to digest for v3."
- service: kokoro-captioned
seedable: false
model_deterministic: true
image_tag_mutable: true
notes: "Same image as kokoro proper; same mutability story. Response carries timestamps."
- service: chatterbox
seedable: false
model_deterministic: true