catalog-contract: add response-decomposition fields (audio_field, timestamps_field, audio_format_field)

asset_engine consumer needed to render kokoro-captioned, whose wire
shape is a JSON envelope carrying base64-encoded audio plus a
structured timestamps array. Modeling it as response.type=json
would force either a per-service-id renderer (forbidden by
brief §1.7) or extending the closed response-type vocabulary
(forbidden by brief §2.2 without a coordinated bump).

Resolution (per althing thread 01KRCF4W66X3): keep response.type
closed at the existing six values and decompose at the response
*field* level instead — the same flexibility seam already used by
mime / mime_from_field / output_field. Adds three optional keys:

  - audio_field: JSON key holding base64-encoded audio bytes
  - audio_format_field: JSON key holding the decoded audio MIME
  - timestamps_field: JSON key holding a structured timestamps array
    (independent of type, declared by any service emitting time-
    aligned markers)

Validators in CatalogResponse enforce sane combinations:
  - audio_field requires response.type=audio
  - audio_field forbids mime_from_field
  - audio_format_field requires audio_field

This is additive and backward-compatible — no catalog_version bump,
existing services parse unchanged. CATALOG-CONTRACT.md updated with
the new rows in the response-field table and a versioning-policy
row codifying that adding optional keys to response: doesn't bump.

kokoro-captioned re-shaped to use the new schema:
  response:
    type: audio
    audio_field: audio
    audio_format_field: audio_format
    timestamps_field: timestamps
And marked status: experimental until the asset_engine consumer's
audio-with-timestamps renderer ships.

JSON Schema regenerated to reflect the new Pydantic shape.

Pydantic-model side of this change lives in the asset_engine repo
at src/asset_engine/catalog.py — committed there separately.
This commit is contained in:
vh
2026-05-11 14:55:15 -07:00
parent 44c565ac77
commit d3faeb0314
3 changed files with 70 additions and 17 deletions
+37
View File
@@ -332,6 +332,7 @@
},
"CatalogResponse": {
"additionalProperties": false,
"description": "How to interpret the inference response.\n\nThe `type` is the *renderer dispatch* \u2014 what kind of asset the user\nultimately sees (audio player, text block, etc). The other fields\ndescribe how to *extract* that asset from the wire shape:\n\n - Raw-bytes wire (e.g. /v1/audio/speech returns raw audio):\n type: audio\n mime: audio/wav (or mime_from_field for dynamic)\n\n - JSON-envelope wire with base64-encoded asset (e.g. captioned\n speech returns {audio: <b64>, audio_format: ..., timestamps: ...}):\n type: audio\n audio_field: audio # JSON key with the base64 bytes\n audio_format_field: audio_format # JSON key with the MIME\n timestamps_field: timestamps # optional structured aside\n\n - JSON-envelope wire with text payload (e.g. ASR transcript):\n type: text\n output_field: text # JSON key with the text body\n timestamps_field: segments # optional, for ASR-with-timestamps\n\nRules:\n - When audio_field is set, the renderer expects a JSON envelope and\n will base64-decode that key. The wire MIME is application/json\n regardless of `mime`; `mime_from_field` is incompatible.\n - timestamps_field is independent of type \u2014 any service that\n emits structured timestamps alongside its primary output can\n declare it; the renderer extends accordingly.",
"properties": {
"type": {
"enum": [
@@ -380,6 +381,42 @@
],
"default": null,
"title": "Output Field"
},
"audio_field": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Audio Field"
},
"audio_format_field": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Audio Format Field"
},
"timestamps_field": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Timestamps Field"
}
},
"required": [