docs/asset-engine: promote services.yaml to first-class contract

Adds the supporting infra around the service catalog now that it
has external consumers (the asset_engine UI being the first; CLIs,
monitoring, other services may follow):

- CATALOG-CONTRACT.md: the consumer-facing contract. Defines
  versioning policy (catalog_version vs per-service version),
  closed field-type and response-type vocabularies, recommended
  vendor+drift-check sync workflow, known-consumers list, service
  authoring notes.
- services.schema.json: JSON Schema (draft 2020-12) for the
  catalog. Generated from the Pydantic model in
  ~/development/asset_engine/src/asset_engine/catalog.py via
  `uv run scripts/dump_schema.py --publish`. Lets non-Python
  consumers validate against the same shape.
- services.yaml: adds catalog_version: 1 at the root and reframes
  the file's header to call out its first-class-contract status.
  Quotes a vibevoice label that contained an unescaped colon
  (caught by the asset_engine's strict YAML parser on first sync).
This commit is contained in:
vh
2026-05-11 08:57:35 -07:00
parent 8d8d45b7ca
commit 0157066d6e
3 changed files with 789 additions and 10 deletions
+159
View File
@@ -0,0 +1,159 @@
# services.yaml — contract for downstream consumers
`services.yaml` in this directory is the **canonical, versioned**
description of the inference services this fleet exposes. It is a
first-class artifact: external tools (the asset-engine UI is the
first; CLIs, monitoring dashboards, and other internal services may
follow) consume it programmatically, and changes to its shape or
contents have downstream effect.
This document is the consumer contract.
## Where the data lives
- **Canonical source**: `docs/asset-engine/services.yaml` in the
`eshpfi-management` repo (this file's directory).
- **Schema**: `docs/asset-engine/services.schema.json` (JSON Schema
draft 2020-12). Generated from the Pydantic model in
`~/development/asset_engine/src/asset_engine/catalog.py` —
regenerate via `cd ~/development/asset_engine && uv run scripts/dump_schema.py`
whenever the model changes.
- **Change history**: git log on this directory.
## Data model — high level
```yaml
catalog_version: 1 # bump on SCHEMA changes
services: [Service] # the entries; closed list
reproducibility_audit: [Audit] # one entry per service
```
### Service entry
| field | type | required | meaning |
|-------------------|-------------------------------------------|----------|---------|
| `id` | string | yes | stable URL-safe identifier; never reused |
| `name` | string | yes | display name |
| `description` | string | no | one-paragraph summary |
| `category` | enum: tts, asr, sfx, music, image | yes | groups in pickers |
| `version` | int | yes | per-service schema version; bump on field changes |
| `status` | enum: ready, catalog-deferred, experimental | no | default: ready |
| `host` | string (host alias) | yes | which fleet host serves this |
| `endpoint` | URL | yes | absolute URL of the inference endpoint |
| `method` | string | yes | HTTP method (POST common) |
| `content_type` | MIME type | yes | request body type |
| `model.id` | string | yes | upstream model id (HF or other) |
| `model.revision` | string \| null | no | SHA when known |
| `model.image` | string | yes | container image ref this is hosted from |
| `fields` | list[Field] | no | request parameters; empty for catalog-deferred |
| `response.type` | enum: audio, image, video, text, json, file | yes | response renderer hint |
| `response.mime` | string | no | static response MIME |
| `response.mime_from_field` | string | no | name of a field whose value determines the MIME |
| `response.output_field` | string | no | for JSON responses, the key holding the asset |
| `reproducibility.seedable` | bool | yes | does the endpoint accept a seed? |
| `reproducibility.deterministic` | bool | yes | same params → same bytes? |
| `reproducibility.notes` | string | no | gotchas |
| `estimated_latency.cold_start_s` | int \| float | no | informational only |
| `estimated_latency.warm_per_unit` | string | no | informational only |
| `license` | string | yes | upstream license |
| `license_warning` | string | no | non-empty → consumer MUST surface a warning before use |
| `notes` | string | no | freeform |
### Field entry
The field-type vocabulary is **closed**. Adding a new type requires:
(a) updating the Pydantic model in `asset_engine/catalog.py`,
(b) bumping `catalog_version`, (c) coordinating with all consumers.
| `type` | meaning |
|------------|------------------------------------------------------|
| `text` | single-line string |
| `textarea` | multi-line string |
| `number` | int or float, no bounded range |
| `slider` | bounded numeric — requires `min`, `max`, optional `step` |
| `select` | enum — `options:` list OR remote `source_url:` + `source_jsonpath:` |
| `bool` | true/false |
| `file` | file upload — requires `accepted_types:` list of MIME |
| `json` | freeform JSON value (arrays of floats, ref objects, etc) |
Optional/required: a field is required UNLESS it has either
`required: false` or `optional: true`. (Both ways are accepted; new
entries should prefer `required: false`.)
### Response renderer vocabulary
Closed set: `audio`, `image`, `video`, `text`, `json`, `file`.
Same change-management as field types.
## Versioning policy
| Change | Bump |
|----------------------------------------------|-----------------------|
| Add a new service | nothing |
| Add a non-required field to an existing service | service `version:` |
| Change a field's type, range, or default | service `version:` |
| Remove a service | service `version:` (sentinel: removed=true), then drop in next catalog_version bump |
| Add a new entry to the field-type vocabulary | `catalog_version:` |
| Add a new entry to the response-type vocabulary | `catalog_version:` |
| Drop a field-type or response-type | `catalog_version:` (BREAKING) |
| Pure prose edit (description, notes) | nothing |
Consumers should pin their tested `catalog_version`. When this repo
bumps it, downstreams should re-test before consuming the new shape.
## Sync workflow for downstream consumers
The recommended pattern is **vendor + drift-check**, not
fetch-at-runtime. Reasons: deployments stay reproducible; airgapped
or network-flaky deploys still work; schema changes surface in PRs
instead of breaking production silently.
A consumer should:
1. Vendor a copy of `services.yaml` in their repo (e.g. at
`data/services.yaml`).
2. Record the source SHA they synced from in a `services.yaml.lock`
alongside (path, source SHA, sync timestamp).
3. Provide a `sync_catalog` script that pulls the latest from the
canonical path and updates both files.
4. Validate the vendored file against `services.schema.json` on
load — refuse to start if validation fails.
5. CI: re-validate on every commit.
The asset-engine repo at `~/development/asset_engine/` is the
reference implementation of this pattern; copy the script shape from
`scripts/sync_catalog.py` over there.
## Known consumers
| Consumer | Vendored at | Pin policy |
|---------------------------------------------|--------------------------------------|-------------------|
| `asset_engine` (FastAPI/HTMX UI; ~/development/asset_engine/) | `data/services.yaml` + `data/services.yaml.lock` | catalog_version=1 |
When you add a consumer, add a row here in the same PR that lands
the consumer. This list is what we audit when planning a
`catalog_version` bump.
## Service authoring notes
When **adding** a service to the catalog:
- The Pydantic model in code is the ground truth for `fields:`. Read
the actual server code, not the README; READMEs rot. The
reproducibility audit at the bottom of `services.yaml` MUST get a
matching entry.
- If a service can't produce reproducible output (no seed AND
non-deterministic model), that's a **bug in the service** — fix it
there before adding to the catalog. The asset-engine relies on
reproducibility to power "regenerate" / "fork" affordances.
- If the service has a non-permissive license, populate
`license_warning:` with the user-facing language. The UI is
required to surface it before a generation; missing it is a
contract violation.
- Image refs SHOULD be digest-pinned (`@sha256:...`) for true
reproducibility. Tag-pinned (`:v1.2.3`) is acceptable but
vulnerable to upstream re-pushes.
When **removing** a service: see the versioning policy table.
Removal is a two-step process; surprise removal breaks consumers'
asset libraries (regenerate-from-history breaks).
+613
View File
@@ -0,0 +1,613 @@
{
"$defs": {
"CatalogAuditEntry": {
"additionalProperties": false,
"properties": {
"service": {
"title": "Service",
"type": "string"
},
"seedable": {
"title": "Seedable",
"type": "boolean"
},
"model_deterministic": {
"title": "Model Deterministic",
"type": "boolean"
},
"image_tag_mutable": {
"title": "Image Tag Mutable",
"type": "boolean"
},
"notes": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Notes"
}
},
"required": [
"service",
"seedable",
"model_deterministic",
"image_tag_mutable"
],
"title": "CatalogAuditEntry",
"type": "object"
},
"CatalogField": {
"additionalProperties": false,
"description": "One request parameter, rendered as one form control.",
"properties": {
"name": {
"title": "Name",
"type": "string"
},
"type": {
"enum": [
"text",
"textarea",
"number",
"slider",
"select",
"bool",
"file",
"json"
],
"title": "Type",
"type": "string"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
},
"required": {
"default": true,
"title": "Required",
"type": "boolean"
},
"optional": {
"default": false,
"title": "Optional",
"type": "boolean"
},
"default": {
"default": null,
"title": "Default"
},
"min": {
"anyOf": [
{
"type": "integer"
},
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Min"
},
"max": {
"anyOf": [
{
"type": "integer"
},
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Max"
},
"step": {
"anyOf": [
{
"type": "integer"
},
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Step"
},
"options": {
"anyOf": [
{
"items": {
"anyOf": [
{
"type": "string"
},
{
"$ref": "#/$defs/FieldOption"
}
]
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Options"
},
"source_url": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Source Url"
},
"source_jsonpath": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Source Jsonpath"
},
"description": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Description"
},
"max_length": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Length"
},
"accepted_types": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Accepted Types"
}
},
"required": [
"name",
"type"
],
"title": "CatalogField",
"type": "object"
},
"CatalogLatency": {
"additionalProperties": false,
"properties": {
"cold_start_s": {
"anyOf": [
{
"type": "integer"
},
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Cold Start S"
},
"warm_per_unit": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Warm Per Unit"
}
},
"title": "CatalogLatency",
"type": "object"
},
"CatalogModel": {
"additionalProperties": false,
"properties": {
"id": {
"title": "Id",
"type": "string"
},
"revision": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Revision"
},
"image": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Image"
}
},
"required": [
"id"
],
"title": "CatalogModel",
"type": "object"
},
"CatalogReproducibility": {
"additionalProperties": false,
"properties": {
"seedable": {
"title": "Seedable",
"type": "boolean"
},
"deterministic": {
"title": "Deterministic",
"type": "boolean"
},
"notes": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Notes"
}
},
"required": [
"seedable",
"deterministic"
],
"title": "CatalogReproducibility",
"type": "object"
},
"CatalogResponse": {
"additionalProperties": false,
"properties": {
"type": {
"enum": [
"audio",
"image",
"video",
"text",
"json",
"file"
],
"title": "Type",
"type": "string"
},
"mime": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Mime"
},
"mime_from_field": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Mime From Field"
},
"output_field": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Output Field"
}
},
"required": [
"type"
],
"title": "CatalogResponse",
"type": "object"
},
"CatalogService": {
"additionalProperties": false,
"properties": {
"id": {
"title": "Id",
"type": "string"
},
"name": {
"title": "Name",
"type": "string"
},
"description": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Description"
},
"category": {
"enum": [
"tts",
"asr",
"sfx",
"music",
"image",
"multimodal"
],
"title": "Category",
"type": "string"
},
"version": {
"title": "Version",
"type": "integer"
},
"status": {
"default": "ready",
"enum": [
"ready",
"catalog-deferred",
"experimental"
],
"title": "Status",
"type": "string"
},
"host": {
"title": "Host",
"type": "string"
},
"endpoint": {
"title": "Endpoint",
"type": "string"
},
"method": {
"default": "POST",
"title": "Method",
"type": "string"
},
"content_type": {
"default": "application/json",
"title": "Content Type",
"type": "string"
},
"model": {
"$ref": "#/$defs/CatalogModel"
},
"fields": {
"items": {
"$ref": "#/$defs/CatalogField"
},
"title": "Fields",
"type": "array"
},
"response": {
"$ref": "#/$defs/CatalogResponse"
},
"reproducibility": {
"$ref": "#/$defs/CatalogReproducibility"
},
"estimated_latency": {
"anyOf": [
{
"$ref": "#/$defs/CatalogLatency"
},
{
"type": "null"
}
],
"default": null
},
"license": {
"title": "License",
"type": "string"
},
"license_warning": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "License Warning"
},
"notes": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Notes"
}
},
"required": [
"id",
"name",
"category",
"version",
"host",
"endpoint",
"model",
"response",
"reproducibility",
"license"
],
"title": "CatalogService",
"type": "object"
},
"FieldOption": {
"additionalProperties": false,
"description": "A select option. May be just the value (when label = value) or {value, label}.",
"properties": {
"value": {
"anyOf": [
{
"type": "string"
},
{
"type": "integer"
},
{
"type": "number"
},
{
"type": "boolean"
}
],
"title": "Value"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
}
},
"required": [
"value"
],
"title": "FieldOption",
"type": "object"
}
},
"additionalProperties": false,
"description": "Schema for the canonical inference-services catalog. The Python Pydantic model in asset_engine/catalog.py is the source of truth; this JSON Schema is a derived view for non-Python consumers.",
"properties": {
"catalog_version": {
"default": 1,
"title": "Catalog Version",
"type": "integer"
},
"generated_at": {
"anyOf": [
{
"format": "date-time",
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Generated At"
},
"services": {
"items": {
"$ref": "#/$defs/CatalogService"
},
"title": "Services",
"type": "array"
},
"reproducibility_audit": {
"items": {
"$ref": "#/$defs/CatalogAuditEntry"
},
"title": "Reproducibility Audit",
"type": "array"
}
},
"required": [
"services"
],
"title": "Asset Engine \u2014 services.yaml catalog",
"type": "object",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://eshpfi.local/schemas/services.schema.json"
}
+17 -10
View File
@@ -1,15 +1,22 @@
# services.yaml — inference services catalog on irv-ml1
# services.yaml — canonical catalog of inference services on irv-ml1
#
# Drives the forthcoming asset-generation UI: each entry produces one
# form (auto-generated from `fields:`) and one response renderer
# (dispatched on `response.type`).
# THIS IS A FIRST-CLASS CONTRACT. External consumers depend on the
# shape and contents of this file. See CATALOG-CONTRACT.md alongside
# for: schema reference, change-management policy, sync workflow for
# downstream consumers, and known consumers list.
#
# Generated by an Explore-agent pass over stacks/<service>/{README.md,
# compose.yaml,.env.example,server.py|infer-api.py}. Pydantic models
# in code are the ground truth where they disagree with READMEs.
# Drives form generators that auto-render UIs against the inference
# services. Each entry produces one form (from `fields:`) and one
# response renderer (dispatched on `response.type`). The field-type
# vocabulary and response-type vocabulary are closed sets — see the
# JSON Schema at services.schema.json or the contract doc.
#
# Schema is v1; bump per-service `version:` when fields change so old
# Asset rows can still be interpreted.
# `catalog_version`: bump when the SCHEMA changes (a field type added,
# a required field removed, etc). Per-service `version:` bumps when a
# specific service's parameter shape changes. Both let downstream
# consumers detect drift.
catalog_version: 1
services:
- id: kokoro
@@ -438,7 +445,7 @@ services:
default: vibevoice
- name: input
type: textarea
label: Text (or Speaker N: ... script)
label: "Text (or Speaker N: ... script)"
required: true
description: >
Single-speaker: plain text. Multi-speaker: "Speaker 0: ...\nSpeaker 1: ..."