docs/asset-engine: promote services.yaml to first-class contract
Adds the supporting infra around the service catalog now that it has external consumers (the asset_engine UI being the first; CLIs, monitoring, other services may follow): - CATALOG-CONTRACT.md: the consumer-facing contract. Defines versioning policy (catalog_version vs per-service version), closed field-type and response-type vocabularies, recommended vendor+drift-check sync workflow, known-consumers list, service authoring notes. - services.schema.json: JSON Schema (draft 2020-12) for the catalog. Generated from the Pydantic model in ~/development/asset_engine/src/asset_engine/catalog.py via `uv run scripts/dump_schema.py --publish`. Lets non-Python consumers validate against the same shape. - services.yaml: adds catalog_version: 1 at the root and reframes the file's header to call out its first-class-contract status. Quotes a vibevoice label that contained an unescaped colon (caught by the asset_engine's strict YAML parser on first sync).
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
# services.yaml — contract for downstream consumers
|
||||
|
||||
`services.yaml` in this directory is the **canonical, versioned**
|
||||
description of the inference services this fleet exposes. It is a
|
||||
first-class artifact: external tools (the asset-engine UI is the
|
||||
first; CLIs, monitoring dashboards, and other internal services may
|
||||
follow) consume it programmatically, and changes to its shape or
|
||||
contents have downstream effect.
|
||||
|
||||
This document is the consumer contract.
|
||||
|
||||
## Where the data lives
|
||||
|
||||
- **Canonical source**: `docs/asset-engine/services.yaml` in the
|
||||
`eshpfi-management` repo (this file's directory).
|
||||
- **Schema**: `docs/asset-engine/services.schema.json` (JSON Schema
|
||||
draft 2020-12). Generated from the Pydantic model in
|
||||
`~/development/asset_engine/src/asset_engine/catalog.py` —
|
||||
regenerate via `cd ~/development/asset_engine && uv run scripts/dump_schema.py`
|
||||
whenever the model changes.
|
||||
- **Change history**: git log on this directory.
|
||||
|
||||
## Data model — high level
|
||||
|
||||
```yaml
|
||||
catalog_version: 1 # bump on SCHEMA changes
|
||||
services: [Service] # the entries; closed list
|
||||
reproducibility_audit: [Audit] # one entry per service
|
||||
```
|
||||
|
||||
### Service entry
|
||||
|
||||
| field | type | required | meaning |
|
||||
|-------------------|-------------------------------------------|----------|---------|
|
||||
| `id` | string | yes | stable URL-safe identifier; never reused |
|
||||
| `name` | string | yes | display name |
|
||||
| `description` | string | no | one-paragraph summary |
|
||||
| `category` | enum: tts, asr, sfx, music, image | yes | groups in pickers |
|
||||
| `version` | int | yes | per-service schema version; bump on field changes |
|
||||
| `status` | enum: ready, catalog-deferred, experimental | no | default: ready |
|
||||
| `host` | string (host alias) | yes | which fleet host serves this |
|
||||
| `endpoint` | URL | yes | absolute URL of the inference endpoint |
|
||||
| `method` | string | yes | HTTP method (POST common) |
|
||||
| `content_type` | MIME type | yes | request body type |
|
||||
| `model.id` | string | yes | upstream model id (HF or other) |
|
||||
| `model.revision` | string \| null | no | SHA when known |
|
||||
| `model.image` | string | yes | container image ref this is hosted from |
|
||||
| `fields` | list[Field] | no | request parameters; empty for catalog-deferred |
|
||||
| `response.type` | enum: audio, image, video, text, json, file | yes | response renderer hint |
|
||||
| `response.mime` | string | no | static response MIME |
|
||||
| `response.mime_from_field` | string | no | name of a field whose value determines the MIME |
|
||||
| `response.output_field` | string | no | for JSON responses, the key holding the asset |
|
||||
| `reproducibility.seedable` | bool | yes | does the endpoint accept a seed? |
|
||||
| `reproducibility.deterministic` | bool | yes | same params → same bytes? |
|
||||
| `reproducibility.notes` | string | no | gotchas |
|
||||
| `estimated_latency.cold_start_s` | int \| float | no | informational only |
|
||||
| `estimated_latency.warm_per_unit` | string | no | informational only |
|
||||
| `license` | string | yes | upstream license |
|
||||
| `license_warning` | string | no | non-empty → consumer MUST surface a warning before use |
|
||||
| `notes` | string | no | freeform |
|
||||
|
||||
### Field entry
|
||||
|
||||
The field-type vocabulary is **closed**. Adding a new type requires:
|
||||
(a) updating the Pydantic model in `asset_engine/catalog.py`,
|
||||
(b) bumping `catalog_version`, (c) coordinating with all consumers.
|
||||
|
||||
| `type` | meaning |
|
||||
|------------|------------------------------------------------------|
|
||||
| `text` | single-line string |
|
||||
| `textarea` | multi-line string |
|
||||
| `number` | int or float, no bounded range |
|
||||
| `slider` | bounded numeric — requires `min`, `max`, optional `step` |
|
||||
| `select` | enum — `options:` list OR remote `source_url:` + `source_jsonpath:` |
|
||||
| `bool` | true/false |
|
||||
| `file` | file upload — requires `accepted_types:` list of MIME |
|
||||
| `json` | freeform JSON value (arrays of floats, ref objects, etc) |
|
||||
|
||||
Optional/required: a field is required UNLESS it has either
|
||||
`required: false` or `optional: true`. (Both ways are accepted; new
|
||||
entries should prefer `required: false`.)
|
||||
|
||||
### Response renderer vocabulary
|
||||
|
||||
Closed set: `audio`, `image`, `video`, `text`, `json`, `file`.
|
||||
Same change-management as field types.
|
||||
|
||||
## Versioning policy
|
||||
|
||||
| Change | Bump |
|
||||
|----------------------------------------------|-----------------------|
|
||||
| Add a new service | nothing |
|
||||
| Add a non-required field to an existing service | service `version:` |
|
||||
| Change a field's type, range, or default | service `version:` |
|
||||
| Remove a service | service `version:` (sentinel: removed=true), then drop in next catalog_version bump |
|
||||
| Add a new entry to the field-type vocabulary | `catalog_version:` |
|
||||
| Add a new entry to the response-type vocabulary | `catalog_version:` |
|
||||
| Drop a field-type or response-type | `catalog_version:` (BREAKING) |
|
||||
| Pure prose edit (description, notes) | nothing |
|
||||
|
||||
Consumers should pin their tested `catalog_version`. When this repo
|
||||
bumps it, downstreams should re-test before consuming the new shape.
|
||||
|
||||
## Sync workflow for downstream consumers
|
||||
|
||||
The recommended pattern is **vendor + drift-check**, not
|
||||
fetch-at-runtime. Reasons: deployments stay reproducible; airgapped
|
||||
or network-flaky deploys still work; schema changes surface in PRs
|
||||
instead of breaking production silently.
|
||||
|
||||
A consumer should:
|
||||
|
||||
1. Vendor a copy of `services.yaml` in their repo (e.g. at
|
||||
`data/services.yaml`).
|
||||
2. Record the source SHA they synced from in a `services.yaml.lock`
|
||||
alongside (path, source SHA, sync timestamp).
|
||||
3. Provide a `sync_catalog` script that pulls the latest from the
|
||||
canonical path and updates both files.
|
||||
4. Validate the vendored file against `services.schema.json` on
|
||||
load — refuse to start if validation fails.
|
||||
5. CI: re-validate on every commit.
|
||||
|
||||
The asset-engine repo at `~/development/asset_engine/` is the
|
||||
reference implementation of this pattern; copy the script shape from
|
||||
`scripts/sync_catalog.py` over there.
|
||||
|
||||
## Known consumers
|
||||
|
||||
| Consumer | Vendored at | Pin policy |
|
||||
|---------------------------------------------|--------------------------------------|-------------------|
|
||||
| `asset_engine` (FastAPI/HTMX UI; ~/development/asset_engine/) | `data/services.yaml` + `data/services.yaml.lock` | catalog_version=1 |
|
||||
|
||||
When you add a consumer, add a row here in the same PR that lands
|
||||
the consumer. This list is what we audit when planning a
|
||||
`catalog_version` bump.
|
||||
|
||||
## Service authoring notes
|
||||
|
||||
When **adding** a service to the catalog:
|
||||
|
||||
- The Pydantic model in code is the ground truth for `fields:`. Read
|
||||
the actual server code, not the README; READMEs rot. The
|
||||
reproducibility audit at the bottom of `services.yaml` MUST get a
|
||||
matching entry.
|
||||
- If a service can't produce reproducible output (no seed AND
|
||||
non-deterministic model), that's a **bug in the service** — fix it
|
||||
there before adding to the catalog. The asset-engine relies on
|
||||
reproducibility to power "regenerate" / "fork" affordances.
|
||||
- If the service has a non-permissive license, populate
|
||||
`license_warning:` with the user-facing language. The UI is
|
||||
required to surface it before a generation; missing it is a
|
||||
contract violation.
|
||||
- Image refs SHOULD be digest-pinned (`@sha256:...`) for true
|
||||
reproducibility. Tag-pinned (`:v1.2.3`) is acceptable but
|
||||
vulnerable to upstream re-pushes.
|
||||
|
||||
When **removing** a service: see the versioning policy table.
|
||||
Removal is a two-step process; surprise removal breaks consumers'
|
||||
asset libraries (regenerate-from-history breaks).
|
||||
Reference in New Issue
Block a user