vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b19581558633dcf9ad1e17754ec4413437fbcb3d
esh-pfi-infrastructure/docs/asset-engine/CATALOG-CONTRACT.md
T
vh c98a12baf4 catalog(fish-s2): opt references into togglable; catalog_version 1->2 
asset-engine shipped the per-field enable-toggle (v0.1.14/.15) — the
durable fix for the "form submits untouched fields" family. A field
marked togglable:true renders with an OFF-by-default switch: while off
the control is disabled (excluded from submission) AND the server skips
injecting its default, so it is genuinely not sent until the user opts
in.

Per operator direction, opt fish-s2's `references` (inline-base64
Custom-clone) field in — it already satisfies the togglable-requires-
optional validator (optional:true, no default). The advanced clone
field now renders dormant and can never silently override the Voice
dropdown again.

This is a SCHEMA change (new CatalogField property), so:
- services.schema.json: add `togglable` (boolean, default false),
  mirroring the asset_engine Pydantic model that generates this schema.
- catalog_version 1 -> 2 (header: bump on schema changes).
- CATALOG-CONTRACT.md: consumer pin note -> catalog_version=2.

Scoped to `references` only. The chatterbox/dia2 clone fields are the
same family but NOT toggled: dia2 deliberately defaults voice_mode=clone
+ a clone ref as its stable out-of-box voice, and toggling that field
would change dia2's default-voice behavior (the earlier 404 fix).

Validated: jsonschema accepts togglable; additionalProperties:false
guard still rejects unknown props.
2026-06-01 18:05:56 -07:00

189 lines
12 KiB
Markdown
Raw Blame History

# services.yaml — contract for downstream consumers
`services.yaml` in this directory is the **canonical, versioned**
description of the inference services this fleet exposes. It is a
first-class artifact: external tools (the asset-engine UI is the
first; CLIs, monitoring dashboards, and other internal services may
follow) consume it programmatically, and changes to its shape or
contents have downstream effect.
This document is the consumer contract.
## Where the data lives
- **Canonical source**: `docs/asset-engine/services.yaml` in the
`eshpfi-management` repo (this file's directory).
- **Schema**: `docs/asset-engine/services.schema.json` (JSON Schema
draft 2020-12). Generated from the Pydantic model in
`~/development/asset_engine/src/asset_engine/catalog.py` —
regenerate via `cd ~/development/asset_engine && uv run scripts/dump_schema.py`
whenever the model changes.
- **Change history**: git log on this directory.
## Data model — high level
```yaml
catalog_version: 1 # bump on SCHEMA changes
services: [Service] # the entries; closed list
reproducibility_audit: [Audit] # one entry per service
```
### Service entry
| field | type | required | meaning |
|-------------------|-------------------------------------------|----------|---------|
| `id` | string | yes | stable URL-safe identifier; never reused |
| `name` | string | yes | display name |
| `description` | string | no | one-paragraph summary |
| `category` | enum: tts, asr, sfx, music, image | yes | groups in pickers |
| `version` | int | yes | per-service schema version; bump on field changes |
| `status` | enum: ready, catalog-deferred, experimental, down | no | default: ready |
**Status semantics:**
| value | meaning | picker behavior |
|---|---|---|
| `ready` | Deployed, exercised, current default | clickable, primary surface |
| `experimental` | Deployed, may break under unusual params or while a custom renderer is being built (e.g. kokoro-captioned pre-renderer) | clickable, surfaced with a "experimental" tag |
| `catalog-deferred` | By-design out of scope this catalog version (e.g. ComfyUI requires per-asset workflow templates the catalog can't express today) | greyed, non-clickable, "deferred" tooltip |
| `down` | *Temporarily* unreachable on the host fleet — deployment gap, scheduled maintenance, container reaped, etc. Distinct from `catalog-deferred` because the intent is to flip back to `ready` once redeployed; the catalog row still describes a real intended service. | greyed, non-clickable, "service down" tooltip |
| `host` | string (host alias) | yes | which fleet host serves this |
| `endpoint` | URL | yes | absolute URL of the inference endpoint |
| `method` | string | yes | HTTP method (POST common) |
| `content_type` | MIME type | yes | request body type |
| `model.id` | string | yes | upstream model id (HF or other) |
| `model.revision` | string \| null | no | SHA when known |
| `model.image` | string | yes | container image ref this is hosted from |
| `fields` | list[Field] | no | request parameters; empty for catalog-deferred |
| `section_groups` | list[SectionGroup] | no | named groups for progressive disclosure; consumers render fields under tabs/accordions per group. SectionGroup = `{id, label, hint?}`. Field.section refs must resolve here. |
| `response.type` | enum: audio, image, video, text, json, file | yes | renderer dispatch (closed vocabulary) |
| `response.mime` | string | no | static response MIME (raw-bytes wire) |
| `response.mime_from_field` | string | no | name of a field whose value determines the MIME (raw-bytes wire) |
| `response.output_field` | string | no | for text/json responses, JSON key holding the asset |
| `response.audio_field` | string | no | for `type: audio` with JSON-envelope wire, JSON key holding base64-encoded audio bytes |
| `response.audio_format_field` | string | no | for `type: audio` with JSON-envelope wire, JSON key holding the decoded audio MIME (e.g. "audio/wav") |
| `response.timestamps_field` | string | no | JSON key holding a structured timestamps array — independent of type, declared by any service that emits time-aligned markers alongside its primary output |
| `reproducibility.seedable` | bool | yes | does the endpoint accept a seed? |
| `reproducibility.deterministic` | bool | yes | same params → same bytes? |
| `reproducibility.notes` | string | no | gotchas |
| `estimated_latency.cold_start_s` | int \| float | no | informational only |
| `estimated_latency.warm_per_unit` | string | no | informational only |
| `license` | string | yes | upstream license |
| `license_warning` | string | no | non-empty → consumer MUST surface a warning before use |
| `notes` | string | no | freeform |
### Field entry
The field-type vocabulary is **closed**. Adding a new type requires:
(a) updating the Pydantic model in `asset_engine/catalog.py`,
(b) bumping `catalog_version`, (c) coordinating with all consumers.
| `type` | meaning |
|------------|------------------------------------------------------|
| `text` | single-line string |
| `textarea` | multi-line string |
| `number` | int or float, no bounded range |
| `slider` | bounded numeric — requires `min`, `max`, optional `step` |
| `select` | enum — `options:` list OR remote `source_url:` + `source_jsonpath:` |
| `bool` | true/false |
| `file` | file upload — requires `accepted_types:` list of MIME |
| `json` | freeform JSON value (arrays of floats, ref objects, etc) |
Optional/required: a field is required UNLESS it has either
`required: false` or `optional: true`. (Both ways are accepted; new
entries should prefer `required: false`.)
### Response renderer vocabulary
Closed set: `audio`, `image`, `video`, `text`, `json`, `file`.
Same change-management as field types.
## Versioning policy
| Change | Bump |
|----------------------------------------------|-----------------------|
| Add a new service | nothing |
| Add a non-required field to an existing service | service `version:` |
| Add an optional key to the `response:` schema (e.g. audio_field, timestamps_field) | nothing — additive, backward-compatible |
| Add an optional key to a Field (e.g. section) or to Service (e.g. section_groups) | nothing — additive, backward-compatible |
| Extend an existing enum (StatusT, FieldTypeT, ResponseTypeT, CategoryT) with a non-conflicting new value, with the consumer updated in the same coordinated change | nothing — older consumers don't encounter the new value; updated consumers parse it correctly |
| Change a field's type, range, or default | service `version:` |
| Remove a service | service `version:` (sentinel: removed=true), then drop in next catalog_version bump |
| Add a new entry to the field-type vocabulary | `catalog_version:` |
| Add a new entry to the response-type vocabulary | `catalog_version:` |
| Drop a field-type or response-type | `catalog_version:` (BREAKING) |
| Pure prose edit (description, notes) | nothing |
Consumers should pin their tested `catalog_version`. When this repo
bumps it, downstreams should re-test before consuming the new shape.
## Sync workflow for downstream consumers
The recommended pattern is **vendor + drift-check**, not
fetch-at-runtime. Reasons: deployments stay reproducible; airgapped
or network-flaky deploys still work; schema changes surface in PRs
instead of breaking production silently.
A consumer should:
1. Vendor a copy of `services.yaml` in their repo (e.g. at
`data/services.yaml`).
2. Record the source SHA they synced from in a `services.yaml.lock`
alongside (path, source SHA, sync timestamp).
3. Provide a `sync_catalog` script that pulls the latest from the
canonical path and updates both files.
4. Validate the vendored file against `services.schema.json` on
load — refuse to start if validation fails.
5. CI: re-validate on every commit.
The asset-engine repo at `~/development/asset_engine/` is the
reference implementation of this pattern; copy the script shape from
`scripts/sync_catalog.py` over there.
## Known consumers
| Consumer | Vendored at | Pin policy |
|---------------------------------------------|--------------------------------------|-------------------|
| `asset_engine` (FastAPI/HTMX UI; ~/development/asset_engine/) | `data/services.yaml` + `data/services.yaml.lock` | catalog_version=2 |
When you add a consumer, add a row here in the same PR that lands
the consumer. This list is what we audit when planning a
`catalog_version` bump.
## Service authoring notes
When **adding** a service to the catalog:
- **Source-of-truth precedence.** OpenAPI is convenient but often lies
by omission (bare `string` types where the dispatch chain enforces
a specific enum, missing required-only-in-the-handler fields, etc).
The actual ground truth is, in order:
1. The Pydantic input model — for field shape, types, required-ness.
2. The handler / pipeline code — for enum dispatches and runtime
validation.
3. The Gradio UI / client code (when present) — for blessed
defaults and slider ranges, since the model authors picked
these for human-facing UX.
4. The README — least reliable; rots fastest.
Every catalog change should cite which of these was read. "OpenAPI
said X" is not enough on its own — past audits have caught three
bugs in a single service (ace-step) that all looked correct in
OpenAPI.
- The reproducibility audit at the bottom of `services.yaml` MUST get
a matching entry for any new service.
- If a service can't produce reproducible output (no seed AND
non-deterministic model), that's a **bug in the service** — fix it
there before adding to the catalog. The asset-engine relies on
reproducibility to power "regenerate" / "fork" affordances.
- If the service has a non-permissive license, populate
`license_warning:` with the user-facing language. The UI is
required to surface it before a generation; missing it is a
contract violation.
- Image refs SHOULD be digest-pinned (`@sha256:...`) for true
reproducibility. Tag-pinned (`:v1.2.3`) is acceptable but
vulnerable to upstream re-pushes.
When **removing** a service: see the versioning policy table.
Removal is a two-step process; surprise removal breaks consumers'
asset libraries (regenerate-from-history breaks).
Reference in New Issue View Git Blame Copy Permalink