diff --git a/docs/asset-engine/design-brief.md b/docs/asset-engine/design-brief.md new file mode 100644 index 0000000..d4b86b6 --- /dev/null +++ b/docs/asset-engine/design-brief.md @@ -0,0 +1,401 @@ +# Asset Generation Engine — Design Brief + +For an internal UI that fronts the inference services on `irv-ml1` +catalogued in [`services.yaml`](services.yaml). Hand this to a design +agent before any pixels. + +> **What this is.** A FastAPI + HTMX + Shoelace web app that replaces +> `curl` for routine use of ~12 self-hosted inference services +> (TTS, ASR, SFX, music). Single-tenant in v1; the data model and +> code seams are designed so v2/v3 can grow into multi-user with +> auth, tagging, projects, and async job queues without +> architectural churn. +> +> **Not** a public product. **Not** multi-tenant on day 1. **Not** a +> generic "AI playground" — it's specifically a tool for *generating +> reusable assets* (voiceover takes, hold tones, music beds, SFX, +> transcripts) that get pulled into other projects. + +--- + +## 1. Locked architectural decisions + +These shape the data model and the code seams. They are not +negotiable in v1, because retrofitting them after the database has +real rows in it is expensive. + +### 1.1 Asset is a first-class entity + +Every successful generation writes one `Asset` row: + +``` +Asset + id uuid primary key + service_id text -- 'kokoro', 'sao', 'ace-step', ... + service_version int -- catalog `version:` at generation time + model_id text -- 'hexgrad/Kokoro-82M' + model_revision text? -- HF SHA when known, else null + image_ref text -- container image at generation time (tag or digest) + params json -- the request body that produced this asset + output_hash text -- sha256 of the output bytes + output_path text -- 'outputs/.' + output_mime text -- 'audio/wav', 'image/png', 'text/plain' + duration_s float? -- for audio/video + byte_size int + created_by int -- FK -> User.id (always 1 in v1) + created_at datetime + tags json -- list[str]; empty in v1, surfaced in v2 + collection_id uuid? -- FK -> Collection; null in v1 + retention text -- 'ephemeral' | 'keep' | 'pinned' + source_asset_id uuid? -- FK -> Asset; non-null when this is a fork/regenerate +``` + +The presence of this row from day 1 is what enables the v2/v3 stories +(library, search, dedupe, regenerate, share, export). Backfilling +this from server logs after the fact is grim — don't. + +### 1.2 Content-addressed output storage + +Outputs live at `outputs/.`. The asset row +points at the hash. This gets us, for free: + +- **Dedupe**: same prompt + seed + model produces the same bytes → + same hash → one file on disk, multiple Asset rows referencing it. +- **Garbage collection**: `delete from blobs where ref_count = 0`. +- **Verification**: an Asset row whose `output_hash` doesn't match + the bytes on disk is corrupt; we know to re-run. +- **Migration safety**: blob layout is stable; you can move + `outputs/` to a different filesystem without touching the DB. + +Path conventions like `outputs/2026/05/10/kokoro_.wav` rot +within months. Don't. + +### 1.3 Reproducibility is a hard requirement + +Every Asset must be regeneratable from its `params` row. This means: + +- **Explicit seed** for any service that supports sampling. (SAO and + ace-step do; everything else is deterministic-without-sampling and + requires `image_ref` pinning instead.) +- **`model_revision`** when the upstream exposes a SHA. When it + doesn't, `image_ref` becomes the reproducibility anchor. +- **Service schema versioning** — when `services.yaml` changes a + field's shape, bump that service's `version:`. Old Asset rows + remember which version they were generated against. + +If a service can't produce reproducible output, that's a bug in the +*service* to fix before it goes in the catalog — not a UI concern. + +### 1.4 Job table from day 1 + +Every Asset is produced by a Job: + +``` +Job + id uuid primary key + asset_id uuid? -- FK -> Asset; null until done + service_id text + params json + state text -- 'queued' | 'running' | 'done' | 'failed' + error text? + worker text? -- which process picked it up; null in v1 + queued_at datetime + started_at datetime? + finished_at datetime? +``` + +In v1 this is synchronous: insert → run inline → update → return. +In v2 it becomes a worker pool. In v3 it gains priority + per-user +quota. **The query surface stays the same throughout** — UI +templates that render "running jobs" or "job history" don't change. + +### 1.5 Auth as a no-op dependency, not absent + +Every request goes through a `current_user()` dependency: + +```python +def current_user() -> User: + # v1: hardcoded; v2: header-injection from upstream proxy; + # v3: OIDC. + return User(id=1, name="me") +``` + +Downstream code uses `request.user.id` from day 1. v2's swap is a +dependency-injection change, not a refactor. **A code base that +assumes "no users" cannot be retrofitted with auth without +touching everything.** + +### 1.6 API surface ≠ UI surface + +Every UI action should be powered by a JSON API route. The HTMX +templates call those API routes via `hx-post` / `hx-get` and render +the JSON into HTML partials, OR the API routes serve dual content +(JSON + partial) by route prefix: + +- `POST /api/v1/jobs` — JSON body in, JSON body out +- `POST /ui/jobs` — same handler called server-side, returns HTMX partial + +Pick the **separate-route** pattern, not content negotiation by +`Accept:` header. It's noisier but clearer; debugging API issues +in v3 with content negotiation in the way is hell. + +### 1.7 Service catalog is git-versioned and additive + +[`services.yaml`](services.yaml) is the contract. Adding a service is +a YAML edit + server reload — no template authoring, no Python +changes. Each service entry carries `version:`. Schema changes bump +the version. Old assets remember which version they were generated +against (via `Asset.service_version`). + +### 1.8 Tags + collections plumbed in the model from day 1 + +`Asset.tags: list[str]`, `Asset.collection_id: uuid?`. **No UI +required in v1** — but the columns must exist. Adding the columns +later is a migration touching every existing row; harmless to leave +empty for now. + +--- + +## 2. The contract: `services.yaml` + +The UI's form generator and response renderer are dispatched off +[`services.yaml`](services.yaml). The agent should treat this file +as the contract — **everything in the UI is parameterized by it**. + +### 2.1 Field-type vocabulary + +The form generator must support exactly these field types. No +others. New input shapes added to a service must reduce to these: + +| `type:` | Renders as | Notes | +|-------------|-------------------------------------------|-------| +| `text` | `` single-line | for short strings | +| `textarea` | `` multi-line, autosize | prompts, lyrics | +| `number` | `` | for integer/float without bounded range | +| `slider` | `` + numeric readout | requires `min`, `max`, optional `step` | +| `select` | `` | static `options:` OR remote `source_url:` | +| `bool` | `` | | +| `file` | `` + drag-drop zone | requires `accepted_types:` | +| `json` | `` with monospace + JSON parse on submit | for arrays-of-floats, refs, etc. | + +Field attributes (per service entry): + +- `name` (required) +- `type` (required) +- `label` (defaults to titlecased name) +- `required` (default false) +- `optional: true` (sugar for `required: false`) +- `default` +- `min` / `max` / `step` (for `number` / `slider`) +- `options: [...]` (for `select` with static options) +- `source_url: ...` + `source_jsonpath: ...` (for `select` with + dynamic options — the UI fetches once per session, caches) +- `description` (rendered as Shoelace help-text) +- `max_length` +- `accepted_types: [...]` (for `file`) + +If a service's Pydantic model needs something this vocabulary can't +express, the catalog is wrong, not the vocabulary. **Do not extend +the vocabulary** — fix the catalog or fix the service. + +### 2.2 Response-renderer vocabulary + +| `response.type:` | Renders as | +|------------------|-----------------------------------------------| +| `audio` | inline `