# Asset Generation Engine — Design Brief For an internal UI that fronts the inference services on `irv-ml1` catalogued in [`services.yaml`](services.yaml). Hand this to a design agent before any pixels. > **What this is.** A FastAPI + HTMX + Shoelace web app that replaces > `curl` for routine use of ~12 self-hosted inference services > (TTS, ASR, SFX, music). Single-tenant in v1; the data model and > code seams are designed so v2/v3 can grow into multi-user with > auth, tagging, projects, and async job queues without > architectural churn. > > **Not** a public product. **Not** multi-tenant on day 1. **Not** a > generic "AI playground" — it's specifically a tool for *generating > reusable assets* (voiceover takes, hold tones, music beds, SFX, > transcripts) that get pulled into other projects. --- ## 1. Locked architectural decisions These shape the data model and the code seams. They are not negotiable in v1, because retrofitting them after the database has real rows in it is expensive. ### 1.1 Asset is a first-class entity Every successful generation writes one `Asset` row: ``` Asset id uuid primary key service_id text -- 'kokoro', 'sao', 'ace-step', ... service_version int -- catalog `version:` at generation time model_id text -- 'hexgrad/Kokoro-82M' model_revision text? -- HF SHA when known, else null image_ref text -- container image at generation time (tag or digest) params json -- the request body that produced this asset output_hash text -- sha256 of the output bytes output_path text -- 'outputs/.' output_mime text -- 'audio/wav', 'image/png', 'text/plain' duration_s float? -- for audio/video byte_size int created_by int -- FK -> User.id (always 1 in v1) created_at datetime tags json -- list[str]; empty in v1, surfaced in v2 collection_id uuid? -- FK -> Collection; null in v1 retention text -- 'ephemeral' | 'keep' | 'pinned' source_asset_id uuid? -- FK -> Asset; non-null when this is a fork/regenerate ``` The presence of this row from day 1 is what enables the v2/v3 stories (library, search, dedupe, regenerate, share, export). Backfilling this from server logs after the fact is grim — don't. ### 1.2 Content-addressed output storage Outputs live at `outputs/.`. The asset row points at the hash. This gets us, for free: - **Dedupe**: same prompt + seed + model produces the same bytes → same hash → one file on disk, multiple Asset rows referencing it. - **Garbage collection**: `delete from blobs where ref_count = 0`. - **Verification**: an Asset row whose `output_hash` doesn't match the bytes on disk is corrupt; we know to re-run. - **Migration safety**: blob layout is stable; you can move `outputs/` to a different filesystem without touching the DB. Path conventions like `outputs/2026/05/10/kokoro_.wav` rot within months. Don't. ### 1.3 Reproducibility is a hard requirement Every Asset must be regeneratable from its `params` row. This means: - **Explicit seed** for any service that supports sampling. (SAO and ace-step do; everything else is deterministic-without-sampling and requires `image_ref` pinning instead.) - **`model_revision`** when the upstream exposes a SHA. When it doesn't, `image_ref` becomes the reproducibility anchor. - **Service schema versioning** — when `services.yaml` changes a field's shape, bump that service's `version:`. Old Asset rows remember which version they were generated against. If a service can't produce reproducible output, that's a bug in the *service* to fix before it goes in the catalog — not a UI concern. ### 1.4 Job table from day 1 Every Asset is produced by a Job: ``` Job id uuid primary key asset_id uuid? -- FK -> Asset; null until done service_id text params json state text -- 'queued' | 'running' | 'done' | 'failed' error text? worker text? -- which process picked it up; null in v1 queued_at datetime started_at datetime? finished_at datetime? ``` In v1 this is synchronous: insert → run inline → update → return. In v2 it becomes a worker pool. In v3 it gains priority + per-user quota. **The query surface stays the same throughout** — UI templates that render "running jobs" or "job history" don't change. ### 1.5 Auth as a no-op dependency, not absent Every request goes through a `current_user()` dependency: ```python def current_user() -> User: # v1: hardcoded; v2: header-injection from upstream proxy; # v3: OIDC. return User(id=1, name="me") ``` Downstream code uses `request.user.id` from day 1. v2's swap is a dependency-injection change, not a refactor. **A code base that assumes "no users" cannot be retrofitted with auth without touching everything.** ### 1.6 API surface ≠ UI surface Every UI action should be powered by a JSON API route. The HTMX templates call those API routes via `hx-post` / `hx-get` and render the JSON into HTML partials, OR the API routes serve dual content (JSON + partial) by route prefix: - `POST /api/v1/jobs` — JSON body in, JSON body out - `POST /ui/jobs` — same handler called server-side, returns HTMX partial Pick the **separate-route** pattern, not content negotiation by `Accept:` header. It's noisier but clearer; debugging API issues in v3 with content negotiation in the way is hell. ### 1.7 Service catalog is git-versioned and additive [`services.yaml`](services.yaml) is the contract. Adding a service is a YAML edit + server reload — no template authoring, no Python changes. Each service entry carries `version:`. Schema changes bump the version. Old assets remember which version they were generated against (via `Asset.service_version`). ### 1.8 Tags + collections plumbed in the model from day 1 `Asset.tags: list[str]`, `Asset.collection_id: uuid?`. **No UI required in v1** — but the columns must exist. Adding the columns later is a migration touching every existing row; harmless to leave empty for now. --- ## 2. The contract: `services.yaml` The UI's form generator and response renderer are dispatched off [`services.yaml`](services.yaml). The agent should treat this file as the contract — **everything in the UI is parameterized by it**. ### 2.1 Field-type vocabulary The form generator must support exactly these field types. No others. New input shapes added to a service must reduce to these: | `type:` | Renders as | Notes | |-------------|-------------------------------------------|-------| | `text` | `` single-line | for short strings | | `textarea` | `` multi-line, autosize | prompts, lyrics | | `number` | `` | for integer/float without bounded range | | `slider` | `` + numeric readout | requires `min`, `max`, optional `step` | | `select` | `` | static `options:` OR remote `source_url:` | | `bool` | `` | | | `file` | `` + drag-drop zone | requires `accepted_types:` | | `json` | `` with monospace + JSON parse on submit | for arrays-of-floats, refs, etc. | Field attributes (per service entry): - `name` (required) - `type` (required) - `label` (defaults to titlecased name) - `required` (default false) - `optional: true` (sugar for `required: false`) - `default` - `min` / `max` / `step` (for `number` / `slider`) - `options: [...]` (for `select` with static options) - `source_url: ...` + `source_jsonpath: ...` (for `select` with dynamic options — the UI fetches once per session, caches) - `description` (rendered as Shoelace help-text) - `max_length` - `accepted_types: [...]` (for `file`) If a service's Pydantic model needs something this vocabulary can't express, the catalog is wrong, not the vocabulary. **Do not extend the vocabulary** — fix the catalog or fix the service. ### 2.2 Response-renderer vocabulary | `response.type:` | Renders as | |------------------|-----------------------------------------------| | `audio` | inline `