feat(arbo): add irv-ml1 co-located engine stack (ADR-0001)

New stack mirroring the canonical convention for the Arbo (catalog) engine,
co-located beside comfyui on irv-ml1 per ADR-0001 D1/D3:

- engine<->ComfyUI over traefik-net container DNS (http://comfyui:8188),
  SSH dependency eliminated; file ops bind the shared basedir input/output
- named local-disk volumes for the gallery SQLite (arbo_db, restic-backed)
  and hero images (arbo_heroes); catalog as a ro git-checkout mount (D2)
- ENGINE_TOKEN + GRANITE_KEY via on-host .env; GRANITE via the LiteLLM gateway
- Q5 catalog-pull: manual day-1, ytvc-style webhook follow-on

Image build, /healthz, catalog in-container path, and non-root UID are
comfy-dev's to confirm (CONFIRM items in README).
This commit is contained in:
vh
2026-06-12 10:25:04 -07:00
parent 005effd664
commit ee57e69ce8
3 changed files with 184 additions and 0 deletions
+72
View File
@@ -0,0 +1,72 @@
# arbo — catalog-driven ComfyUI engine
Arbo's `comfy_engine` (FastAPI) reads a versioned workflow catalog and drives
**ComfyUI** to generate images/video, serving an SPA + `/workflows` + `/run`.
Owned by **comfy-dev** (engine code + catalog + image build); this stack is the
**infra side** — compose, deploy, secrets, volumes, pull automation. See
`vh/arbo` `docs/adr/0001-engine-production-deployment.md` for the architecture.
## Placement (ADR-0001)
- **Host:** irv-ml1, **co-located beside the `comfyui` stack** (D1/D3).
- **engine ↔ ComfyUI:** `http://comfyui:8188` over the shared `traefik-net`
(the "localhost-equivalent" — no scp, no WireGuard hop). File ops use the
**same host dirs** comfyui mounts (`/worktank/comfyui/basedir/{input,output}`),
bound into the engine — so `ARBO_COMFY_SSH_HOST` stays **unset** and the
engine's local-cp path (comfy-dev's D3 follow-on) handles file ops.
## Deploy
```bash
# 1. comfy-dev builds + pushes the image; pin the tag in .env (ARBO_IMAGE).
# 2. On irv-ml1, clone the catalog checkout (the git-pull target, D2):
# git clone git@gitea.phasefinal.com:vh/arbo.git /worktank/arbo/repo
# 3. Create stacks/arbo/.env from .env.example on the host, fill secrets.
# 4. From this workspace:
scripts/deploy-stack.sh irv-ml1 arbo # diffs vs live, prompts y/N
```
## Q3 — Secrets
Both injected via the on-host **`.env`** (gitignored; never the althing bus),
the standard fleet pattern:
- **`ENGINE_TOKEN`** — auth is **OPEN today**. Mint a real bearer
(`openssl rand -hex 32`), put it in `.env`. Closing this is the one security
fix the move buys us.
- **`GRANITE_KEY`** — the LiteLLM **virtual key scoped to arbo** (granite-only).
comfy-dev already holds `arbo-prompt-enhance` (issued 2026-06-09); reuse or
rotate it. Never the master `sk-corvid`.
## Q4 — Volumes + backup
| Volume | Holds | Durability |
|---|---|---|
| `arbo_db` (named, local disk) | gallery/history SQLite (`ARBO_DB`) | **backed up** — restic/Backrest daily |
| `arbo_heroes` (named, local disk) | runtime hero images | persisted; regenerable, lower backup priority |
| `/worktank/arbo/repo` (bind, ro) | catalog checkout | reproducible from git — not backed up |
`ARBO_DB` is a **named volume on local disk, NOT NFS** (the DB-off-NFS rule;
SQLite-on-NFS locking is a foot-gun). The gallery DB is the durable asset → it
joins the restic file backup. (Adds irv-ml1's `arbo_db` path to a Backrest
repo — small; flag if irv-ml1 isn't yet a restic source.)
## Q5 — Catalog-pull automation
- **Day 1:** manual `git pull` in `/worktank/arbo/repo` + `deploy-stack.sh
irv-ml1 arbo` restart (or `docker compose restart arbo`).
- **Follow-on (recommended):** mirror the **yt-voice-clipper webhook** already
live on irv-ml1 — gitea webhook → HMAC listener → `git pull` + `compose
restart`, so a comfy-dev catalog push reaches prod in one action (D2's "cheap
to reach prod"). Layered after the stack is up.
## Items needing comfy-dev's image (jointly owned)
1. **`ARBO_IMAGE` tag** — the built `gitea.phasefinal.com/vh/arbo:<tag>`.
2. **`/healthz`** — the healthcheck assumes it; comfy-dev is adding it.
3. **Catalog in-container path** — compose mounts the checkout's `catalog/` +
`graphs/` to `/app/catalog` + `/app/graphs` (ro). Confirm against the image's
expected layout (or expose a catalog-dir env), so the engine reloads from the
mount on restart.
4. **Run-as UID** — container runs `1000:1000` to write comfyui's basedir +
volumes cleanly; confirm the image tolerates a non-root user.