feat(waterland-studio): containerise the GPU render service on irv-ml1

Replaces a bare nohup on irv-ml1:8410 that would not have survived a reboot,
handed over by waterland-dev. Tracks vh/waterland @ main (PR #4 merged; main
HEAD is exactly the pinned 8025366).

Build context is a checkout at /opt/waterland-studio/src, deliberately OUTSIDE
the compose dir — deploy-stack.sh rsyncs stacks/<stack>/ with --delete and
would otherwise eat it. The Dockerfile is passed out-of-context.

Three landmines, all measured:

1. Both uv extras are load-bearing at build AND run. jobs.py shells the
   renderer out as a literal  with no --extra flags, so uv
   would re-sync at runtime and prune cupy — silently dropping to the numpy
   path at ~21x wall time. UV_NO_SYNC pins it; UV_OFFLINE makes any failure
   loud instead of quietly slow.

2. cupy needs CUDA HEADERS for its NVRTC compile, not just the driver and the
   wheel's runtime libs. The host has a system CUDA toolkit so the nohup
   process found them by accident; a slim image does not, and every render
   died 1.7s in with 'Failed to find CUDA headers' printed through argparse's
   usage banner — which reads like a CLI bug, not a missing toolkit. Fixed
   with cupy-cuda12x[ctk] (hundreds of MB, vs ~6 GB for a -devel base).

3. The A6000 is host device 1 but container device 0, since compose exposes
   exactly one GPU. CUDA_VISIBLE_DEVICES_TARGET=0 inside; copying the host's
   value selects a device that does not exist.

/root/.cupy is a volume because the NVRTC compile costs ~17s: verified at
23.3s cold vs 6.1s warm, and re-verified across a restart (23.2s on a fresh
cache volume, 6.0s once populated). Warm 256^2+anim beats the 7.4s recorded
against bare metal, so containerising cost nothing.

Job store seeded with the 4 jobs from the displaced instance. Serial by design
(one replica, one card) and unauthenticated, so it stays LAN/WireGuard-only.

stacks/waterland-studio/README.md

@@ -0,0 +1,115 @@
+# waterland-studio — watercolour render service (irv-ml1)
+
+FastAPI + vanilla-JS SPA fronting the `waterland` CLI: upload an image, get a
+watercolour plate and a painted-in reveal animation. Every job shells out to
+the CLI, which runs a fluid simulation on the **A6000**.
+
+- **Host:** irv-ml1 (10.100.79.3, WireGuard-only) · **Port:** 8410
+- **URL:** http://10.100.79.3:8410/ · **Health:** `GET /api/health`
+- **Source:** `gitea.phasefinal.com/vh/waterland`, tracking **`main`**
+
+Handed over by `waterland-dev` on 2026-08-19, replacing a bare `nohup` that
+would not have survived a reboot.
+
+## Layout — the build context is deliberately outside this directory
+
+| path | what |
+|---|---|
+| `/opt/waterland-studio/src` | checkout of `vh/waterland` @ `main` — the build context |
+| `/opt/docker/compose/waterland-studio/` | `compose.yaml`, `Dockerfile`, `update.sh`, `.env` |
+| volume `waterland-studio_waterland_studio_data` | job store (`/data`) |
+| volume `waterland-studio_waterland_studio_kernels` | cupy JIT cache (`/root/.cupy`) |
+
+**The checkout must NOT live under the compose directory.** `deploy-stack.sh`
+rsyncs `stacks/<stack>/` with `--delete`, so a checkout kept beside
+`compose.yaml` would be destroyed by the next deploy of this stack. The
+Dockerfile is passed out-of-context to keep both trees clean.
+
+Refresh source + rebuild:
+
+```bash
+ssh infra-ops@10.100.79.3 /opt/docker/compose/waterland-studio/update.sh
+```
+
+## Three landmines, all of them measured rather than guessed
+
+**1. Both uv extras are load-bearing, at build AND at run.** `gpu` carries
+`cupy-cuda12x`; a bare `uv sync` prunes it and the renderer silently drops to
+the numpy path at ~21x the wall time — it does not error, it just gets slow.
+Worse, `studio/jobs.py` shells the renderer out as a literal `uv run waterland`
+with no `--extra` flags, so uv would re-sync at *runtime* and prune cupy right
+back out. `UV_NO_SYNC=1` stops that; `UV_OFFLINE=1` means that if the pin ever
+stops working the job fails loudly instead of quietly rebuilding a slower
+environment.
+
+**2. cupy needs CUDA *headers*, which the host never had to declare.** cupy
+compiles kernels at runtime through NVRTC, which needs toolkit headers — not
+just the driver and the runtime libs bundled in the wheel. On irv-ml1 a CUDA
+toolkit is installed system-wide, so the bare `nohup` process found them by
+accident; a slim image has none. Every render died 1.7s in with
+
+```
+RuntimeError: Failed to find CUDA headers.
+```
+
+printed *through argparse's usage banner*, which makes it read like a CLI
+argument bug rather than a missing toolkit — that misdirection is the reason
+this is written down. Fixed with `uv pip install "cupy-cuda12x[ctk]"`, which
+pulls the headers as wheels: a few hundred MB against ~6 GB for a `-devel`
+base image. It runs *after* `uv sync`, because sync prunes what it does not
+know about.
+
+**3. The GPU index inside the container is not the host's.** The app pins
+`CUDA_DEVICE_ORDER=PCI_BUS_ID` and selects `CUDA_VISIBLE_DEVICES_TARGET`
+(default `1`, correct on the host). Compose exposes exactly one GPU
+(`device_ids: ["1"]`, the A6000 in Docker's ordering), so **inside** the
+container that card is index **0** — hence `CUDA_VISIBLE_DEVICES_TARGET=0`.
+Copying the host's value selects a device that does not exist. Device 0 on the
+host is the 3090, which hosts the TTS zoo and must not be touched.
+
+## Performance, measured on this host
+
+| job | wall |
+|---|---|
+| 256², animation, **cold container** | 23.3 s |
+| 256², animation, warm | 6.1 s |
+| 256², plate only (`--codec none`) | 3.9 s |
+| 512², plate only | 6.4 s |
+
+Warm numbers beat the 7.4 s at 256² recorded against the bare-metal process, so
+containerising cost nothing. The cold-vs-warm gap is **cupy's NVRTC compile**,
+which is why `/root/.cupy` is a volume: verified by recreating the container
+(fresh cache → 23.2 s first render) and then restarting it (populated cache →
+6.0 s). Without that volume every restart makes the next user wait 4x and the
+service merely looks slow.
+
+## Operational constraints — from waterland-dev, not inferred
+
+- **Serial by design. One replica, one card.** A render is 20–45 s of near-full
+  GPU and the app runs a single worker thread. Two of these on the same A6000
+  would OOM or thrash. Throughput is a conversation about hardware, not replica
+  count.
+- **No authentication, and it accepts arbitrary file uploads.** It must stay
+  inside the LAN / WireGuard boundary. Do **not** paper over this with a proxy
+  password — waterland-dev has offered to add a real auth layer if it ever
+  needs wider reach. Ask.
+- Job store is scratch output, not source-of-truth: ~12 MB per animated job,
+  self-evicting at 40 retained jobs (`RETAIN` in `studio/jobs.py`), so steady
+  state is bounded around 500 MB. The 4 jobs from the bare-metal instance were
+  copied in at cutover.
+- Internal render timeout is 480 s, which is why the healthcheck interval is
+  loose — an aggressive probe would measure queue depth rather than liveness.
+
+## ⚠️ `update.sh` needs a credential this host does not have
+
+The repo is not anonymously readable — an unauthenticated clone 403s. The
+initial checkout was made with the operator's `vh` site-admin token passed
+inline and **not persisted**: the on-disk remote is the plain URL and
+`.git/config` holds no token (verified). Consequently `git fetch` in
+`update.sh` will fail until the host has a credential of its own.
+
+`claude-bot` 404s on `vh/waterland`, so it currently lacks read access. The
+right fix is a read-only deploy token for this host, or granting `claude-bot`
+read on the repo — **not** writing the site-admin token to disk on a GPU box.
+Raised with waterland-dev; until then, re-run the authenticated clone by hand
+to update.