From d0f68a3b18b4206ee546678902291a8e0a99ef72 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 28 Sep 2026 08:39:43 -0700 Subject: [PATCH] feat(blender): blender-run one-shot headless wrapper + FLEETTOOLS entry scripts/blender-run launches each call as a docker run --rm of the Blender image on fv-ml1 GPU 3, capped at 64g / 48 CPUs. It needs no desktop and does not affect the GUI container's lifecycle. --job DIR stages a local directory to /tank/blender/jobs//, runs Blender with that as the cwd, and copies results back. It always passes --python-exit-code 1, because Blender otherwise exits 0 when a --python script raises (measured). Tested headless: Cycles GPU and CPU, EEVEE via EGL, Workbench, an STL round-trip, and exit codes (3, 7 and 1 pass through). There is no STEP importer. Written for draupnir's design work, and indexed in FLEETTOOLS with a detail file. --- docs/fleettools/FLEETTOOLS.md | 6 ++++ docs/fleettools/blender.md | 56 +++++++++++++++++++++++++++++ scripts/blender-run | 66 +++++++++++++++++++++++++++++++++++ stacks/blender/README.md | 10 +++++- 4 files changed, 137 insertions(+), 1 deletion(-) create mode 100644 docs/fleettools/blender.md create mode 100755 scripts/blender-run diff --git a/docs/fleettools/FLEETTOOLS.md b/docs/fleettools/FLEETTOOLS.md index 57eb79e..8bd085a 100644 --- a/docs/fleettools/FLEETTOOLS.md +++ b/docs/fleettools/FLEETTOOLS.md @@ -76,6 +76,12 @@ live contract for its API. Fetch it rather than trusting a transcription. *When:* you need images rendered, or a character LoRA trained. *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/arbo.md` +- **Blender** — Blender 5.2 LTS on fv-ml1 GPU 3 (borrowed, on demand). `scripts/blender-run` for + headless one-shot renders and conversions; `scripts/blender-mcp` for agent-driven scenes + (register per task). + *When:* 3D rendering, STL → image, scene building. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/blender.md` + ## Working on the fleet itself - **elway** — SSH playbook runner for **CHANGING** things. diff --git a/docs/fleettools/blender.md b/docs/fleettools/blender.md new file mode 100644 index 0000000..fdb1956 --- /dev/null +++ b/docs/fleettools/blender.md @@ -0,0 +1,56 @@ +# Blender: headless and agent-driven 3D on fv-ml1 GPU 3 + +**Blender 5.2.2 LTS** (bundled Python 3.13), runs on **fv-ml1 GPU 3** (RTX PRO 6000 Blackwell, +96 GB). Stack and full notes: `/home/lkraven/development/eshpfi-management/stacks/blender/README.md`. + +⚠ **GPU 3 is borrowed.** It is the fleet's reserve card for a full-size vLLM seat. Blender +runs only while in use, and this access ends if a big seat moves onto the card. + +## Two ways in + +| You are… | Use | Shape | +|---|---|---| +| a script or CLI caller (renders, conversions) | `scripts/blender-run` | one-shot `docker run --rm`: no desktop, gone when Blender exits | +| an agent building scenes interactively | `scripts/blender-mcp` (MCP, per task) | a GUI Blender plus the mcp-for-blender add-on; `up` / `status` / `down` | + +Both scripts are in `/home/lkraven/development/eshpfi-management/scripts/` and run from nh3-dev, +reaching fv-ml1 as `infra-ops@10.251.50.54` over ssh. **No HTTP API** and no openapi.json. + +## blender-run (headless) + +```sh +scripts/blender-run --job /path/to/jobdir -- --python render.py -- out.png +scripts/blender-run -- --python-expr 'import bpy; print(bpy.app.version_string)' +``` + +- Always adds `-b --factory-startup --python-exit-code 1`. Arguments go after `--`. +- **Files:** fv-ml1 does not mount `/mnt/smithy`. `--job DIR` copies DIR to + `fv-ml1:/tank/blender/jobs//`, runs with that as the working directory, and copies + new or changed files back into DIR. Nothing is deleted on either side. Use relative + input paths inside the job. +- **Engines headless (tested 2026-09-28):** Cycles on GPU (OptiX/CUDA), Cycles on CPU, EEVEE + (EGL, no display needed, ~9 s with the shader compile on first use), Workbench. In 5.2 the + EEVEE id is `BLENDER_EEVEE` (`BLENDER_EEVEE_NEXT` is gone). For Cycles GPU, set + `prefs.compute_device_type = 'OPTIX'` and enable the OPTIX devices, then + `scene.cycles.device = 'GPU'`. Factory startup defaults to CPU. +- **Import:** STL is built in (`bpy.ops.wm.stl_import`). **No STEP importer** is installed or + built in. +- **Budget:** each run is capped at 64 GB RAM and 48 CPUs, with up to the whole 96 GB of VRAM. + Keep to about 2 concurrent renders. It is not on irv-ml1, so irv-ml1's working-set budget + does not apply. + +⚠ **Foot-guns** (all measured): +- A `--python` script that raises exits **0** unless `--python-exit-code` is set. blender-run + sets it. +- `render.filepath` must be **absolute**. Blender does not resolve a relative output path + against the working directory ("cannot save 'out.png'"). Importers and Python file I/O do. +- "HIPEW initialization failed" on stderr is harmless: it is the AMD backend probing. +- The first OptiX render in a process includes about 1-2 s of kernel load. + +## blender-mcp (agents) + +Register it **per task**, never user- or project-wide (Prime, 2026-09-27): +`claude mcp add blender -- /home/lkraven/development/eshpfi-management/scripts/blender-mcp`. +Then run `scripts/blender-mcp up`, wait for `status` to say answering, work, and run `down` +when finished. Safe mode is on (no os/open/network in agent code), so the startup hook has +already pointed Cycles at OptiX. Save to `/work/…`. Details and traps are in the stack README. diff --git a/scripts/blender-run b/scripts/blender-run new file mode 100755 index 0000000..7759f50 --- /dev/null +++ b/scripts/blender-run @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# blender-run — one-shot HEADLESS Blender on fv-ml1 GPU 3, for scripted/CLI callers (draupnir etc.). +# The agent-driven, interactive path is scripts/blender-mcp; this is the batch path. +# +# scripts/blender-run [--job DIR] -- +# scripts/blender-run --job /mnt/smithy/draupnir/j42 -- --python render.py -- --out out.png +# scripts/blender-run -- --python-expr 'import bpy; print(bpy.app.version_string)' +# +# Each call is its own `docker run --rm` of the stacks/blender image (Blender 5.2.2 LTS, Python +# 3.13): no desktop, no MCP socket, gone when Blender exits. So it never collides with the on-demand +# GUI container, and GPU 3 goes back to 0 when the render ends. Always added: `-b --factory-startup +# --python-exit-code 1`. Without that last flag, Blender exits 0 even when a --python script raises +# (measured). +# +# ⚠ Give render.filepath an ABSOLUTE path (os.path.abspath). Blender does not resolve a relative +# output path against the working directory: "cannot save 'out.png'". Python file I/O and +# importers do use the cwd. +# +# Files: fv-ml1 does NOT mount /mnt/smithy. With --job DIR, DIR is copied to +# fv-ml1:/tank/blender/jobs//, Blender runs WITH THAT AS ITS WORKING DIRECTORY, and new or +# changed files are copied back into DIR afterwards (nothing is deleted on either side). Use +# relative paths inside the job. Without --job, the only paths Blender sees are under +# /work (= fv-ml1:/tank/blender). +# +# Budget: GPU 3 (96 GB, borrowed from the vLLM reserve: this ends if a full-size seat moves in). +# The container is capped at 64 GB RAM / 48 CPUs so a runaway render cannot starve the inference +# seats on the same host. +set -euo pipefail + +HOST=${BLENDER_SSH_HOST:-infra-ops@10.251.50.54} +ENV_FILE=/opt/docker/compose/blender/.env +JOB="" +while [ $# -gt 0 ]; do + case "$1" in + --job) JOB=${2:?--job needs a directory}; shift 2 ;; + --) shift; break ;; + -h|--help) sed -n 2,23p "$0"; exit 0 ;; + *) echo "blender-run: unknown option $1 (blender args go after --)" >&2; exit 2 ;; + esac +done + +WORKDIR=/work +if [ -n "$JOB" ]; then + [ -d "$JOB" ] || { echo "blender-run: --job $JOB is not a directory" >&2; exit 2; } + NAME=$(basename "$(realpath "$JOB")") + [[ $NAME =~ ^[A-Za-z0-9._-]+$ ]] || { echo "blender-run: job dir name '$NAME' must be [A-Za-z0-9._-]" >&2; exit 2; } + ssh -n -o BatchMode=yes "$HOST" "mkdir -p /tank/blender/jobs/$NAME" + rsync -a "$JOB"/ "$HOST:/tank/blender/jobs/$NAME/" + WORKDIR=/work/jobs/$NAME +fi + +# Arguments travel as one shell-quoted string: ssh flattens argv into a remote command line. +ARGS=$(printf '%q ' "$@") +set +e +ssh -n -o BatchMode=yes "$HOST" "IMG=\$(grep '^IMAGE=' $ENV_FILE | cut -d= -f2) && \ + exec docker run --rm --name blender-run-\$\$ --runtime nvidia \ + -e NVIDIA_VISIBLE_DEVICES=3 -e NVIDIA_DRIVER_CAPABILITIES=all \ + --user 1002:1003 -e HOME=/tmp --memory 64g --cpus 48 \ + -v /tank/blender:/work -w $WORKDIR --entrypoint /blender/blender \"\$IMG\" \ + -b --factory-startup --python-exit-code 1 $ARGS" +RC=$? +set -e +if [ -n "$JOB" ]; then + rsync -a --update "$HOST:/tank/blender/jobs/$NAME/" "$JOB"/ +fi +exit $RC diff --git a/stacks/blender/README.md b/stacks/blender/README.md index d6a44c8..2439867 100644 --- a/stacks/blender/README.md +++ b/stacks/blender/README.md @@ -24,7 +24,15 @@ ssh infra-ops@10.251.50.54 'cd /opt/docker/compose/blender && docker compose dow `restart: "no"`, so a reboot never brings it back. **When a big seat moves onto GPU 3, Blender stays down.** -## Headless rendering (no desktop needed) +## Headless rendering: `scripts/blender-run` (for scripts and CLI callers) + +A one-shot `docker run --rm` of this image with Blender as the entrypoint. It needs no desktop and +does not collide with the GUI container's up/down. `--job DIR` stages a local dir to +`/tank/blender/jobs//` and copies results back. Engines tested headless on 2026-09-28: +Cycles GPU and CPU, EEVEE (EGL), Workbench. STL import is built in; there is **no STEP importer**. +Foot-guns and budget are in `docs/fleettools/blender.md`. First consumer: draupnir. + +## Headless rendering inside the running GUI container ```bash ssh infra-ops@10.251.50.54 'docker exec -u abc blender blender -b /work/.blend -E CYCLES -o /work/out/frame_#### -a -- --cycles-device OPTIX'