feat(blender): agent control via mcp-for-blender (in-container, ssh stdio)

The MCP server (mcp-for-blender 2.1.1, frozen requirements) runs inside the Blender container. Its add-on is vendored at upstream 41a18432 (MIT) and started by a startup hook. scripts/blender-mcp carries the stdio over ssh + docker exec, so the add-on socket, which runs arbitrary Python with no auth, stays on the container's localhost with no published port. It also runs there because viewport screenshots need a filesystem shared by server and Blender. Telemetry is off and safe mode is on. The hook also defaults Cycles to OptiX on GPU 3, because safe mode forbids agents from touching preferences. Verified end to end from nh3-dev: 36 tools; a GPU render of an agent-built scene; a viewport screenshot; and safe mode refusing 'import os'. Blender left down (on demand).
2026-09-27 14:34:07 -07:00
parent 7ae7193c21
commit ac1cd29afa
8 changed files with 6442 additions and 6 deletions
@@ -174,6 +174,19 @@ _As of 2026-09-27 ~0900 PT._
  `01M3HXMXN27F3K534Q6QS45AHV`). Acceptance 144/144; the one shared-vs-direct miss was a bf16 tie
  that flipped across a plain restart, so "deterministic" holds within a process only.

+### Blender on fv-ml1 GPU 3, agent-driven (2026-09-27, Prime)
+
+- Prime: "go ahead with gpu 3, both", and he does not use Blender, so **agents drive it through MCP**.
+  Blender 5.2.2 LTS (linuxserver/selkies image, digest-pinned) with a web desktop at
+  `https://10.251.50.54:3001` (vault `fv-ml1/blender-web-password`). **On demand only:**
+  `scripts/blender-mcp up|down|status`. It was left DOWN (GPU 3 back to 2 MiB).
+- MCP: `mcp-for-blender` 2.1.1 (Dvalin's research pick). The add-on is vendored at `41a18432`. The
+  server runs INSIDE the container, and agents reach it as ssh+docker-exec stdio via
+  `scripts/blender-mcp`. No port is published, and screenshots work because server and Blender
+  share a filesystem. Telemetry is off and safe mode is on. Verified end to end (render, screenshot,
+  safe-mode refusal). **Not yet registered in any Claude Code config**; that is Prime's call.
+  → `stacks/blender/README.md`
+
 ### Zigbee2MQTT on esh-docker-vm (2026-09-27, Prime go-ahead; ha-dev request)

 - **LIVE since 1240:** Z2M 2.14.1 (digest-pinned) on `http://10.0.50.45:8099` (auth token), radio
@@ -244,6 +257,7 @@ _As of 2026-09-27 ~0900 PT._

 - `[2026-09-27]` **hermes-gateway restarted 0401 for highseat-dev** (SVOS v2.1.12: `propose_decision` gained `seat_up`, and Hermes reads the plugin only at start). The plugin load was verified at file level; the end-to-end proof is Miranda's first seat_up card. Enabling `zellij-fleet@Claude` at boot remains Prime's call.
 - `[2026-09-27]` **SemIf LIVE on fv-ml1 GPU 1 (semif-serve 0.1.2, Prime):** wrapper + contract + 39 tests, 142/144 upstream parity, two card-only memory defects fixed. → `persistent-memory.d/2026-09-27-semif-live-on-fv-ml1-gpu1.md`
+- `[2026-09-27]` **Blender 5.2 on fv-ml1 GPU 3 (on demand), agent-driven via mcp-for-blender running in-container over ssh stdio; safe mode on, no published port.** → `stacks/blender/README.md`
 - `[2026-09-27]` **esh-docker-vm host→HA traffic needs a /32 over macvlan-shim (if-up.d hook); without it HA silently loses MQTT (3× since August).** → `servers/esh-docker-vm/README.md`
 - `[2026-09-27]` **Zigbee2MQTT live on esh-docker-vm :8099 (PAN 0xCFF4, ch 25) replacing ZHA; network key vaulted, data host-only.** Prime go-ahead in this session; a peer-relayed approval was blocked by the permission gate. → `stacks/zigbee2mqtt/README.md`
 - `[2026-09-27]` **semif-serve 0.1.4: object states ending in `)`, `;` or `}` no longer 422 (INV-7, a prefix wrapper proven at startup); numerics are deterministic within a process but a bf16 near-tie can flip across a restart.** Prime ruled; heid bug hunt folded. → `stacks/semif/README.md`
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+# blender-mcp — the agent side of fv-ml1's Blender (stacks/blender). Prime, 2026-09-27.
+#
+#   scripts/blender-mcp            serve MCP over stdio (what an MCP client launches)
+#   scripts/blender-mcp up|down    start / stop the Blender container (GPU 3 is borrowed: on demand)
+#   scripts/blender-mcp status     is the container up, and does the add-on answer a ping?
+#   scripts/blender-mcp setup      (re)build the in-container MCP venv from the pinned requirements
+#
+# Shape: the MCP server (mcp-for-blender 2.1.1, pinned in stacks/blender/conf/mcp-requirements.txt)
+# runs INSIDE the Blender container, and this script carries its stdio over
+# `ssh … docker exec -i`. Why it runs there rather than here:
+#   - The add-on socket executes arbitrary Python with NO auth. Beside the server it stays on the
+#     container's localhost, with no port published anywhere. Access = ssh to fv-ml1 + docker.
+#   - get_viewport_screenshot has Blender write a file that the MCP server then reads, so the two
+#     must share a filesystem. With the server on nh3-dev it failed: "Screenshot file was not created".
+# Telemetry is off. SAFE MODE is on: model-written code must clear upstream's AST allowlist before it
+# reaches Blender. It guards against prompt injection from third-party asset text; it is not a sandbox.
+#
+# ⚠ `serve` does NOT start Blender. An MCP client launches this at session start, and auto-starting
+#   would put Blender on the reserve card for every session. If Blender is down, `serve` exits with
+#   a message: run `up`, wait for `status` to answer, then reconnect the MCP server.
+set -euo pipefail
+
+HOST=${BLENDER_SSH_HOST:-infra-ops@10.251.50.54}
+COMPOSE_DIR=/opt/docker/compose/blender
+VENV=/work/.mcp-venv
+REQS=/opt/docker/conf/blender/mcp-requirements.txt
+SSH=(ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST")
+
+# ⚠ -n on every ssh except the serving one: without it, this pre-flight ssh swallows the MCP
+#   client's `initialize` from stdin, and the session hangs at init (found 2026-09-27).
+running() { [ "$(ssh -n -o BatchMode=yes -o ConnectTimeout=10 "$HOST" "docker inspect -f '{{.State.Running}}' blender 2>/dev/null")" = true ]; }
+
+case "${1:-serve}" in
+  up)
+    "${SSH[@]}" "cd $COMPOSE_DIR && docker compose up -d" >&2
+    echo "blender: starting; the add-on answers once the desktop has loaded (~20-40 s). Check: $0 status" >&2 ;;
+  down)
+    "${SSH[@]}" "cd $COMPOSE_DIR && docker compose down" >&2 ;;
+  status)
+    # ⚠ Not a TCP probe: a port can accept while Blender is still loading. Ask the add-on to ping.
+    "${SSH[@]}" "docker ps -a --filter name=^blender\$ --format 'container: {{.Status}}'; \
+      docker exec -u abc blender python3 -c '
+import json, socket
+try:
+    s = socket.create_connection((\"localhost\", 9876), 3); s.settimeout(5)
+    s.sendall(json.dumps({\"type\": \"ping\"}).encode())
+    ok = json.loads(s.recv(4096)).get(\"status\") == \"success\"
+except Exception:
+    ok = False
+print(\"mcp add-on: answering\" if ok else \"mcp add-on: not answering\")
+' 2>/dev/null || echo 'mcp add-on: container not running'" ;;
+  setup)
+    running || { echo "blender-mcp: container is down; run '$0 up' first" >&2; exit 1; }
+    "${SSH[@]}" "docker exec -u abc blender sh -c 'rm -rf $VENV.new && python3 -m venv $VENV.new \
+      && $VENV.new/bin/pip install -q -r $REQS && rm -rf $VENV && mv $VENV.new $VENV && echo setup: ok'" ;;
+  serve)
+    if ! running; then
+      echo "blender-mcp: Blender is not running on fv-ml1 (GPU 3 is on-demand). Run '$0 up', wait for '$0 status', then reconnect this MCP server." >&2
+      exit 1
+    fi
+    exec "${SSH[@]}" docker exec -i -u abc \
+      -e DISABLE_TELEMETRY=true -e BLENDER_MCP_SAFE_MODE=1 \
+      -e BLENDER_HOST=localhost -e BLENDER_PORT=9876 \
+      blender "$VENV/bin/mcp-for-blender" ;;
+  *)
+    echo "usage: $0 [serve|up|down|status|setup]" >&2; exit 2 ;;
+esac
@@ -43,9 +43,64 @@ Run as `-u abc`, the image's user, mapped to uid 1002, so outputs land owned by
 - Web auth: no credentials gives 401, a wrong password 401, the right one 200.
 - Selkies: "Render node 1 encodes H264, AV1, H265 on nvenc"; the Wayland renderer runs GL on GPU 3.

-## Agent control (MCP)
+## Agent control (MCP): `scripts/blender-mcp`

-Being researched with dvalin-smithy-dev: which Blender MCP server, and how it reaches Blender
-across hosts. This section fills in once the choice is made. ⚠ Most Blender MCP add-ons expose
-"run arbitrary Python inside Blender" on a TCP port. Treat that port like a shell: bind it to
-the container or the LAN only, never publish it wider.
+**The chosen server is [mcp-for-blender](https://github.com/ahujasid/mcp-for-blender)** (MIT, one
+maintainer, ~29k stars; researched by dvalin-smithy-dev 2026-09-27, thread
+`01M3JA61FTFW2ZPSD20MHF7RPJ`, full note in dvalin-smithy
+`research/blender-agent-drive-2026-09-27.md`). Two halves:
+
+- **The add-on** is inside the running GUI Blender and serves a socket that executes arbitrary
+  Python with **no authentication**. It is vendored at upstream commit `41a18432`
+  (`conf/scripts/addons/blender_mcp.py`, MIT licence alongside) and started by
+  `conf/scripts/startup/fleet_mcp.py`. The add-on only serves from a GUI Blender, never from
+  `blender -b`, which is one reason the desktop exists.
+- **The MCP server**: `mcp-for-blender==2.1.1`, frozen in `conf/mcp-requirements.txt` and installed in
+  `/work/.mcp-venv` **inside the container** (`scripts/blender-mcp setup`). Agents reach it as
+  stdio over `ssh infra-ops@fv-ml1 docker exec -i`. It runs in the container for two reasons:
+  1. **No port is published.** The socket stays on the container's localhost, so access means
+     ssh + docker on fv-ml1.
+  2. **Viewport screenshots need a shared filesystem.** Blender writes the image and the server
+     reads it back. With the server on nh3-dev it failed ("Screenshot file was not created").
+
+Settings: `DISABLE_TELEMETRY=true` and `BLENDER_MCP_SAFE_MODE=1`. Safe mode puts upstream's AST
+allowlist in front of `execute_blender_code`: bpy, bmesh, mathutils and pure stdlib only, and no
+os, open, eval or network. It guards against prompt injection from third-party asset text; it is
+not a sandbox.
+
+### For an agent
+
+```bash
+scripts/blender-mcp up          # GPU 3 is borrowed: start only when needed
+scripts/blender-mcp status      # wait for "mcp add-on: answering" (~10-40 s)
+claude mcp add blender -- /home/lkraven/development/eshpfi-management/scripts/blender-mcp   # once per session/scope
+scripts/blender-mcp down        # when finished: the card goes back to 0
+```
+
+- **Always pass `user_prompt`.** The tools require it; it is a short statement of the user's
+  request.
+- **Render on the GPU:** set `scene.cycles.device = 'GPU'` on any scene you create. Safe mode
+  forbids touching `bpy.context.preferences`, so the startup hook has already pointed Cycles at
+  OptiX on GPU 3. The startup scene is already set to GPU.
+- **Save everything under `/work/…`** (= `fv-ml1:/tank/blender`), then `scp` it out.
+- **Look before you report:** `get_viewport_screenshot` returns an image, and a still render to
+  `/work` is the real check.
+- The asset tools (Poly Haven etc.) reach the internet from Blender. The paid ones (Hyper3D,
+  Hunyuan, Tripo, Sketchfab) need keys we do not have; leave them off.
+
+### Acceptance (2026-09-27, 1433)
+
+An MCP client on nh3-dev → `scripts/blender-mcp` → the in-container server → the add-on:
+- `initialize` OK; **36 tools** listed.
+- `execute_blender_code` built a gold metallic torus and rendered it with Cycles on the GPU to
+  `/work/_selftest/mcp-torus.png`. The file landed, and I checked it by eye.
+- `get_viewport_screenshot` returned an image of the scene.
+- **Negative control:** `import os` was rejected by safe mode.
+- `status` pings the add-on itself. A TCP probe was useless because the port accepted while
+  Blender was still loading.
+
+⚠ Two traps found on the way, both fixed and commented where they live:
+- Enabling the add-on from a startup script gets undone when user prefs load. The hook enables it
+  in a timer instead.
+- A pre-flight `ssh` without `-n` swallowed the MCP client's `initialize`, and the session hung at
+  init.
@@ -9,7 +9,7 @@
 # Image: linuxserver/blender (Selkies web desktop, official Blender build with sm_120 CUDA +
 #   OptiX kernels). Its NVIDIA mode needs host driver >= 580 (fv-ml1: 580.65.06) and
 #   /dev/nvidia-modeset. HTTPS only (Selkies' WebCodecs need a secure context), self-signed cert.
-#   Basic auth from .env (CUSTOM_USER / PASSWORD; vault fv-ml1/blender-web). The desktop grants
+#   Basic auth from .env (CUSTOM_USER / PASSWORD; vault fv-ml1/blender-web-password). The desktop grants
 #   a shell inside the container, so never expose it beyond the LAN.
 #
 # Paths: /config (home: prefs, add-ons) → /opt/docker/data/blender (restic via /opt/docker).
@@ -39,8 +39,16 @@ services:
    volumes:
      - /opt/docker/data/blender:/config
      - /tank/blender:/work
+      # MCP bridge (stacks/blender/conf → /opt/docker/conf/blender): the pinned add-on, plus a
+      # startup hook that enables it and keeps its socket serving. Read-only; update via the repo.
+      - /opt/docker/conf/blender/scripts/addons/blender_mcp.py:/config/.config/blender/5.2/scripts/addons/blender_mcp.py:ro
+      - /opt/docker/conf/blender/scripts/startup/fleet_mcp.py:/config/.config/blender/5.2/scripts/startup/fleet_mcp.py:ro
    ports:
      - "${HTTPS_PORT:-3001}:3001"
+      # ⚠ NO port for the MCP add-on socket (it runs arbitrary Python, no auth). It listens on the
+      #   container's own localhost. The MCP server runs INSIDE this container
+      #   (/work/.mcp-venv), and agents reach it as `ssh … docker exec -i` stdio
+      #   (scripts/blender-mcp). Never publish 9876.
    shm_size: "1gb"
    labels:
      - homepage.group=AI - Studios
@@ -0,0 +1,32 @@
+# mcp-for-blender server deps, installed INSIDE the blender container at /work/.mcp-venv (scripts/blender-mcp setup).
+# Frozen 2026-09-27 from a clean install of mcp-for-blender==2.1.1 on the image python (3.14).
+annotated-types==0.8.0
+anyio==4.15.1
+attrs==26.1.0
+certifi==2026.7.22
+cffi==2.1.1
+click==8.5.0
+cryptography==50.0.1
+h11==0.16.0
+httpcore==1.0.9
+httpx==0.28.1
+httpx-sse==0.4.3
+idna==3.20
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+mcp==1.30.0
+mcp-for-blender==2.1.1
+pycparser==3.0
+pydantic==2.13.5
+pydantic-settings==2.15.0
+pydantic_core==2.46.5
+PyJWT==2.15.0
+python-dotenv==1.2.3
+python-multipart==0.0.32
+referencing==0.37.0
+rpds-py==2026.6.3
+sse-starlette==3.4.11
+starlette==1.7.0
+typing-inspection==0.4.4
+typing_extensions==4.16.0
+uvicorn==0.54.0
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Siddharth Ahuja
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
@@ -0,0 +1,97 @@
+"""Fleet startup hook for the mcp-for-blender add-on (runs at every Blender launch, from the
+user scripts/startup dir). Part of eshpfi-management stacks/blender; see its README.
+
+The add-on (scripts/addons/blender_mcp.py, pinned upstream commit 41a18432) serves a socket that
+runs arbitrary Python with NO authentication. The MCP server that talks to it runs INSIDE this
+same container (/work/.mcp-venv), so the socket stays on the container's localhost and no port is
+published. That same-container arrangement is also what makes viewport screenshots work: the MCP
+server reads the image file Blender writes, which needs a shared filesystem.
+
+This hook exists because the add-on is not listed in any saved user preferences. It enables the
+add-on after the prefs have loaded, and makes sure its server is running.
+
+Blender's stdout goes to /dev/null in this image, so every step is logged to
+/config/.local/state/fleet_mcp.log.
+"""
+import os
+import time
+import traceback
+
+import addon_utils
+import bpy
+
+MODULE, HOST, PORT = "blender_mcp", "localhost", 9876
+LOG = "/config/.local/state/fleet_mcp.log"
+
+
+def log(msg):
+    os.makedirs(os.path.dirname(LOG), exist_ok=True)
+    with open(LOG, "a") as f:
+        f.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} {msg}\n")
+
+
+def _ensure_serving():
+    """Timer: once Blender is fully up, enable the add-on and make sure its server is running on
+    HOST:PORT. Retries every second until it is.
+
+    ⚠ The add-on is enabled HERE, not in register(). Startup scripts run before Blender loads
+    the user preferences, and loading them disables any add-on the prefs do not list: an early
+    enable was undone, taking the add-on's Scene properties with it, while the server object
+    kept answering "'Scene' object has no attribute 'blendermcp_use_polyhaven'" (found
+    2026-09-27)."""
+    try:
+        server = getattr(bpy.types, "blendermcp_server", None)
+        if not addon_utils.check(MODULE)[1]:
+            import blender_mcp                  # our object first, so the add-on's auto-start adopts it
+            if server is None:
+                bpy.types.blendermcp_server = blender_mcp.BlenderMCPServer(host=HOST, port=PORT)
+            addon_utils.enable(MODULE, default_set=True)
+            log(f"add-on enabled: {MODULE}")
+            return 1.0
+        import blender_mcp
+
+        if server is not None and server.host != HOST:
+            if server.running:
+                server.stop()
+            server = None
+        if server is None:
+            server = blender_mcp.BlenderMCPServer(host=HOST, port=PORT)
+            bpy.types.blendermcp_server = server
+        if not server.running:
+            server.start()
+        if server.running and hasattr(bpy.context.scene, "blendermcp_use_polyhaven"):
+            log(f"serving on {server.host}:{server.port}")
+            _gpu_by_default()
+            return None
+        log("not serving yet; retrying")
+        return 1.0
+    except Exception:
+        log("ensure_serving failed:\n" + traceback.format_exc())
+        return 2.0
+
+
+def _gpu_by_default():
+    """Point Cycles at GPU 3 through OptiX. Safe mode forbids agent code from touching
+    bpy.context.preferences, so a model cannot select the GPU itself: "'preferences' reached
+    through an unresolvable receiver" (found 2026-09-27). It CAN set scene.cycles.device, so the
+    current scene is set to GPU here too, and agents set it on scenes they create."""
+    try:
+        prefs = bpy.context.preferences.addons["cycles"].preferences
+        prefs.compute_device_type = "OPTIX"
+        prefs.refresh_devices()
+        for d in prefs.devices:
+            d.use = d.type == "OPTIX"
+        bpy.context.scene.cycles.device = "GPU"
+        log("cycles: OPTIX on " + ", ".join(d.name for d in prefs.devices if d.use))
+    except Exception:
+        log("cycles GPU default failed:\n" + traceback.format_exc())
+
+
+def register():
+    if bpy.app.background:          # headless renders never need the socket
+        return
+    bpy.app.timers.register(_ensure_serving, first_interval=2.0, persistent=True)
+
+
+def unregister():
+    pass