feat(blender): agent control via mcp-for-blender (in-container, ssh stdio)
The MCP server (mcp-for-blender 2.1.1, frozen requirements) runs inside the Blender container. Its add-on is vendored at upstream 41a18432 (MIT) and started by a startup hook. scripts/blender-mcp carries the stdio over ssh + docker exec, so the add-on socket, which runs arbitrary Python with no auth, stays on the container's localhost with no published port. It also runs there because viewport screenshots need a filesystem shared by server and Blender. Telemetry is off and safe mode is on. The hook also defaults Cycles to OptiX on GPU 3, because safe mode forbids agents from touching preferences. Verified end to end from nh3-dev: 36 tools; a GPU render of an agent-built scene; a viewport screenshot; and safe mode refusing 'import os'. Blender left down (on demand).
This commit is contained in:
@@ -43,9 +43,64 @@ Run as `-u abc`, the image's user, mapped to uid 1002, so outputs land owned by
|
||||
- Web auth: no credentials gives 401, a wrong password 401, the right one 200.
|
||||
- Selkies: "Render node 1 encodes H264, AV1, H265 on nvenc"; the Wayland renderer runs GL on GPU 3.
|
||||
|
||||
## Agent control (MCP)
|
||||
## Agent control (MCP): `scripts/blender-mcp`
|
||||
|
||||
Being researched with dvalin-smithy-dev: which Blender MCP server, and how it reaches Blender
|
||||
across hosts. This section fills in once the choice is made. ⚠ Most Blender MCP add-ons expose
|
||||
"run arbitrary Python inside Blender" on a TCP port. Treat that port like a shell: bind it to
|
||||
the container or the LAN only, never publish it wider.
|
||||
**The chosen server is [mcp-for-blender](https://github.com/ahujasid/mcp-for-blender)** (MIT, one
|
||||
maintainer, ~29k stars; researched by dvalin-smithy-dev 2026-09-27, thread
|
||||
`01M3JA61FTFW2ZPSD20MHF7RPJ`, full note in dvalin-smithy
|
||||
`research/blender-agent-drive-2026-09-27.md`). Two halves:
|
||||
|
||||
- **The add-on** is inside the running GUI Blender and serves a socket that executes arbitrary
|
||||
Python with **no authentication**. It is vendored at upstream commit `41a18432`
|
||||
(`conf/scripts/addons/blender_mcp.py`, MIT licence alongside) and started by
|
||||
`conf/scripts/startup/fleet_mcp.py`. The add-on only serves from a GUI Blender, never from
|
||||
`blender -b`, which is one reason the desktop exists.
|
||||
- **The MCP server**: `mcp-for-blender==2.1.1`, frozen in `conf/mcp-requirements.txt` and installed in
|
||||
`/work/.mcp-venv` **inside the container** (`scripts/blender-mcp setup`). Agents reach it as
|
||||
stdio over `ssh infra-ops@fv-ml1 docker exec -i`. It runs in the container for two reasons:
|
||||
1. **No port is published.** The socket stays on the container's localhost, so access means
|
||||
ssh + docker on fv-ml1.
|
||||
2. **Viewport screenshots need a shared filesystem.** Blender writes the image and the server
|
||||
reads it back. With the server on nh3-dev it failed ("Screenshot file was not created").
|
||||
|
||||
Settings: `DISABLE_TELEMETRY=true` and `BLENDER_MCP_SAFE_MODE=1`. Safe mode puts upstream's AST
|
||||
allowlist in front of `execute_blender_code`: bpy, bmesh, mathutils and pure stdlib only, and no
|
||||
os, open, eval or network. It guards against prompt injection from third-party asset text; it is
|
||||
not a sandbox.
|
||||
|
||||
### For an agent
|
||||
|
||||
```bash
|
||||
scripts/blender-mcp up # GPU 3 is borrowed: start only when needed
|
||||
scripts/blender-mcp status # wait for "mcp add-on: answering" (~10-40 s)
|
||||
claude mcp add blender -- /home/lkraven/development/eshpfi-management/scripts/blender-mcp # once per session/scope
|
||||
scripts/blender-mcp down # when finished: the card goes back to 0
|
||||
```
|
||||
|
||||
- **Always pass `user_prompt`.** The tools require it; it is a short statement of the user's
|
||||
request.
|
||||
- **Render on the GPU:** set `scene.cycles.device = 'GPU'` on any scene you create. Safe mode
|
||||
forbids touching `bpy.context.preferences`, so the startup hook has already pointed Cycles at
|
||||
OptiX on GPU 3. The startup scene is already set to GPU.
|
||||
- **Save everything under `/work/…`** (= `fv-ml1:/tank/blender`), then `scp` it out.
|
||||
- **Look before you report:** `get_viewport_screenshot` returns an image, and a still render to
|
||||
`/work` is the real check.
|
||||
- The asset tools (Poly Haven etc.) reach the internet from Blender. The paid ones (Hyper3D,
|
||||
Hunyuan, Tripo, Sketchfab) need keys we do not have; leave them off.
|
||||
|
||||
### Acceptance (2026-09-27, 1433)
|
||||
|
||||
An MCP client on nh3-dev → `scripts/blender-mcp` → the in-container server → the add-on:
|
||||
- `initialize` OK; **36 tools** listed.
|
||||
- `execute_blender_code` built a gold metallic torus and rendered it with Cycles on the GPU to
|
||||
`/work/_selftest/mcp-torus.png`. The file landed, and I checked it by eye.
|
||||
- `get_viewport_screenshot` returned an image of the scene.
|
||||
- **Negative control:** `import os` was rejected by safe mode.
|
||||
- `status` pings the add-on itself. A TCP probe was useless because the port accepted while
|
||||
Blender was still loading.
|
||||
|
||||
⚠ Two traps found on the way, both fixed and commented where they live:
|
||||
- Enabling the add-on from a startup script gets undone when user prefs load. The hook enables it
|
||||
in a timer instead.
|
||||
- A pre-flight `ssh` without `-n` swallowed the MCP client's `initialize`, and the session hung at
|
||||
init.
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
# Image: linuxserver/blender (Selkies web desktop, official Blender build with sm_120 CUDA +
|
||||
# OptiX kernels). Its NVIDIA mode needs host driver >= 580 (fv-ml1: 580.65.06) and
|
||||
# /dev/nvidia-modeset. HTTPS only (Selkies' WebCodecs need a secure context), self-signed cert.
|
||||
# Basic auth from .env (CUSTOM_USER / PASSWORD; vault fv-ml1/blender-web). The desktop grants
|
||||
# Basic auth from .env (CUSTOM_USER / PASSWORD; vault fv-ml1/blender-web-password). The desktop grants
|
||||
# a shell inside the container, so never expose it beyond the LAN.
|
||||
#
|
||||
# Paths: /config (home: prefs, add-ons) → /opt/docker/data/blender (restic via /opt/docker).
|
||||
@@ -39,8 +39,16 @@ services:
|
||||
volumes:
|
||||
- /opt/docker/data/blender:/config
|
||||
- /tank/blender:/work
|
||||
# MCP bridge (stacks/blender/conf → /opt/docker/conf/blender): the pinned add-on, plus a
|
||||
# startup hook that enables it and keeps its socket serving. Read-only; update via the repo.
|
||||
- /opt/docker/conf/blender/scripts/addons/blender_mcp.py:/config/.config/blender/5.2/scripts/addons/blender_mcp.py:ro
|
||||
- /opt/docker/conf/blender/scripts/startup/fleet_mcp.py:/config/.config/blender/5.2/scripts/startup/fleet_mcp.py:ro
|
||||
ports:
|
||||
- "${HTTPS_PORT:-3001}:3001"
|
||||
# ⚠ NO port for the MCP add-on socket (it runs arbitrary Python, no auth). It listens on the
|
||||
# container's own localhost. The MCP server runs INSIDE this container
|
||||
# (/work/.mcp-venv), and agents reach it as `ssh … docker exec -i` stdio
|
||||
# (scripts/blender-mcp). Never publish 9876.
|
||||
shm_size: "1gb"
|
||||
labels:
|
||||
- homepage.group=AI - Studios
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# mcp-for-blender server deps, installed INSIDE the blender container at /work/.mcp-venv (scripts/blender-mcp setup).
|
||||
# Frozen 2026-09-27 from a clean install of mcp-for-blender==2.1.1 on the image python (3.14).
|
||||
annotated-types==0.8.0
|
||||
anyio==4.15.1
|
||||
attrs==26.1.0
|
||||
certifi==2026.7.22
|
||||
cffi==2.1.1
|
||||
click==8.5.0
|
||||
cryptography==50.0.1
|
||||
h11==0.16.0
|
||||
httpcore==1.0.9
|
||||
httpx==0.28.1
|
||||
httpx-sse==0.4.3
|
||||
idna==3.20
|
||||
jsonschema==4.26.0
|
||||
jsonschema-specifications==2025.9.1
|
||||
mcp==1.30.0
|
||||
mcp-for-blender==2.1.1
|
||||
pycparser==3.0
|
||||
pydantic==2.13.5
|
||||
pydantic-settings==2.15.0
|
||||
pydantic_core==2.46.5
|
||||
PyJWT==2.15.0
|
||||
python-dotenv==1.2.3
|
||||
python-multipart==0.0.32
|
||||
referencing==0.37.0
|
||||
rpds-py==2026.6.3
|
||||
sse-starlette==3.4.11
|
||||
starlette==1.7.0
|
||||
typing-inspection==0.4.4
|
||||
typing_extensions==4.16.0
|
||||
uvicorn==0.54.0
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Siddharth Ahuja
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,97 @@
|
||||
"""Fleet startup hook for the mcp-for-blender add-on (runs at every Blender launch, from the
|
||||
user scripts/startup dir). Part of eshpfi-management stacks/blender; see its README.
|
||||
|
||||
The add-on (scripts/addons/blender_mcp.py, pinned upstream commit 41a18432) serves a socket that
|
||||
runs arbitrary Python with NO authentication. The MCP server that talks to it runs INSIDE this
|
||||
same container (/work/.mcp-venv), so the socket stays on the container's localhost and no port is
|
||||
published. That same-container arrangement is also what makes viewport screenshots work: the MCP
|
||||
server reads the image file Blender writes, which needs a shared filesystem.
|
||||
|
||||
This hook exists because the add-on is not listed in any saved user preferences. It enables the
|
||||
add-on after the prefs have loaded, and makes sure its server is running.
|
||||
|
||||
Blender's stdout goes to /dev/null in this image, so every step is logged to
|
||||
/config/.local/state/fleet_mcp.log.
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
import traceback
|
||||
|
||||
import addon_utils
|
||||
import bpy
|
||||
|
||||
MODULE, HOST, PORT = "blender_mcp", "localhost", 9876
|
||||
LOG = "/config/.local/state/fleet_mcp.log"
|
||||
|
||||
|
||||
def log(msg):
|
||||
os.makedirs(os.path.dirname(LOG), exist_ok=True)
|
||||
with open(LOG, "a") as f:
|
||||
f.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} {msg}\n")
|
||||
|
||||
|
||||
def _ensure_serving():
|
||||
"""Timer: once Blender is fully up, enable the add-on and make sure its server is running on
|
||||
HOST:PORT. Retries every second until it is.
|
||||
|
||||
⚠ The add-on is enabled HERE, not in register(). Startup scripts run before Blender loads
|
||||
the user preferences, and loading them disables any add-on the prefs do not list: an early
|
||||
enable was undone, taking the add-on's Scene properties with it, while the server object
|
||||
kept answering "'Scene' object has no attribute 'blendermcp_use_polyhaven'" (found
|
||||
2026-09-27)."""
|
||||
try:
|
||||
server = getattr(bpy.types, "blendermcp_server", None)
|
||||
if not addon_utils.check(MODULE)[1]:
|
||||
import blender_mcp # our object first, so the add-on's auto-start adopts it
|
||||
if server is None:
|
||||
bpy.types.blendermcp_server = blender_mcp.BlenderMCPServer(host=HOST, port=PORT)
|
||||
addon_utils.enable(MODULE, default_set=True)
|
||||
log(f"add-on enabled: {MODULE}")
|
||||
return 1.0
|
||||
import blender_mcp
|
||||
|
||||
if server is not None and server.host != HOST:
|
||||
if server.running:
|
||||
server.stop()
|
||||
server = None
|
||||
if server is None:
|
||||
server = blender_mcp.BlenderMCPServer(host=HOST, port=PORT)
|
||||
bpy.types.blendermcp_server = server
|
||||
if not server.running:
|
||||
server.start()
|
||||
if server.running and hasattr(bpy.context.scene, "blendermcp_use_polyhaven"):
|
||||
log(f"serving on {server.host}:{server.port}")
|
||||
_gpu_by_default()
|
||||
return None
|
||||
log("not serving yet; retrying")
|
||||
return 1.0
|
||||
except Exception:
|
||||
log("ensure_serving failed:\n" + traceback.format_exc())
|
||||
return 2.0
|
||||
|
||||
|
||||
def _gpu_by_default():
|
||||
"""Point Cycles at GPU 3 through OptiX. Safe mode forbids agent code from touching
|
||||
bpy.context.preferences, so a model cannot select the GPU itself: "'preferences' reached
|
||||
through an unresolvable receiver" (found 2026-09-27). It CAN set scene.cycles.device, so the
|
||||
current scene is set to GPU here too, and agents set it on scenes they create."""
|
||||
try:
|
||||
prefs = bpy.context.preferences.addons["cycles"].preferences
|
||||
prefs.compute_device_type = "OPTIX"
|
||||
prefs.refresh_devices()
|
||||
for d in prefs.devices:
|
||||
d.use = d.type == "OPTIX"
|
||||
bpy.context.scene.cycles.device = "GPU"
|
||||
log("cycles: OPTIX on " + ", ".join(d.name for d in prefs.devices if d.use))
|
||||
except Exception:
|
||||
log("cycles GPU default failed:\n" + traceback.format_exc())
|
||||
|
||||
|
||||
def register():
|
||||
if bpy.app.background: # headless renders never need the socket
|
||||
return
|
||||
bpy.app.timers.register(_ensure_serving, first_interval=2.0, persistent=True)
|
||||
|
||||
|
||||
def unregister():
|
||||
pass
|
||||
Reference in New Issue
Block a user