feat(blender): agent control via mcp-for-blender (in-container, ssh stdio)
The MCP server (mcp-for-blender 2.1.1, frozen requirements) runs inside the Blender container. Its add-on is vendored at upstream 41a18432 (MIT) and started by a startup hook. scripts/blender-mcp carries the stdio over ssh + docker exec, so the add-on socket, which runs arbitrary Python with no auth, stays on the container's localhost with no published port. It also runs there because viewport screenshots need a filesystem shared by server and Blender. Telemetry is off and safe mode is on. The hook also defaults Cycles to OptiX on GPU 3, because safe mode forbids agents from touching preferences. Verified end to end from nh3-dev: 36 tools; a GPU render of an agent-built scene; a viewport screenshot; and safe mode refusing 'import os'. Blender left down (on demand).
This commit is contained in:
@@ -43,9 +43,64 @@ Run as `-u abc`, the image's user, mapped to uid 1002, so outputs land owned by
|
||||
- Web auth: no credentials gives 401, a wrong password 401, the right one 200.
|
||||
- Selkies: "Render node 1 encodes H264, AV1, H265 on nvenc"; the Wayland renderer runs GL on GPU 3.
|
||||
|
||||
## Agent control (MCP)
|
||||
## Agent control (MCP): `scripts/blender-mcp`
|
||||
|
||||
Being researched with dvalin-smithy-dev: which Blender MCP server, and how it reaches Blender
|
||||
across hosts. This section fills in once the choice is made. ⚠ Most Blender MCP add-ons expose
|
||||
"run arbitrary Python inside Blender" on a TCP port. Treat that port like a shell: bind it to
|
||||
the container or the LAN only, never publish it wider.
|
||||
**The chosen server is [mcp-for-blender](https://github.com/ahujasid/mcp-for-blender)** (MIT, one
|
||||
maintainer, ~29k stars; researched by dvalin-smithy-dev 2026-09-27, thread
|
||||
`01M3JA61FTFW2ZPSD20MHF7RPJ`, full note in dvalin-smithy
|
||||
`research/blender-agent-drive-2026-09-27.md`). Two halves:
|
||||
|
||||
- **The add-on** is inside the running GUI Blender and serves a socket that executes arbitrary
|
||||
Python with **no authentication**. It is vendored at upstream commit `41a18432`
|
||||
(`conf/scripts/addons/blender_mcp.py`, MIT licence alongside) and started by
|
||||
`conf/scripts/startup/fleet_mcp.py`. The add-on only serves from a GUI Blender, never from
|
||||
`blender -b`, which is one reason the desktop exists.
|
||||
- **The MCP server**: `mcp-for-blender==2.1.1`, frozen in `conf/mcp-requirements.txt` and installed in
|
||||
`/work/.mcp-venv` **inside the container** (`scripts/blender-mcp setup`). Agents reach it as
|
||||
stdio over `ssh infra-ops@fv-ml1 docker exec -i`. It runs in the container for two reasons:
|
||||
1. **No port is published.** The socket stays on the container's localhost, so access means
|
||||
ssh + docker on fv-ml1.
|
||||
2. **Viewport screenshots need a shared filesystem.** Blender writes the image and the server
|
||||
reads it back. With the server on nh3-dev it failed ("Screenshot file was not created").
|
||||
|
||||
Settings: `DISABLE_TELEMETRY=true` and `BLENDER_MCP_SAFE_MODE=1`. Safe mode puts upstream's AST
|
||||
allowlist in front of `execute_blender_code`: bpy, bmesh, mathutils and pure stdlib only, and no
|
||||
os, open, eval or network. It guards against prompt injection from third-party asset text; it is
|
||||
not a sandbox.
|
||||
|
||||
### For an agent
|
||||
|
||||
```bash
|
||||
scripts/blender-mcp up # GPU 3 is borrowed: start only when needed
|
||||
scripts/blender-mcp status # wait for "mcp add-on: answering" (~10-40 s)
|
||||
claude mcp add blender -- /home/lkraven/development/eshpfi-management/scripts/blender-mcp # once per session/scope
|
||||
scripts/blender-mcp down # when finished: the card goes back to 0
|
||||
```
|
||||
|
||||
- **Always pass `user_prompt`.** The tools require it; it is a short statement of the user's
|
||||
request.
|
||||
- **Render on the GPU:** set `scene.cycles.device = 'GPU'` on any scene you create. Safe mode
|
||||
forbids touching `bpy.context.preferences`, so the startup hook has already pointed Cycles at
|
||||
OptiX on GPU 3. The startup scene is already set to GPU.
|
||||
- **Save everything under `/work/…`** (= `fv-ml1:/tank/blender`), then `scp` it out.
|
||||
- **Look before you report:** `get_viewport_screenshot` returns an image, and a still render to
|
||||
`/work` is the real check.
|
||||
- The asset tools (Poly Haven etc.) reach the internet from Blender. The paid ones (Hyper3D,
|
||||
Hunyuan, Tripo, Sketchfab) need keys we do not have; leave them off.
|
||||
|
||||
### Acceptance (2026-09-27, 1433)
|
||||
|
||||
An MCP client on nh3-dev → `scripts/blender-mcp` → the in-container server → the add-on:
|
||||
- `initialize` OK; **36 tools** listed.
|
||||
- `execute_blender_code` built a gold metallic torus and rendered it with Cycles on the GPU to
|
||||
`/work/_selftest/mcp-torus.png`. The file landed, and I checked it by eye.
|
||||
- `get_viewport_screenshot` returned an image of the scene.
|
||||
- **Negative control:** `import os` was rejected by safe mode.
|
||||
- `status` pings the add-on itself. A TCP probe was useless because the port accepted while
|
||||
Blender was still loading.
|
||||
|
||||
⚠ Two traps found on the way, both fixed and commented where they live:
|
||||
- Enabling the add-on from a startup script gets undone when user prefs load. The hook enables it
|
||||
in a timer instead.
|
||||
- A pre-flight `ssh` without `-n` swallowed the MCP client's `initialize`, and the session hung at
|
||||
init.
|
||||
|
||||
Reference in New Issue
Block a user