vh
39da1d4a97
The board mixed tools with endpoints. A vLLM seat whose href is a /docs page
sat in the same band as ComfyUI; the MQTT broker and the RustDesk relay, which
have no page at all, sat in Apps; and `Service Networking` was thirteen members
spanning three AdGuards, five Dockges, two Traefiks and four headless agents.
Every group is now one of two kinds and they never mix. TOOLS are expanded and
sit at the top of their tab. ENDPOINTS — an API, a broker, a background agent,
an href that is /docs or /ping or nothing — carry `initiallyCollapsed: true`
and sit at the bottom. Collapsed is not hidden: the eyebrow and its rule still
render, so the tab still says the thing exists and one click expands it.
A second rule fell out of the same pass and now shapes the group boundaries: a
group's members should all carry a widget or none should. A stat strip makes a
card ~50px taller, so one widget card in a row of plain ones opens a void under
the plain ones. That is why AdGuard and Traefik get their own groups rather
than sharing one with Dockge, and it is most of why the old Service Networking
band looked broken. AdGuard (ANA) was the last short card in its row and now
carries the same query/blocked/latency strip as its two siblings — one
infra-ops AdGuard login authenticates against all three instances, verified
against each; it lives in that stack's .env on the host and is vaulted.
The sixteen GPU-backed model seats were deliberately NOT relabelled.
`homepage.group` is read at container creation, so clearer names for
`AI - Inference` and friends would have cost a recreate on six vLLM seats, four
eval seats and four TTS engines — multi-minute model reloads on endpoints peers
reach through the gateway. Order plus `initiallyCollapsed` buys the same
separation for nothing, so those names stay as they are on purpose.
28 containers that ARE cheap to bounce were relabelled, across five hosts, via
rerunnable elway playbooks. Their label steps are gated on the old value still
being present, so a second run reports skipped rather than churning. Two verify
steps were wrong on first contact and are fixed with the reason recorded: the
traefik check raced its own recreate, and asserting a model seat is "running"
cannot answer "did I bounce it" when a seat may be legitimately stopped —
container age can, and now does.
The canonical stacks/ tree was synced to the deployed labels afterwards, so
intent and reality agree again on all fourteen tracked stacks.
Also documents the real nature of the post-recreate blank dashboard, which cost
~25 minutes here and an hour on 2026-08-19. `initialSettings":{}` in the served
HTML is the catch branch of the page's data loader, not a warm-up and not a
cache — and the error can vanish entirely, because the logger is assigned inside
the same try and the catch only logs if the logger exists. Ruled out by
measurement this time: all four API routes return 200 with correct content while
the page serves {}, and the previous known-good settings.yaml reproduces it
identically. The README now carries the one-command test and the next lead.
Before/after, all four tabs: http://10.100.10.50:8090/b/homepage-relayout/
ComfyUI
Node-based Stable Diffusion / Flux inference UI.
Server: irv-ml1 (Irvine, WireGuard-only)
Port: 8188 (configurable via .env)
GPUs: both (RTX 3090 + RTX A6000 both exposed; select per-workflow)
Image: mmartial/comfyui-nvidia-docker
Native coexistence: runs independently of /opt/ComfyUI (the existing native install). Migrate models by copy or move — no cross-binding.
Path layout
All user state lives under a single host tree on /worktank, owned
by lkraven:lkraven (UID/GID 1000) so external tooling can read and
write workflow files directly. ComfyUI-Manager puts downloaded weights
into the same tree, so everything is visible on the host.
| Host path | Container path | Purpose | Restic? |
|---|---|---|---|
/worktank/comfyui/basedir/models/ |
/basedir/models/ |
Checkpoints, loras, vae, unet, controlnet, upscale, etc. | excluded (bulk, regenerable) |
/worktank/comfyui/basedir/user/default/workflows/ |
/basedir/user/default/workflows/ |
Workflow JSON files — external tools edit here | included |
/worktank/comfyui/basedir/custom_nodes/ |
/basedir/custom_nodes/ |
ComfyUI-Manager-installed nodes | included |
/worktank/comfyui/basedir/input/ |
/basedir/input/ |
Uploaded source images | included |
/worktank/comfyui/basedir/output/ |
/basedir/output/ |
Generated images | excluded (regenerable) |
/worktank/comfyui/run/ |
/comfy/mnt |
ComfyUI source + venv + pip cache (~5 GB) | excluded (disposable) |
Standard ComfyUI model subdirs the image creates under models/:
checkpoints, clip, clip_vision, configs, controlnet,
diffusers, embeddings, gligen, hypernetworks, loras,
photomaker, style_models, unet, upscale_models, vae,
vae_approx.
First-time deploy on irv-ml1
# 1. Push compose + env template from workstation
scripts/deploy-stack.sh irv-ml1 comfyui
# 2. Create basedir + run on the host (the image expects both to exist
# as the target UID/GID before first up; it refuses to chown
# mounted paths). One-time sudo because /worktank is root-owned.
ssh -t irv-ml1 'sudo mkdir -p /worktank/comfyui/{basedir,run} && \
sudo chown -R lkraven:lkraven /worktank/comfyui'
# 3. Create .env and bring up. First `up` takes several minutes —
# the image installs ~5 GB of Python packages on first boot.
ssh irv-ml1 '
cd /opt/docker/compose/comfyui && \
cp -n .env.example .env && \
docker compose config >/dev/null && \
docker compose up -d && \
docker compose logs -f --tail=30
'
Once the logs settle on "Starting server" / "To see the GUI go to …",
the UI is reachable at http://10.100.79.3:8188 (from the WG-tunnel
side). The homepage hub on ana-docker auto-discovers it via the Dozzle
agent + the homepage Docker socket integration already wired on
irv-ml1.
Populate models
Native install lives at /opt/ComfyUI/models/ (owned llmuser).
Copy or move into the container's tree — paths match 1:1:
# Example: copy one subdir at a time, preserving timestamps
ssh -t irv-ml1 'sudo -u lkraven rsync -av \
/opt/ComfyUI/models/checkpoints/ \
/worktank/comfyui/basedir/models/checkpoints/'
Repeat per subdir. rsync -av --remove-source-files if you want a
true move (then clean up empty source dirs). The native install will
continue to work with whatever remains under /opt/ComfyUI/models/.
Editing workflows from outside the container
Workflows live as JSON under
/worktank/comfyui/basedir/user/default/workflows/ on the host,
owned lkraven:lkraven. External tools running as lkraven can
read and write them without coordination — ComfyUI picks up file
changes on the next workflow load (no container restart needed).
Deploy updates
# After editing compose.yaml or .env.example here
scripts/deploy-stack.sh irv-ml1 comfyui
ssh irv-ml1 'cd /opt/docker/compose/comfyui && docker compose up -d'
Image version bump
# Pick a new tag from
# https://hub.docker.com/r/mmartial/comfyui-nvidia-docker/tags
ssh -t irv-ml1 '
cd /opt/docker/compose/comfyui && \
sed -i "s/^COMFYUI_VERSION=.*/COMFYUI_VERSION=<new-tag>/" .env && \
docker compose pull && \
docker compose up -d
'
Runtime lives at /worktank/comfyui/run/. For a fresh bootstrap
(force venv + pip cache rebuild against the new image):
ssh irv-ml1 '
cd /opt/docker/compose/comfyui && \
docker compose down && \
rm -rf /worktank/comfyui/run/* && \
docker compose pull && \
docker compose up -d
'
User state under /worktank/comfyui/basedir/ is untouched by either
pattern. But a run/* wipe drops any pip deps that live only in the
venv — see RTX VSR below; re-run its pip step after every fresh bootstrap.
Custom nodes needing extra provisioning — RTX Video Super Resolution
The RTX Video Super Resolution node (Nvidia_RTX_Nodes_ComfyUI, Comfy-Org)
needs the NVIDIA MAXINE Video Effects SDK bindings (nvidia-vfx, import name
nvvfx), served from NVIDIA's package index, not public PyPI. Two pieces with
different durability:
| Piece | Lives in | Survives a run/* venv wipe? |
|---|---|---|
the node Nvidia_RTX_Nodes_ComfyUI |
basedir/custom_nodes/ (persistent, restic-included) |
yes |
the nvidia-vfx wheel |
the venv under run/ (disposable, restic-excluded) |
NO — re-install it |
So the node itself is durable, but any venv rebuild (rm -rf run/*, the
fresh-bootstrap on an image bump) drops nvidia-vfx and the node breaks on
import nvvfx until it's reinstalled. irv-ml1 already meets the node's
requirements (A6000 Ampere+ Tensor cores; driver 595.58 ≥ 590.44; container OS
fine).
Provision from scratch — both commands run as UID 1000; a root / -u 0
install corrupts venv ownership and crash-loops the boot torch step (see the
chown -R 1000:1000 venv lesson):
# 1. The node — only if basedir/custom_nodes was wiped (normally persistent):
ssh irv-ml1 'git clone https://github.com/Comfy-Org/Nvidia_RTX_Nodes_ComfyUI \
/worktank/comfyui/basedir/custom_nodes/Nvidia_RTX_Nodes_ComfyUI'
# 2. The pip dep — RE-RUN AFTER EVERY venv rebuild (it lives in the disposable venv):
ssh irv-ml1 'docker exec -u 1000:1000 comfyui \
/comfy/mnt/venv/bin/pip install nvidia-vfx --extra-index-url https://pypi.nvidia.com'
# Verify:
ssh irv-ml1 'docker exec -u 1000:1000 comfyui \
/comfy/mnt/venv/bin/python -c "import nvvfx; print(\"nvvfx OK\")"'
The --extra-index-url is kept scoped to this one install, deliberately NOT
promoted to a global PIP_EXTRA_INDEX_URL in compose: a fleet-wide extra index
could perturb the delicate pinned torch/cuda + SageAttention resolution the boot
bootstrap depends on (DISABLE_UPGRADES=true, torch 2.12.1). The node's own
requirements.txt lists nvidia-vfx, so if ComfyUI ever auto-installs node
requirements it will need this index reachable — but scoping it to the manual
step keeps the boot bootstrap untouched.