Native-allocator expandable segments to cut Qwen-Image-Edit fragmentation
OOMs on the A6000 (a ~2 GB alloc failing with 1.75 GB free while 45 GB sat
allocated + reserved-but-unallocated). Cache-preserving — packs better
without unloading the checkpoint, so no edit-latency hit. Paired with the
existing --disable-cuda-malloc (incompatible with cudaMallocAsync).
Deployed + recreated on irv-ml1; verified env present, PyTorch reads it,
container healthy. comfy-dev request 2026-06-25.
ComfyUI 0.24.1 added native attention selection; the node-based
BlehGlobalSageAttention errors "does not support the new ComfyUI attention
changes". Add --use-sage-attention to COMFY_CMDLINE_EXTRA so the in-image
sageattention v2.2.0 sm_86 build (rebuilt vs pinned torch 2.12.1) binds via
the native path. OOM flags preserved. Deployed to irv-ml1 + recreated; log
confirms "Using sage attention", container healthy, serving 200.
(comfy-dev request, thread 01KVE89T2DKC)
Operator consolidation (2026-06-18): give ComfyUI the full 48 GB A6000 and move the
audio/TTS zoo to the 3090.
- comfyui: NVIDIA_VISIBLE_DEVICES all -> 1 (A6000 only), + DISABLE_UPGRADES=true to
pin torch at 2.12.1+cu129 so the mmartial boot script stops auto-upgrading it and
the compiled SageAttention kernels stay matched (comfy-dev torch-pin, approved).
- parakeet: NVIDIA_VISIBLE_DEVICES all -> 0 (3090).
Other GPU reassignments are deployment-side (not repo compose): chatterbox-fast via
its .env CBF_GPU_DEVICES=0; vibevoice device_ids ["1"]->["0"] (deployed from
/worktank/vibevoice/build); yt-voice-clipper worker via its override. dia2-2b,
ace-step, csm-expressiva downed (stale/unused).
Result: A6000 = ComfyUI alone (48.3 GB free); 3090 = chatterbox + parakeet + the
on-demand audio (vibevoice/ytvc/kokoro). SageAttention rebuilt against the pinned
torch; OOM cmdline (COMFY_CMDLINE_EXTRA) preserved; /object_info still lists the 9
acceleration nodes.
ComfyUI's ~325G model tree moved off the near-full worktank NVMe (97%->26%,
342G free) to /storetank/arbo (roomy SATA SSD on irv-ml1), overlay-mounted
back at /basedir/models so ComfyUI behaviour is unchanged. rsync byte-verified
(src==dst), one comfyui restart, worktank original removed. Inventory of the
set in docs/arbo-comfyui-model-catalog.md for the retain decision. The older
919G /storetank/image-models/comfy archive is untouched (separate reclaim).
ComfyUI's default cudaMallocAsync allocator phantom-OOMs ('allocation
would exceed allowed memory', 0 bytes allocated) when the A6000 is
shared with the pinned TTS services (~22 GB used). --disable-cuda-malloc
switches to PyTorch's native allocator. --fp8_e4m3fn-text-enc loads the
FLUX.2 Qwen3-8B text encoder as fp8 (~8.7 GB) instead of upcasting the
fp8 file to fp16 (~16 GB), matching the box's Ampere-fp8 posture.
Applied via COMFY_CMDLINE_EXTRA in the canonical compose; verified the
allocator flipped to 'native' and both flags are on the live cmdline.
New stack mirroring PFI convention (stacks/comfyui/) using
mmartial/comfyui-nvidia-docker:ubuntu24_cuda12.8-20260312. Both GPUs
exposed, pinned to CUDA 12.8 to match the host's 570.x driver and the
native cuda-toolkit already in place.
Layout — single tree under /worktank/comfyui/ (462G dedicated, 1%
used pre-deploy):
- basedir/ → /basedir user state (models, workflows, custom_nodes,
input, output); owned 1000:1000 so external
tools can edit workflow JSON directly.
- run/ → /comfy/mnt ComfyUI source + venv + pip cache (~7.8G
after bootstrap). Bind mount instead of
named volume — the image refuses to chown
mounted paths at startup, so keeping this
lkraven-owned avoids the sudo dance.
servers/irv-ml1/README.md refreshed: Docker upgraded to 29.4.1 with
traefik-net in place; dockge + beszel-agent + dozzle-agent already
present; /storetank dropped 92% → 64%; restic coverage to
rest-server-nh3 is operational (not "currently none" as prior text).
vh