fix(comfyui): native allocator + fp8 text-enc to stop FLUX.2 phantom-OOM

ComfyUI's default cudaMallocAsync allocator phantom-OOMs ('allocation
would exceed allowed memory', 0 bytes allocated) when the A6000 is
shared with the pinned TTS services (~22 GB used). --disable-cuda-malloc
switches to PyTorch's native allocator. --fp8_e4m3fn-text-enc loads the
FLUX.2 Qwen3-8B text encoder as fp8 (~8.7 GB) instead of upcasting the
fp8 file to fp16 (~16 GB), matching the box's Ampere-fp8 posture.

Applied via COMFY_CMDLINE_EXTRA in the canonical compose; verified the
allocator flipped to 'native' and both flags are on the live cmdline.
This commit is contained in:
2026-06-05 01:43:28 -07:00
parent 03d73a212e
commit 89a020a3cc
+11
View File
@@ -36,6 +36,17 @@ services:
- BASE_DIRECTORY=/basedir
- SECURITY_LEVEL=${COMFYUI_SECURITY_LEVEL:-normal}
- USE_UV=true
# Extra ComfyUI launch flags (image appends these to main.py, then adds
# --base-directory + --enable-manager itself):
# --disable-cuda-malloc — use PyTorch's native allocator instead of the
# default cudaMallocAsync, which phantom-OOMs ("allocation would exceed
# allowed memory", 0 B allocated) when the A6000 is shared with the
# pinned TTS services (chatterbox-fast / csm-expressiva, ~22 GB).
# --fp8_e4m3fn-text-enc — load the FLUX.2 Qwen3-8B text encoder as fp8
# (~8.7 GB) instead of upcasting the fp8 file to fp16 (~16 GB). Matches
# the box's Ampere-fp8 posture; the encoder runs once per gen so the
# upcast-on-compute cost is negligible.
- COMFY_CMDLINE_EXTRA=--disable-cuda-malloc --fp8_e4m3fn-text-enc
volumes:
- ${COMFYUI_BASEDIR}:/basedir
- ${COMFYUI_RUNDIR}:/comfy/mnt