feat(comfyui): switch allocator to cudaMallocAsync (A/B won, fixes LTX OOM)
Mirror comfy-dev's operator-run allocator A/B result off irv-ml1: drop --disable-cuda-malloc (ComfyUI keeps CUDA's default async allocator) and remove PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (native-allocator-only, inert under cudaMallocAsync). The native+expandable_segments combo was fragmenting/over-reserving (~45 GB allocated-but-unused) and OOMing the LTX-2.3 v1.5.0 LoRA stack at Gemma TE load; cudaMallocAsync packs tighter + returns freed blocks so the job fits (stress test peaks ~82% VRAM, 0 OOM). The shared-A6000 phantom-OOM that --disable-cuda-malloc guarded is gone since TTS moved to the 3090 (2026-06-18).
This commit is contained in:
+17
-13
@@ -43,10 +43,6 @@ services:
|
||||
- USE_UV=true
|
||||
# Extra ComfyUI launch flags (image appends these to main.py, then adds
|
||||
# --base-directory + --enable-manager itself):
|
||||
# --disable-cuda-malloc — use PyTorch's native allocator instead of the
|
||||
# default cudaMallocAsync, which phantom-OOMs ("allocation would exceed
|
||||
# allowed memory", 0 B allocated) when the A6000 is shared with the
|
||||
# pinned TTS services (chatterbox-fast / csm-expressiva, ~22 GB).
|
||||
# --fp8_e4m3fn-text-enc — load the FLUX.2 Qwen3-8B text encoder as fp8
|
||||
# (~8.7 GB) instead of upcasting the fp8 file to fp16 (~16 GB). Matches
|
||||
# the box's Ampere-fp8 posture; the encoder runs once per gen so the
|
||||
@@ -56,15 +52,23 @@ services:
|
||||
# ComfyUI attention changes"). Binds the in-image sageattention v2.2.0
|
||||
# sm_86 build (rebuilt against the pinned torch 2.12.1). Global speedup
|
||||
# across Flux/SDXL/Wan (comfy-dev benchmarking, 2026-06-18).
|
||||
- COMFY_CMDLINE_EXTRA=--disable-cuda-malloc --fp8_e4m3fn-text-enc --use-sage-attention
|
||||
# Let PyTorch's native caching allocator grow segments instead of
|
||||
# pre-carving fixed blocks — cuts fragmentation OOMs on long-lived
|
||||
# Qwen-Image-Edit sessions (a ~2 GB alloc failing with 1.75 GB free while
|
||||
# 45 GB sat allocated + reserved-but-unallocated). Cache-preserving: packs
|
||||
# better WITHOUT unloading the checkpoint, so no edit-latency hit. Only
|
||||
# valid under the native allocator (paired with --disable-cuda-malloc
|
||||
# above; incompatible with cudaMallocAsync). comfy-dev request 2026-06-25.
|
||||
- PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
|
||||
#
|
||||
# ALLOCATOR (2026-07-19 — comfy-dev A/B, operator-run — cudaMallocAsync WON).
|
||||
# We deliberately DO NOT pass --disable-cuda-malloc, so ComfyUI keeps CUDA's
|
||||
# default async allocator (cudaMallocAsync). History: --disable-cuda-malloc +
|
||||
# PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True were added when the A6000 was
|
||||
# SHARED with the TTS zoo (the native allocator dodged a cudaMallocAsync
|
||||
# phantom-OOM). TTS moved to the 3090 (2026-06-18), removing that trigger; the
|
||||
# LTX-2.3 v1.5.0 LoRA stack (DMD+OmniNFT+act LoRAs patching the DiT + the 12B
|
||||
# Gemma text-encoder) then began hitting the 48 GB ceiling under the native
|
||||
# allocator, which fragments/over-reserves (~45 GB allocated+reserved-but-
|
||||
# unallocated) and OOMs at TE load. cudaMallocAsync packs tighter + promptly
|
||||
# returns freed blocks, so the same job now FITS: the operator's previously-
|
||||
# OOMing stress test peaks ~82% VRAM (~40/48 GB) with headroom, 0 OOM/errors.
|
||||
# COUPLING: PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is native-allocator-
|
||||
# only, so it is REMOVED here (inert/invalid under cudaMallocAsync) — revert BOTH
|
||||
# together. If the TTS zoo ever moves back onto the A6000, re-evaluate the pair.
|
||||
- COMFY_CMDLINE_EXTRA=--fp8_e4m3fn-text-enc --use-sage-attention
|
||||
volumes:
|
||||
- ${COMFYUI_BASEDIR}:/basedir
|
||||
# models/ overlaid from storetank. The ~325 GB model tree was migrated
|
||||
|
||||
Reference in New Issue
Block a user