From f099caa238219130690928a9643136b3ff3e5d15 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 1 Sep 2026 23:05:19 -0700 Subject: [PATCH] =?UTF-8?q?memory:=20snapshot=20=E2=80=94=20H3=20encoder?= =?UTF-8?q?=20pin=20resolved;=20my=20question=20had=20the=20direction=20ba?= =?UTF-8?q?ckwards?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Checked the disk rather than waiting on comfy-dev. Both builds are there, pulled a minute apart on Aug 23: a 26 GB int8 and a 15 GB nvfp4-awq. I had asked whether their nvfp4 pin was set under a Blackwell assumption, since Ada has no native nvfp4, which would make the int8 file the right one on the new box. It cannot be. They pinned it on irv-ml1's A6000, which is Ampere sm_86 and has neither native nvfp4 nor native fp8. Ada sm_89 supports a strict superset, so a pin that was correct on the weaker card cannot be invalidated by moving to the stronger one. The migration is incapable of breaking it. The pin is about VRAM, not architecture. Eleven gigabytes on a 48 GB card that also holds a DiT and two VAEs decides whether a graph runs, and a text encoder runs once per prompt rather than once per diffusion step, so its throughput matters far less than the DiT's. That also explains why this pin went the opposite way from their other one without either being inconsistent. The RTX 6000 Ada is also 48 GB, so nothing relaxes. Reclassified the 26 GB int8 from orphan to spare: with the extra drives the destination lands near 14% full, so disk stops being the constraint and the pin-rot argument says keep it. Question withdrawn to comfy-dev. --- .../2026-09-01-ada-migration-branch-a.md | 39 +++++++++++++++---- persistent-memory.md | 6 ++- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/persistent-memory.d/2026-09-01-ada-migration-branch-a.md b/persistent-memory.d/2026-09-01-ada-migration-branch-a.md index eb5fbca..ed5d931 100644 --- a/persistent-memory.d/2026-09-01-ada-migration-branch-a.md +++ b/persistent-memory.d/2026-09-01-ada-migration-branch-a.md @@ -210,12 +210,37 @@ they had spent part of the day reporting LTX 2.5 as the strategic acquisition of scan and queuing a 15.9 GB REDGraft 2.5 checkpoint on top of a base already present. Their domain, no infra-ops action. -⚠ **OPEN QUESTION back to comfy-dev, unanswered:** `qwen3vl_32b_minimax_h3_int8_convrot` -(25.3 GiB) is listed as an orphan because the pin favours a 15.7 GB nvfp4 build over "the -27.1 GB int8 one". **sm_89 Ada has no native nvfp4** — that is Blackwell, and it is the same -fact that made their OTHER pin (away from a 7.74 GB nvfp4, toward a 12.84 GB int8) correct. -If the H3 pin was set under a Blackwell assumption, the int8 file may be the one that is right -on the new box. Asked, not asserted; infra-ops may be conflating two pins. +## ✅ THE H3 ENCODER PIN — infra-ops' question RESOLVED AGAINST ITSELF, no action + +Verified on disk 2026-09-01, both builds present, pulled a minute apart: + + 26G text_encoders/qwen3vl_32b_minimax_h3_int8_convrot.safetensors Aug 23 21:57 + 15G text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors Aug 23 21:58 + +**The question was:** comfy-dev pinned the nvfp4 build; **sm_89 Ada has no native nvfp4** (that +is Blackwell), and that same fact made their OTHER pin (away from a 7.74 GB nvfp4, toward a +12.84 GB int8) correct — so was the H3 pin set under a Blackwell assumption, making the int8 +file the one that is right on the new box? + +⚠ **NO — the question had the direction backwards, and this is a clean error not a close call.** +comfy-dev pinned it on **irv-ml1's A6000, which is Ampere sm_86** — no native nvfp4 *and* no +native fp8. **Ada sm_89 supports a strict superset** (adds fp8 e4m3, still no fp4). **A pin +correct on the weaker card cannot be invalidated by moving to the stronger one.** The migration +is incapable of breaking it. + +**What the pin is actually about: VRAM, not architecture.** 11 GB of difference on a 48 GB card +that also holds a DiT and two VAEs decides whether a graph runs; and a **text encoder runs once +per prompt, not once per diffusion step**, so its throughput contributes far less to render time +than the DiT's. Trading encode speed for headroom is right at that ratio — and it explains why +this pin went the *opposite* way from their other one without either being inconsistent (that +model was small enough that int8 fit, so speed won). **The RTX 6000 Ada is also 48 GB, so the +constraint does not relax.** Pin stands. + +⚠ **The int8 build is surplus but must NOT be read as a delete signal.** With 6+ SSDs the +destination lands near 14% rather than 84%, so disk stops being the constraint, and comfy-dev's +own pin-rot argument applies: a 26 GB artifact that may not be re-obtainable is worth more idle +than reclaimed. Reclassified from "orphan" to "spare". Question withdrawn to comfy-dev; nothing +owed by them. ## THE PATTERN THIS THREAD KEEPS PRODUCING @@ -298,8 +323,6 @@ comfy-dev raised it; infra-ops concurs. **Operator's call.** Two independent rea - **Retain vs reclaim irv-ml1's `/storetank`** — surfaced to the operator, retain recommended. - **comfy-dev's per-file prune audit** — offered, explicitly NOT gating the cutover, waiting on the operator wanting it. Their judgment, not infra-ops'. -- **The H3 encoder nvfp4-vs-int8 pin** — infra-ops asked whether it was set under a Blackwell - assumption; unanswered. - **comfy-dev's ~112 GB batch** — unheld by infra-ops; they deliberately did not pull until the operator approved putting that much onto his infrastructure. Manifest pinned and staged (`29324e9`). **Landed 2026-09-01**, SHAs green; 93.4 GiB on disk awaiting catalogue diff --git a/persistent-memory.md b/persistent-memory.md index 2ebc159..1b8d6fd 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -188,7 +188,11 @@ _As of 2026-09-01 — **the GX10 is on the operator's desk, NOT racked. Standing RETAIN recommended** (two paywalled pins mean a from-source rebuild cannot reproduce the stack; and under a no-spend striped destination it is the only redundancy). comfy-dev also found ~215 GiB of unreferenced weights and offers a per-file prune audit — **explicitly NOT - gating**, because pruning all of it still lands the as-bought mirror at 72%. + gating**, because pruning all of it still lands the as-bought mirror at 72%. ✅ **H3 encoder + pin RESOLVED** — infra-ops' nvfp4-vs-int8 question had the direction backwards (pinned on + Ampere sm_86; Ada sm_89 is a strict superset, so the move cannot break it). Pin is about + **VRAM not architecture**; stands. The 26 GB int8 is a **spare, not an orphan** — do not + delete. → `persistent-memory.d/2026-09-01-ada-migration-branch-a.md` - **⏸ Worldtree `route_not_found` awaiting the operator's DEPLOY PUSH.** Approved and landed by worldtree-dev as `ece0250c` (wire 2.5.0→2.6.0). main auto-deploys demo. Until it ships,