memory: snapshot — Ada sizing corrected; prune and drives are orthogonal
comfy-dev's disk-vs-catalog diff and a recount of my own figures both landed on this thread. Two numbers were wrong and both were headed for the operator's sizing conversation. - My ~90% was a double-count. I read ALLOC 1.45T while their pull was running and then added the full ~112 GB on top; most of it was already in that reading. "Onboarded" is not "landed". Settled payload is ~1.47 TiB and the as-bought mirror lands at 84%, not 90%. - comfy-dev's "pruning gets us nearer 45%" is the striped figure. On the as-bought pair mirrored, deleting all ~215 GiB of unreferenced weights still lands at 72%, with ~140 GiB of runway on a store that took on ~100 GiB in one day. The constraint is vdev layout, not payload — a 1.75 TiB pool stays 1.75 TiB whatever goes in it. - So the prune audit and the drive purchase are independent decisions and neither gates the cutover. Presenting them to the operator that way rather than as a trade. Also recorded: pool arithmetic (1.75 / 3.49 / 3.57 TiB), the mirror-vdev smallest-member gotcha if the new drives get paired one-each with the 1.92s, comfy-dev's 34 GiB of uncatalogued LTX 2.5, and an open question back to them on whether the H3 encoder's nvfp4 pin was set under a Blackwell assumption that sm_89 does not satisfy. Memory-only; no version bump per the SemVer SKIP list.
This commit is contained in:
@@ -75,17 +75,26 @@ both cheaper and likely faster than four more SATA drives.
|
||||
|
||||
⚠ **CAPACITY, not throughput, is the binding constraint — the as-bought drives are SMALLER
|
||||
than the pool they receive from** (measured 2026-09-01). ComfyUI's `/storetank` on irv-ml1 is
|
||||
a 2x 2 TB mirror: SIZE **1.81 TiB**, ALLOC **1.45 TiB**, CAP **80%**, and growing as
|
||||
comfy-dev's ~112 GB batch lands (-> ~1.56 TiB). The R750xa's 2x **1.92 TB** mirrored is only
|
||||
**~1.74 TiB** — the migration would arrive at **~90% full** with zero growth room, past the
|
||||
~80% line where ZFS allocation degrades. Compression buys nothing: safetensors measured
|
||||
a 2x 2 TB mirror: SIZE **1.81 TiB**, ALLOC **1.45 TiB**, CAP **80%**. Settled payload
|
||||
**~1.47 TiB**. The R750xa's 2x **1.92 TB** mirrored is only **~1.75 TiB** — smaller than the
|
||||
source pool — so the migration arrives at **~84% full** with no growth room, past the ~80%
|
||||
line where ZFS allocation degrades. Compression buys nothing: safetensors measured
|
||||
`compressratio 1.00x`, `logicalused == used`.
|
||||
|
||||
**-> Add 2x 2 TB SATA SSD to the buy list.** Six of eight bays free, HBA355i has the ports;
|
||||
two mirror vdevs striped = **~3.49 TiB at ~45%**, redundancy intact. Cheapest line on this
|
||||
buy list and it does not gate the cutover window. The no-spend alternative is striping the
|
||||
as-bought pair (same ~3.49 TiB, **no redundancy**) — only acceptable if irv-ml1 retains its
|
||||
copy, which makes retain a requirement rather than a recommendation.
|
||||
two mirror vdevs = **~3.57 TiB at ~41%**, redundancy intact. Cheapest line on this buy list
|
||||
and it does not gate the cutover window. The no-spend alternative is striping the as-bought
|
||||
pair (~3.49 TiB, **no redundancy**) — only acceptable if irv-ml1 retains its copy, which makes
|
||||
retain a requirement rather than a recommendation.
|
||||
|
||||
⚠ **Pruning does NOT substitute for the drives.** comfy-dev found ~215 GiB of unreferenced
|
||||
weights; deleting every byte still lands the as-bought *mirror* at **72%**. The constraint is
|
||||
**vdev layout**, not payload — a 1.75 TiB pool stays 1.75 TiB whatever goes in it. The prune
|
||||
audit and the drive purchase are independent decisions.
|
||||
|
||||
⚠ **Build-time: pair like with like.** A mirror vdev caps at its smallest member; pairing each
|
||||
new 2 TB with an existing 1.92 TB caps both vdevs at 1.92 TB and wastes ~150 GiB. Correct:
|
||||
the two 1.92s as one vdev, the two new drives as the other.
|
||||
→ [[2026-09-01-ada-migration-branch-a]]
|
||||
|
||||
**UNCHECKED, and it may moot the whole bay question:** free PCIe slots. Riser Config 0 is
|
||||
@@ -113,7 +122,7 @@ circuit this lands on before racking, not after.**
|
||||
|---|---|---|
|
||||
| RDIMM 16 GB 3200 2Rx8 | **`M04W6`** | **8** → restores 256 GB, all 16 slots, all 8 channels/socket |
|
||||
| NVIDIA 12VHPWR adapter | **`930-00030-1546-000`** | **2** |
|
||||
| 2 TB SATA SSD (any; match/exceed the MX500s) | — | **2** → pool ~3.49 TiB at ~45%; without it the migration lands at ~90% |
|
||||
| 2 TB SATA SSD (any; match/exceed the MX500s) | — | **2** → pool ~3.57 TiB at ~41%; without it the migration lands at ~84%. Pair the two NEW drives together, not one-each with a 1.92 |
|
||||
|
||||
**Only if the iDRAC inventory shows them absent:** `12XPY`, `9TR6X`, `4RW1P`, `W4K7M`,
|
||||
`XC48N`, `6C77X`, `CP67W`, `CXYF8`, `H4D7D`, `N61TK`, `HXJDR`, `W1P56`, `C2JNP`,
|
||||
|
||||
Reference in New Issue
Block a user