memory: snapshot — the Ada inference server is a stripped used R750xa
Dell R750xa JPJ1ZP3, 2x RTX 6000 Ada to be fitted, ComfyUI's new home at NH3. Diffing Dell's factory CSV against the reseller invoice shows four downgrades: half the RAM, the 2400 W PSUs, and the GPU risers, cables and high-performance fans all absent. Records the resolved GPU power chain, correcting my own first answer: the chassis' CPU 8-pin cabling is the right source type and NVIDIA 930-00030-1546-000 bridges it to the card's 12VHPWR, so the PCIe-type RCCWC I first proposed is withdrawn. Also closes the NVMe question — the backplane is SAS/SATA only — and notes that free PCIe slots may moot it. Auto-archival fired at 308 lines; seven entries moved to archival-memory.md. The 250-line target was not reached because the guards hold nearly everything else back as under 14 days or carrying open deferred work.
This commit is contained in:
@@ -0,0 +1,111 @@
|
||||
# `[2026-09-01]` The Ada inference server is a Dell R750xa bought used — and the reseller stripped four things Dell shipped
|
||||
|
||||
Dell PowerEdge R750xa, service tag **`JPJ1ZP3`**, 8-bay 2.5" 2U. Bought from Entera Source
|
||||
(invoice INV-74422, 2026-08-25). **2× NVIDIA RTX 6000 Ada 48 GB** to be fitted — confirmed by
|
||||
the operator, NOT L40/L40S. Destination NH3. This is ComfyUI's new home; the GX10 is a
|
||||
separate machine → [[2026-09-01-pfi-gx10-onboarding]].
|
||||
|
||||
## As bought
|
||||
|
||||
CPU 2x Xeon Silver 4314 16C/32T each = 32C/64T, 135 W each
|
||||
RAM 8x Micron 16 GB PC4-3200 = 128 GB
|
||||
NIC dual 25 GbE SFP28 (Broadcom 57414, OCP 3.0)
|
||||
boot BOSS-S2 + 2x 240 GB M.2 SATA, mirrored
|
||||
storage 2x 1.92 TB SATA SSD (Hynix), 6 of 8 bays free
|
||||
HBA PERC HBA355i (LSI SAS3816)
|
||||
PSU 2x 1400 W Platinum
|
||||
|
||||
## THE DIFF — factory CSV (`JPJ1ZP3.csv`, booth `54-puffin`) vs the invoice
|
||||
|
||||
| | Dell factory | reseller invoice |
|
||||
|---|---|---|
|
||||
| **RAM** | **16× 16 GB `M04W6`**, "Performance Optimized" = **256 GB** | 8× = 128 GB |
|
||||
| **PSU** | **2400 W** redundant (`450-AJEV`) | 2× 1400 W (`7DWXY`) |
|
||||
| **GPUs** | 4× Tesla V100-32GB passive (`490-BHUC`) | none |
|
||||
| **GPU risers** | `12XPY` (right DW), `9TR6X` (left DW) | **not listed** |
|
||||
| **GPU cables** | `470-AEVS` 4-GPU kit: `4RW1P` `W4K7M` `XC48N` `6C77X` | **not listed** |
|
||||
| **Riser cables** | signal `CP67W` `CXYF8` `H4D7D` `N61TK`; power `HXJDR` `W1P56`; pad `C2JNP` | **not listed** |
|
||||
| **Fans** | 6× `FD00R` **Very High Performance** | **not listed** |
|
||||
| **Controller** | PERC H345 | **HBA355i — an UPGRADE for us** |
|
||||
| **Drives** | 2× 480 GB | 2× 1.92 TB |
|
||||
|
||||
⚠ **"Not listed on the invoice" is not proof of removal.** Operator will pull an iDRAC
|
||||
inventory to settle it. Diff that against the CSV rather than the invoice.
|
||||
|
||||
## The GPU power chain — resolved, and my first answer was wrong
|
||||
|
||||
**RTX 6000 Ada: 300 W, ACTIVE cooling, one 16-pin 12VHPWR (CEM5) connector** (nvidia.com).
|
||||
|
||||
The R750xa kit `470-AEVS` is **"W/CPU CONN"** — CPU/EPS 8-pin, because V100 takes that. I
|
||||
initially flagged this as a mismatch requiring the PCIe-type cable `RCCWC` / Dell
|
||||
`470-BBXH`. **That was wrong and is withdrawn** — `RCCWC` is for H100/L40/L40S.
|
||||
|
||||
NVIDIA sells the bridge for exactly this case:
|
||||
|
||||
> **`930-00030-1546-000`** — NVIDIA Power Cable, RTX 6000 Ada Generation,
|
||||
> **1× CPU 8-Pin → 1× CEM5 16-Pin**
|
||||
|
||||
So the chain is `470-AEVS` CPU 8-pin → `930-00030-1546-000` → card. **The chassis' existing
|
||||
cabling is the correct source type.** Two adapters needed, one per card.
|
||||
|
||||
⚠ Factory brackets are `VFYGM` — the **V100S** extended bracket. RTX 6000 Ada is a different
|
||||
physical card; bracket fit is UNKNOWN, check at fitting.
|
||||
|
||||
⚠ **Active cooling in a passive-card chassis.** Six high-performance fans expect to push air
|
||||
through passive cards; the RTX 6000 Ada has its own blower. It works in practice, but Dell
|
||||
never validated this combination for this box.
|
||||
|
||||
## Storage — NVMe in the drive bays is CLOSED
|
||||
|
||||
Factory backplane is `321-BGKG : 2.5" Chassis with up to 8 **SAS/SATA** Drives`. Not
|
||||
NVMe-capable; bays would need a backplane swap.
|
||||
|
||||
Arithmetic if filling bays with SATA:
|
||||
|
||||
2x SATA (today) ~1.1 GB/s
|
||||
6x striped ~3.3 GB/s <- roughly ONE NVMe drive
|
||||
8x striped ~4.4 GB/s
|
||||
backplane ceiling ~6 GB/s (8 lanes x SATA III)
|
||||
|
||||
The HBA355i (PCIe 4.0 x8) is nowhere near limiting. **Drives are the constraint.**
|
||||
|
||||
**ARC probably beats spindles here.** ComfyUI reloads the same checkpoints; with the factory
|
||||
256 GB restored, ZFS ARC caches most of the working set at RAM speed. The RAM upgrade is
|
||||
both cheaper and likely faster than four more SATA drives.
|
||||
|
||||
**UNCHECKED, and it may moot the whole bay question:** free PCIe slots. Riser Config 0 is
|
||||
`330-BBUM : Riser Config 0, 6x16 + 2x8 slots`; two double-wide GPUs will not consume all of
|
||||
it. An NVMe add-in card (U.2 HBA or bifurcating M.2 carrier) bypasses the SAS/SATA backplane
|
||||
entirely. **Add slot inventory to the iDRAC pull.**
|
||||
|
||||
## Power — the ceiling is deliberate now, not discovered later
|
||||
|
||||
2x RTX 6000 Ada 600 W
|
||||
2x Xeon 4314 270 W
|
||||
rest ~150 W
|
||||
~1,020 W sustained
|
||||
|
||||
One 1400 W PSU carries that, so redundancy holds. But factory specced **2400 W** for four
|
||||
V100s: **there is no headroom for a third or fourth GPU.** And ~1 kW is the same class as
|
||||
ana-ml2, the box whose training run tripped a breaker on 2026-08-26 — **decide which NH3
|
||||
circuit this lands on before racking, not after.**
|
||||
|
||||
## Buy list
|
||||
|
||||
**Certain:**
|
||||
|
||||
| item | part | qty |
|
||||
|---|---|---|
|
||||
| RDIMM 16 GB 3200 2Rx8 | **`M04W6`** | **8** → restores 256 GB, all 16 slots, all 8 channels/socket |
|
||||
| NVIDIA 12VHPWR adapter | **`930-00030-1546-000`** | **2** |
|
||||
|
||||
**Only if the iDRAC inventory shows them absent:** `12XPY`, `9TR6X`, `4RW1P`, `W4K7M`,
|
||||
`XC48N`, `6C77X`, `CP67W`, `CXYF8`, `H4D7D`, `N61TK`, `HXJDR`, `W1P56`, `C2JNP`,
|
||||
`FD00R` ×6.
|
||||
|
||||
**Spend order:** RAM → check PCIe slots → only then consider more SATA drives.
|
||||
|
||||
⚠ Dell's Installation & Service Manual publishes **no part numbers at all** for GPU kit
|
||||
hardware — names and quantities only. The factory CSV keyed to the service tag is the only
|
||||
authoritative source of DPNs for this machine. Do not accept a five-character DPN from
|
||||
anywhere else without corroboration.
|
||||
Reference in New Issue
Block a user