memory: reframe the GX10 work as what it was — a baseline and a tooling check
Operator clarified the purpose, so the record now leads with it: this was a baseline for the box and a check that the tooling loads, not a decision about where run 3c runs. The placement reasoning stays because it is sound, but it is marked as a byproduct rather than the deliverable. Two things were actually delivered. The box trains: aarch64 and sm_121 run torch 2.14.0+cu130 with transformers, accelerate, peft, trl, datasets, safetensors and bitsandbytes, plus the harness's own flex_attention backend and chunked-loss path, and nothing beyond python3-dev was needed. And the baseline is 79.35 s/it median across seven timed steps with a 0.19% spread. Also recorded the port scope without executing it, so nobody re-derives it: about 2.5 GB of data, a venv rebuild on aarch64, no encode cache worth moving since the encode runs in 14 seconds, and copy the corpus rather than mounting NFS on a desk box that will be unattended for hours.
This commit is contained in:
@@ -110,7 +110,15 @@ no longer deployed sidecars here. See Recent decisions.)
|
||||
|
||||
_As of 2026-09-01 — **the GX10 is on the operator's desk, NOT racked. Standing it up as an inference AND training box is the next action.** Nothing is training._
|
||||
|
||||
- **▶ NEXT: pfi-gx10 as an inference + training box.** The operator forgot to rack it; it is
|
||||
- **✅ pfi-gx10 BASELINED AND OUTFITTED (2026-09-02) — bare metal, training stack verified.**
|
||||
`uv` + torch 2.14.0+cu130 + transformers/peft/trl/datasets/bitsandbytes all clean on aarch64;
|
||||
nvcc, NVIDIA Container Toolkit, docker group, NOPASSWD sudo. **Baseline: ~79.35 s/it** on the
|
||||
run-3c shape (seq 16384, batch 2, LoRA r64, flex_attention, harness chunked loss) -> 604 steps
|
||||
≈ 13.3 h vs ana-ml2's 2.2-2.7 h. ⚠ **6x slower where compute predicts 2.7x** — likely memory
|
||||
bandwidth; **capacity box, not throughput box.** Port NOT done (operator stood it down; the
|
||||
baseline was the goal). Still on Wi-Fi at `10.100.10.226`, not racked.
|
||||
→ `persistent-memory.d/2026-09-01-pfi-gx10-onboarding.md`
|
||||
- **▶ (superseded) pfi-gx10 as an inference + training box.** The operator forgot to rack it; it is
|
||||
still on his desk on Wi-Fi at `10.100.10.226` (temp DHCP). Already headless and registered.
|
||||
**The box is BARE — no torch, no nvcc, no CUDA stack.** Purpose is local inference *and*
|
||||
**run 3c**, whose 49 GB bf16 base fits 121 GB unified with room where ana-ml2 was tight.
|
||||
|
||||
Reference in New Issue
Block a user