esh-pfi-infrastructure/persistent-memory.d/2026-09-01-pfi-gx10-onboarding.md

[2026-09-01] pfi-gx10 (ASUS Ascent GX10) onboarded headless — and it is the intended new home for run 3c

ASUS Ascent GX10 arrived and was registered, converted to headless, and given a rack-move playbook. It was NOT racked — the operator ran out of day. It is still on his desk, on Wi-Fi, on a temporary DHCP lease.

GPU	NVIDIA GB10, driver 580.173.02, compute capability 12.1 (sm_121)
CPU	20 cores, aarch64
Memory	121 GB UNIFIED — CPU and GPU share it; not 121 GB plus VRAM
Storage	916 GB NVMe, 6% used
OS	Ubuntu 24.04.4, kernel 6.17.0-1031-nvidia
Access	infra-ops NOPASSWD sudo (operator-bootstrapped). lkraven has key auth but needs a password to escalate — automation must connect as infra-ops.

Purpose (operator, 2026-09-01)

Local inference experiments and the failed training — run 3c. That is the whole point: run 3c died on ana-ml2 when a kilowatt-class training box tripped a breaker (2× 300 W GPUs + dual EPYC 9254). The GX10 is a ~240 W appliance, roughly a fifth of the draw, on a different site's circuits.

The memory arithmetic favours it strongly. Run 3c is a LoRA (r=64, batch 2 × accum 8, gradient checkpointing) over gemma4-26b-a4b-it-bf16 — 49 GB of base weights, working set roughly 55–65 GB. The gemma4-charrp compose warns in capitals that "48.10 GiB of BF16 weights CANNOT be served here" on ana-ml2's shared GPU0. 121 GB unified turns that constraint into a non-issue.

What has NOT been established — do not assume any of it

1. The box is bare. No torch, no nvcc, no CUDA stack. Docker 29.2.1 and 822 GB free.
2. aarch64 dependency risk. torch 2.9.1+cu130 ARM64 wheels exist; transformers/TRL/PEFT are pure Python. Compiled deps — flash-attn, bitsandbytes, liger, xformers — are open questions per-arch.
3. Triton has NO sm_121 support (established independently by comfy-dev the same day). Anything reaching for torch.compile or Triton-backed kernels is closed on this silicon.
4. Throughput is unmeasured. Run 3c ran 16.45 s/it on a 300 W RTX PRO 6000. GB10 will be slower; how much decides whether 604 steps is an overnight run or two days. Measure this before porting anything — the recommended first move is a probe: install ARM64 torch, load the base, run ten steps, report s/it.
5. Model transfer: 49 GB from ana-ml2 over the Anaheim↔NH3 link at a measured 32.4 MB/s — about 25 minutes. Note that is a third of the irv-ml1↔NH3 link's 99 MB/s.

The headless conversion, and the lesson inside it

playbooks/gx10-headless.yaml (1b596c8): multi-user.target, gnome-remote-desktop stopped, sleep/suspend/hibernate masked, logind ignores lid and idle, sshd keepalives, hostname corrected gx10-a745 → pfi-gx10.

⚠ gdm is a STATIC unit on Ubuntu — pulled in by display-manager.service, never "enabled". The first version guarded on is-enabled | grep enabled, which always skips, and the verify tested the same wrong property and passed. Six green verifies having not stopped the display manager. Both now test is-active. This is an instance of 2026-09-01-wrong-object-measurement.

The playbook refuses to stop GDM while a seat session is held (--var force_dm_stop=true to override) — automation should not yank a display out from under someone at the machine.

⚠ elway's --sudo applies only to ad-hoc --shell/--upload. Playbook steps run as the connecting user and must carry their own sudo.

The rack move, written but not run

playbooks/gx10-rack-network.yaml (a0c5fc6) — target settled: nh3-servers VLAN 50, static 10.100.50.60, clear of .40/.42/.50/.90 and below the .150 DHCP pool.

Requires nothing from the operator beyond racking it. The wired NIC has its own MAC (30:c5:99:3d:a7:45, distinct from Wi-Fi 50:bb:b5:a2:00:a8), so the post-move address AND switch port are discoverable from the UDM rather than relayed.

Design property worth preserving: the playbook never leaves itself one path back. Wi-Fi stays up while the wired interface is configured beside it; Wi-Fi teardown is explicitly a separate later change. netplan try's auto-rollback needs a TTY that elway cannot provide, so a live Wi-Fi link is what substitutes for it. Two preconditions are asserted as steps rather than assumed: carrier must be 1, and the MAC must match (interface names renumber across kernels; MACs do not).

⚠ The GX10 is NOT the Ada box. Two separate machines — see 2026-09-01-ada-migration-branch-a.

[2026-09-01 23:09] Liveness check — alive, idle, and STILL BARE (nothing has changed)

Probed at 2026-09-01T23:09:51-07:00, read-only:

host       pfi-gx10   Ubuntu 24.04.4 LTS   kernel 6.17.0-1031-nvidia   aarch64
uptime     11h48m (booted 2026-09-01 11:21)   load 0.00 0.01 0.00 — idle
GPU        NVIDIA GB10   driver 580.173.02
memory     121 GB unified, 118 GB free
disk       /dev/nvme0n1p2  916 GB, 48 G used, 822 G free (6%)
net        wlP9s9  10.100.10.226/24 (DHCP)  |  enP7s7 DOWN, NO-CARRIER (no cable)
sessions   2x lkraven desktop (seat0 + :1) — sitting at a GUI login screen

Compute stack unchanged and still absent: no torch, no nvcc, no uv. Docker daemon present but infra-ops is not in the docker group. Nothing has been installed since onboarding.

⚠ nvidia-smi reports FB Memory / BAR1 as N/A. That is CORRECT, not a fault — GB10 is a Grace Blackwell superchip with unified LPDDR5X shared between CPU and GPU, so there is no discrete VRAM figure to report. Do not chase it as a driver problem.

⚠ RACKING IS NOT A PREREQUISITE — the Wi-Fi is not the bottleneck

Assumed the temp Wi-Fi would gate getting a 49 GB base onto the box. Measured, and it does not:

link       SSID PFI, 6 GHz, 160 MHz, EHT (Wi-Fi 7), -48 dBm
           rx/tx bitrate 2401.9 Mbit/s both directions
measured   300 MB nh3-dev -> gx10 in 4.5 s = 67 MB/s over SSH
           (SSH cipher is the limiter, not the radio — link is ~300 MB/s)
implies    49 GB bf16 base transfers in ~12 min

So the throughput probe (ARM64 torch, load base, ten steps, report s/it) can proceed from the desk, today, without racking first. Racking remains one command (playbooks/gx10-rack-network.yaml, VLAN 50, static 10.100.50.60) and is worth doing for permanence, but it does not gate the decision the probe exists to answer.

[2026-09-01 23:2x] OUTFITTED (userspace) — CUDA WORKS on sm_121, full training stack imports

✅ RULING: BARE METAL, not Proxmox/VM. Settled, do not re-litigate.

1. Proxmox VE has no aarch64 build. It ships x86_64 only; the ARM ports are unofficial community work. Running an unsupported hypervisor under a compute asset is a bad trade.
2. The GPU cannot be cleanly passed through. GB10 is a superchip: the GPU sits on NVIDIA's own on-package root complex (bridges 22ce/22d0/22d1, GPU at 000f:01:00.0) and is cache-coherent with the CPU over NVLink-C2C, sharing the same physical LPDDR5X. Passing it to a guest means partitioning unified memory — which destroys the entire value proposition, since 121.6 GiB GPU-addressable is the reason this box exists.
3. Vendor's supported path is bare metal — it already runs NVIDIA's own kernel flavour (6.17.0-1031-nvidia).
4. Fleet convention agrees: ana-ml2 and irv-ml1 are both bare-metal GPU hosts. Hypervisors are separate machines in this fleet, never under the accelerators.
5. Single tenant. VM isolation buys multi-tenancy nobody needs; the rollback benefit is better served by containers + uv venvs, which is what the rest of the fleet does.

Installed and verified (all userspace, NO sudo needed)

uv                0.12.9 (aarch64), ~/.local/bin, wired into .bashrc + .profile
venv              ~/ml/.venv  (python 3.12)
torch             2.14.0+cu130   cuda runtime 13.0, cudnn 92400, triton 3.8.0
stack             numpy 2.5.2, transformers 5.16.1, accelerate 1.14.0, peft 0.20.0,
                  trl 1.12.0, datasets 5.0.1, safetensors 0.8.0, hf_hub 1.29.0,
                  bitsandbytes 0.50.2   -- ALL import clean on aarch64

The "per-arch unknowns" warning did not materialise for the training stack. Everything imported first try.

✅ MEASURED GPU COMPUTE — it works, and the memory is the point

device        NVIDIA GB10, capability sm_121, driver 580.173.02
GPU memory    121.6 GiB total / 110.8 GiB free   <- unified, fully CUDA-addressable
bf16 8192^3   ~93 TFLOP/s        fp16 ~93 TFLOP/s
fp32+tf32     38.2 TFLOP/s       fp32 (no tf32) 19.5 TFLOP/s
bf16/fp32     4.9x -> tensor cores ARE engaged

~93 TFLOP/s dense bf16 is roughly A6000-class throughput with 2.5x the memory. Capacity, not speed, is what this box buys. A 49 GB bf16 base fits with ~70 GB to spare.

⚠⚠ I NEARLY REPORTED A PHANTOM PROBLEM — NO WARMUP MEANS YOU MEASURE THE JIT

First benchmark (10 iterations, no warmup) read 27.0 TFLOP/s bf16 — slower than a 3090 — and I was one step from reporting "tensor cores are not engaging on sm_121." Adding 5 warmup iterations moved it to 93 TFLOP/s. The first measurement was timing the PTX JIT compile.

⚠ This is exactly docs/pfi/training-throughput-playbook.md §4: a run that completes, reports plausible numbers, and is wrong. It would have sent us chasing a cuBLAS fallback that does not exist. Always warm up before timing anything on this box — see the JIT note below for why it bites harder here than elsewhere.

⚠ sm_121 IS NOT IN TORCH'S COMPILED ARCH LIST — everything JITs from sm_120 PTX

torch.cuda.get_arch_list() -> ['sm_80','sm_90','sm_100','sm_110','sm_120']
device capability          -> sm_121

There is no sm_121 cubin in the wheel; it runs by PTX forward-compat JIT from sm_120. Consequences:

• First use of every kernel pays a JIT compile (cached afterward in ~/.nv/ComputeCache). This is what corrupted the benchmark above.
• ⚠ Any library shipping cubin-only binaries with no PTX will simply fail here. That is the real shape of the "per-arch unknown" risk — not source builds, but prebuilt kernels.

✅ PRIVILEGED HALF DONE — and I WAS NEVER BLOCKED

⚠⚠ The block was my own bad test. infra-ops has had NOPASSWD sudo since account creation (/etc/sudoers.d/infra-ops, installed 11:31 with the account). I probed with sudo -n -v, which returns non-zero on sudo >= 1.9.15 because -v refreshes an auth timestamp that a NOPASSWD-only rule never creates — and asked the operator for a password that was never needed.

pfi-gx10    sudo 1.9.15p5   sudo -n -v rc=1   sudo -n true rc=0
nh3-docker  sudo 1.9.13p3   sudo -n -v rc=0   sudo -n true rc=0
ana-docker  sudo 1.9.13p3   sudo -n -v rc=0   sudo -n true rc=0
irv-ml1     sudo 1.9.13p3   sudo -n -v rc=0   sudo -n true rc=0

Always sudo -n true. Fixed in scripts/elway (which carried the same probe and would have started falsely prompting fleet-wide as hosts pass sudo 1.9.13) and in auto-memory reference_infra_ops_sudo_identity, which recommended the broken test. feedback_assert_effective_value_not_substring

⚠ Same shape twice in one session. I also reported "nvcc: ABSENT" from command -v nvcc when the CUDA toolkit was installed all along at /usr/local/cuda/bin/nvcc, just not on PATH. Both times I checked a proxy and reported it as the thing itself.

Credential: the temp password was infra-ops's, not lkraven's. Rotated to 32 random chars and vaulted at pfi-gx10/infra-ops-password (round-trip verified). Kept rather than locked — deviating from the canonical bootstrap's no-password posture deliberately, because this box has no iDRAC/BMC, so a console break-glass credential is the only way back if key auth breaks. ⚠ lkraven (uid 1000, GUI desktop user, sudo group) has a password nobody knows; out of scope, but it is a gap.

⚠ DOCKER'S BUILDKIT DB WAS ALREADY CORRUPT — the restart surfaced it, did not cause it

nvidia-ctk runtime configure + systemctl restart docker failed with error initializing buildkit: error creating buildkit instance: invalid database. Not the nvidia config — daemon.json was valid and nvidia-container-runtime was present. dockerd had been running on a process whose on-disk buildkit state was already bad; it simply could not come back. Remediated by removing /var/lib/docker/buildkit (108 KB of cache, and the box had 0 images, 0 containers), then daemon-reload + reset-failed + start.

⚠ A long-running daemon can be unable to restart for hours before anyone finds out. Nothing reported unhealthy until something asked it to restart.

✅ FINAL STATE — playbook green, 5/5 verifies

playbooks/gx10-outfit-privileged.yaml -> steps 1 ok / 1 changed / 0 failed / 6 skipped
                                         verify 5 ok / 0 failed

NOPASSWD sudo                    OK
nvcc                             /usr/local/cuda/bin/nvcc (NOT on default PATH)
docker socket as infra-ops       OK (docker group)
CUDA container sees the GB10     OK (nvidia/cuda:13.0.0-base -> nvidia-smi -L reports GB10)
userspace torch still sees GPU   OK
docker Runtimes                  nvidia runc io.containerd.runc.v2

The box is fully outfitted for inference + training, bare metal, containers available.

Historical note — what the block looked like before it was disproved

⚠ infra-ops does NOT have NOPASSWD sudo on pfi-gx10 (sudo -n -v -> "a password is required"), unlike the rest of the fleet — see reference_infra_ops_sudo_identity, which should be corrected: gx10 is an exception. No gx10 credential is in the vault (secret list --prefix pfi-gx10 -> 0 items), and scripts/elway prompts via getpass, which a non-interactive session cannot answer.

Still outstanding, all needing root:

- CUDA toolkit / nvcc (absent -- needed to COMPILE kernels, not to run torch)
- NVIDIA Container Toolkit + add infra-ops to the docker group
  (docker daemon runs; infra-ops gets "permission denied" on the socket)
- NOPASSWD sudo for infra-ops, so this box stops being a fleet exception
- racking (playbooks/gx10-rack-network.yaml) -- optional, NOT a prerequisite

Nothing above blocks the run-3c throughput probe, which needs only the driver and torch.

[2026-09-01 23:5x] Run-3c throughput probe — IN FLIGHT, and the ana-ml2 baseline is now on record

The baseline nobody had written down: ana-ml2's real s/it for run 3c

Extracted from /tank/erp-tune/run-03c.log (the actual run, before the breaker killed it):

step 10   10.81 s/it       step 20   12.88 s/it       step 24   15.76 s/it
=> 604 steps on ana-ml2 ~= 2.2-2.7 h

This is the number the GX10 has to be compared against, and it was sitting in a log nobody had reduced. Anything under ~45 s/it on the GX10 makes 604 steps an overnight run.

Exact run-3c geometry (from run-03c.json + a real adapter_config.json)

base            /tank/aimodels/gemma4-26b-a4b-it-bf16   49 GB
                gemma4, 30 layers, hidden 2816, 16 heads / 8 KV, MoE (A4B),
                vocab 262144, sliding_window 1024, multimodal (vision+audio configs)
LoRA            r 64, alpha 128, dropout 0.0, bias none, 205 target modules
                (q/k/v/o + gate/up/down across all 30 language-model layers)
shape           max_seq_len 16384, per_device_batch 2, grad_accum 8 (effective 16)
other           gradient_checkpointing true, bf16, cosine, warmup 0.1, wd 0.01
length          604 optimizer steps, 1 epoch, save_steps 50

⚠ THE BACKEND-DELTA TRAP — CHECKED, and it is NOT present

CLAUDE.md warns that two frontier-panel conclusions were voided because a benchmark and a trainer had silently different attention backends. Enumerated both sides before running:

ana-ml2   flash_attn NOT installed in /tank/erp-tune/venv  -> falls back to sdpa
pfi-gx10  flash_attn not available for aarch64/sm_121      -> falls back to sdpa

Same backend both sides. ⚠ Library versions DO differ and it is recorded rather than assumed harmless:

ana-ml2   torch 2.13.0        transformers 5.15.1   peft 0.20.0   accelerate 1.14.0
pfi-gx10  torch 2.14.0+cu130  transformers 5.16.1   peft 0.20.0   accelerate 1.14.0

The probe reads the resolved _attn_implementation back off the loaded model rather than trusting the request, so the comparison is checkable rather than assumed.

Probe design (§4-disciplined)

~/ml/probe_3c.py on the GX10. Real geometry, synthetic batches — it measures step cost, and padding on the real run is 0.0% so the corpus does not change per-step cost materially.

• 12 steps, first 2 discarded as warmup. Non-negotiable here: an unwarmed benchmark on this box already read 27 TFLOP/s when the true figure was 93, because it was timing the PTX JIT. sm_121 is not in torch's arch list.
• Reports the step-time distribution (median, min/max, every step), not a mean beside a settings dump.
• Emits every provenance key always, null when unused — present-and-null, not absent.

Transfer

49 GB relayed ana-ml2 -> nh3-dev -> pfi-gx10, ~33 MB/s, ~25 min. Relayed rather than direct because ana-ml2's key is not authorised for infra-ops@pfi-gx10 — and relaying costs nothing extra, since the Wi-Fi leg is the bottleneck either way. Deliberately did not add an access grant just to save a hop.

[2026-09-02] GX10 BASELINE + TOOLING VERIFIED — ~79.35 s/it on the run-3c shape

⚠ PURPOSE, stated by the operator so it is not misread later: this was a BASELINE for the box and a check that the tooling loads. It was NOT a decision about where run 3c runs, and no port was undertaken. The placement reasoning below is a byproduct — real and worth keeping, but the deliverable was the number and the working stack.

Deliverable 1 — the tooling loads and runs. aarch64 + sm_121 runs the full training stack: torch 2.14.0+cu130, transformers, accelerate, peft, trl, datasets, safetensors, bitsandbytes, plus the harness's own flex_attention backend and chunked-loss path. Nothing exotic was needed beyond python3-dev. This box can train.

Deliverable 2 — the baseline.

The number

pfi-gx10    MEDIAN 79.36 s/it   min/max 79.30 / 79.45   (10 timed steps, 0.19% spread;
            warmup 85.90 then 79.32, discarded)
            peak memory 75.1 / 121.6 GiB -- 46 GiB SPARE, comfortably not memory-bound
            attn_resolved: flex_attention (matched to the harness, read off the model)
            FINAL, from the probe's own provenance block on the box
            -> 604 steps ~= 13.3 h
            peak 75.1 / 121.6 GiB (PyTorch max_memory_allocated) -- 46 GiB spare
ana-ml2     10.8-15.8 s/it (run-03c.log, the real run)
            -> 604 steps ~= 2.2-2.7 h

⚠ 6x slower where raw compute predicts 2.7x (93 TFLOP/s bf16 measured here vs an RTX PRO 6000 Blackwell's ~250). Hypothesis, not a measurement: the limiter is memory bandwidth — GB10's unified LPDDR5X against dedicated graphics memory, on a workload built to be bandwidth-hungry (seq 16384 with gradient checkpointing, which trades compute for memory traffic by design). Confirming it needs a bandwidth-bound microbenchmark that has not been run.

⚠ THE GX10 IS A CAPACITY BOX, NOT A THROUGHPUT BOX. 121 GB holds models nothing else in the fleet can. It is not the box to move a time-sensitive run to.

Byproduct: IF run 3c ever moves here, the GX10 is the right home — and infra-ops' reversal was WRONG

⚠⚠ infra-ops recommended ana-ml2 and the operator overruled it, correctly. The error is worth more than the recommendation: I costed the failure in the units I was measuring. I had step times, so I priced a breaker trip as "≤50 steps ≈ 11 minutes of lost training" — the recompute cost. That is not what a breaker trip at Anaheim costs.

Operator, 2026-09-02: "when the breaker trips at ana, that's a 40 minute drive to reset it. it's not 25 minutes, it's a giant pain in the ass."

The real cost of an Anaheim breaker trip:

a 40-minute drive EACH WAY, on the operator's time, whenever he notices
13 hosts down: ana-ml2, ana-docker, pfi-pve (hypervisor for most of them),
  pfi-postgres, pbs-ana (the fleet's PRIMARY backup server), ana-wg (WireGuard),
  pfi-ana-webhost, ana-filebot, pfi-pteradactyl, pfi-tacticalrmm
⚠ including THREE SureFire CLIENT hosts (sfsrv-ana, sf-ana-container, sf-r630)
  -- a client's production machines, under a hosting agreement

⚠ save_steps 100 → 50 caps the RECOMPUTE, not the OUTAGE. It was never the mitigation I claimed; it only bounds the cheapest component of the loss.

So: 13.3 h unattended on a desk in NH3, drawing desktop-class power and unable to take down anything, beats 2.5 h that can put a client's hosts dark until someone drives to Anaheim. The GX10's slowness is close to free here — nothing is waiting on this run — and the risk it removes is expensive and lands on the operator personally.

⚠ GENERAL FORM, and the reason this is recorded at length: when you recommend between options, check whether you priced the failure mode in the units you happened to be measuring. A metric in hand will volunteer itself as the unit of risk.

⚠ Power-capping ana-ml2 is a WEAK lever — measured, do not spend time on it

2x RTX PRO 6000 Blackwell Max-Q: limit 300 W, default 300 W, min 250 W, max 325 W

Capping both GPUs to the floor saves 100 W total on a box drawing ~1 kW. Not nothing, but nowhere near enough to make an Anaheim training run safe, and it costs throughput to buy it. Recorded so nobody re-derives it as a clever third option.

Probe methodology — five launches, five distinct causes, all recorded

1. reaped SSH             long remote job piped through a reapable foreground ssh.
                          FIX: setsid nohup, log ON the box. Work survived; output did not.
2. bare LoRA target names q_proj/k_proj/... also match Gemma-4's VISION and AUDIO towers,
                          whose projections are Gemma4ClippableLinear -- PEFT cannot wrap
                          them. FIX: the real run's 205 fully-qualified names, sha recorded.
3. missing python3-dev    Triton JITs a C shim at FIRST USE and needs Python.h. Torch
                          imports, model loads, LoRA attaches, THEN step 1 dies with a bare
                          CalledProcessError -- Triton devnulls the compiler's stdout.
4. naive-loss OOM (32 GiB) the 262,144-wide vocab x 32,768 positions logits tensor in fp32.
                          ⚠ NOT a GX10 limit -- the harness docstring says it OOMs "on a
                          95.60 GiB card (measured)", i.e. on ana-ml2 too.
5. self-inflicted VRAM leak a stuck orphan held 80 GB while PyTorch reported 0 bytes
                          allocated; every relaunch was doomed before it started and blamed
                          the NEW run. ⚠ ASSERT THE GPU IS CLEAR BEFORE EVERY LAUNCH.

⚠ THE BACKEND-DELTA TRAP WAS PRESENT AND I FIRST DECLARED IT ABSENT

CLAUDE.md records two frontier-panel conclusions voided by a benchmark and a trainer having silently different attention backends. I checked for it by asking whether flash-attn was installed — the wrong discriminator entirely — and reported "no delta". The harness sets ATTN_IMPLEMENTATION = "flex_attention" explicitly in code, with tuned kernel options; its own comment records peak 74.3 GiB under flex vs sdpa's 79.71.

⚠ Read the configured value in the code. Absence of an alternative is not evidence of the default. The probe now requests flex_attention and reads _attn_implementation back off the loaded model. RESOLVED: flex_attention — it compiles and runs on sm_121, which was a real open risk (flex compiles through Triton; sm_121 is not in torch's arch list).

Faithfulness of the probe (what makes the number comparable)

LoRA         the real run's 205 fully-qualified modules, sha256 c9ee070c0d67eb08
loss path    copied from the harness: trunk hidden states, DROP IGNORE_INDEX before the
             head, chunk lm_head+CE at loss_chunk_tokens=1024 under checkpoint, softcap 30.0
loss ratio   28.4% of positions masked to match the real 13,310,930 / 18,600,057 loss/ctx
             ratio -- observed 23,351-23,638 loss tokens/step against a 23,449 target
tokens       sampled [1000, 250000) to stay below every multimodal special token
             (boi 255999, boa 256000, image 258880, audio 258881, eoi/eoa/video 258882-4)
warmup       first 2 steps discarded; sm_121 JITs from sm_120 PTX on first use

⚠ NOT DONE, and deliberately: the run-3c port

Scoped but not executed — operator stood it down; the baseline was the goal. Scope, so it does not have to be re-derived:

harness eitri-smithy   26 MB      corpus /mnt/smithy/.../derived   2.3 GB
recipe-r3             485 KB      holdout                           82 MB
run-03c manifests      70 MB      -> ~2.5 GB, ~75 s at 33 MB/s
venv                  3.2 GB      NOT portable (x86-64) -- rebuild on aarch64

⚠ No encode cache to preserve. The launch script's "609 MB encode cache / ~4.3 hours" comment carries its own correction to 145.5 s, and run 3c's log shows [encode] done in 14.4s on 32 workers. Nothing worth moving.

⚠ Copy the corpus; do NOT mount /mnt/smithy over NFS on this box. A desk machine on Wi-Fi is the wrong place for a hard NFS dependency under a 13-hour unattended run — the fleet has already lost a host to a hard mount wedging into unkillable D-state (incident_esh_docker_nfs_boot_race).

The only untested piece is the harness's own 26 MB of Python; every library it leans on is verified above.