feat(pfi-gx10): stage ERP-seat SFT run 3c — verified, not launched

Rehome run 3c from ana-ml2 to pfi-gx10 unchanged: same corpus, base,
recipe and hyperparameters, different host. Slower (~13.3 h vs ~2.5 h)
and correct — an Anaheim breaker trip costs a 40-minute drive each way
and 13 hosts down, three of them SureFire client machines, while the
GX10 is a ~240 W appliance at NH3 that can take nothing else down.

Verified rather than assumed, because ana-ml2 ran transformers 5.15.1
on x86-64 and this box runs 5.16.1 on aarch64 — the silent
backend-delta class that has already voided conclusions here:

  - both 49 GB base shards sha256-match ana-ml2's (size equality is a
    weaker claim and was already true)
  - a full encode was run into a throwaway dir and the encoded corpus
    compared byte-for-byte: 197,360,233 B, sha256 c08bb1fe2ecb0be3,
    identical. Every aggregate matched too. That verified artifact is
    what the run will train on — it is seeded into run-03c/encode-cache
  - the harness's own suite: 122 passed on aarch64
  - the config generator asserts key-by-key that no non-path value
    differs from run-03c.json

The encode-cache filename differs by design (base_model_path is part of
the key) — an input hash, not an output hash. Documented so it is not
misread as drift, or "fixed" by faking /tank on this box.

Corpus is copied to local NVMe; the box mounts no NFS. nh3-nas is now on
the same subnet, which makes mounting it tempting and still wrong under
a 13 h unattended run.

The launcher refuses on a live pidfile rather than a pgrep: `pgrep -f
erp_sft_harness` invoked over ssh matches the invoking shell's own argv.
That self-match cost a shell during staging.

Not launched. 13.3 h is the operator's call.

docs/runbooks/gx10-run-03c.md

@@ -0,0 +1,115 @@
+# pfi-gx10 — ERP-seat SFT run 3c, staged and ready to launch
+
+Staged 2026-09-03. **Not launched** — the launch is the operator's call.
+
+Run 3c is the LoRA that died on ana-ml2 at step 24 of 604 when an Anaheim
+breaker tripped. It is rehomed to pfi-gx10 unchanged: same corpus, same base,
+same hyperparameters, different host.
+
+## Why this box and not the fast one
+
+ana-ml2 finishes this run in ~2.5 h and pfi-gx10 needs ~13.3 h, and the slow box
+is still correct. An Anaheim power event is not priced in lost training steps —
+it is a 40-minute drive each way on the operator's time, with 13 hosts down
+including `pbs-ana` (the fleet's primary backup server) and **three SureFire
+client machines** under a hosting agreement. The GX10 is a ~240 W appliance at
+NH3 that can take nothing else down, and nothing is waiting on this run.
+
+Power-capping ana-ml2 is not a third option: both RTX PRO 6000s floor at 250 W
+against a 300 W default, so the whole lever is 100 W on a ~1 kW box.
+
+## Layout on the box
+
+    /home/infra-ops/models/gemma4-26b-a4b-it-bf16   49 GB base (sha-verified, see below)
+    /home/infra-ops/erp-tune/
+      eitri-smithy/            harness, git 0a6bd2e, tracked tree clean
+      recipe-r3/               recipe, survivors, loss-mask
+      datasets/derived/        2.4 GB corpus  -- COPIED, see the NFS note
+      datasets/holdout/        holdout, for the disjointness assertion
+      run-03c/encode-cache/    pre-seeded with the VERIFIED encode (below)
+      run-03c-gx10.json        the config
+      launch-run-03c.sh        the launcher
+    /home/infra-ops/ml/.venv   torch 2.14.0+cu130, transformers 5.16.1, peft 0.20.0
+
+Canonical copies of the config and launcher live in this repo under
+`scripts/erp-tune-gx10/`.
+
+## ⚠ The corpus is COPIED, and the box mounts no NFS at all
+
+`/mnt/smithy` is deliberately **not** mounted here. A 13-hour unattended run is
+the worst possible place for a hard NFS dependency — the fleet has already lost
+a host to a hard mount wedging into unkillable D-state. The corpus is 2.4 GB and
+copied in 50 s from nh3-dev's mount; there is nothing to gain by mounting it.
+
+Verify before every launch: `mount | grep -E 'nfs|cifs'` returns nothing.
+
+## What was verified, rather than assumed
+
+**The base model is byte-identical to ana-ml2's.** Both shards sha256-matched
+across 51.6 GB, not just size-checked:
+
+    model-00001-of-00002.safetensors  1127684971bbca40465435a5cad69d67ad603bf5e61c6dfd5561fae4a3bcfdb3
+    model-00002-of-00002.safetensors  aab47033e1e8a492ef8e581efae1cf36478d0433567e7729b3c1728bc8970db7
+
+**The tokenizer did not drift across the library delta.** ana-ml2 ran
+transformers 5.15.1 / torch 2.13.0 on x86-64; this box runs 5.16.1 / 2.14.0+cu130
+on aarch64. That is exactly the silent backend-delta class that has already
+voided conclusions on this project, so it was measured, not reasoned about: a
+full encode was run here into a throwaway output dir and the encoded corpus
+compared byte-for-byte against ana-ml2's.
+
+    ana-ml2   encoded-c16316f1c1bb21da.jsonl   197,360,233 B   sha256 c08bb1fe2ecb0be3…
+    pfi-gx10  encoded-fd8fe1944fb316b2.jsonl   197,360,233 B   sha256 c08bb1fe2ecb0be3…
+
+Identical. Every aggregate matched too — 9,504 training vs 8,404 holdout ids with
+0 overlap, 15 unfittable dropped, 9,662 records, ctx 18,600,057 / loss 13,310,930
+tokens, and the five mix shares to four decimals.
+
+⚠ **The cache-key filenames differ and that is correct, not a mismatch.**
+`base_model_path` is part of the encode-cache key by design, so rehoming the base
+changes the key while leaving the content identical. The key is an input hash;
+the sha above is the output. Do not read the differing filenames as drift.
+
+That verified artifact was then copied into `run-03c/encode-cache/`, so the run
+trains on the exact bytes that were compared — it will report `[encode] cache
+hit` rather than re-encoding.
+
+**The harness runs on aarch64.** Its own suite: 122 passed.
+
+**Throughput.** ~79.4 s/it measured on this exact geometry (median of 10 timed
+steps, 0.19% spread), peak 75.1 of 121.6 GiB. 604 steps ≈ 13.3 h. Against
+ana-ml2's 10.8–15.8 s/it that is ~6× slower where raw compute predicts ~2.7×;
+the unconfirmed hypothesis is unified-LPDDR5X bandwidth, not a misconfiguration.
+
+## Launch
+
+    ssh infra-ops@10.100.50.60 '~/erp-tune/launch-run-03c.sh'
+
+The script refuses rather than proceeds if the GPU is not clear, if
+`run-03c.pid` names a live process, if the log already exists, if any input is
+missing, or if there is under 40 GB free. It detaches with `setsid nohup` and
+logs on the box, because a long remote job piped through a foreground SSH has
+already been reaped once on this hardware — the work survived, the output did not.
+
+Watching, and stopping:
+
+    tail -f ~/erp-tune/run-03c.log | tr '\r' '\n'
+    kill $(cat ~/erp-tune/run-03c.pid)
+
+⚠ **Never `pkill -f erp_sft_harness` over SSH.** The pattern appears in your own
+ssh argv, so it kills your shell as well as the run. This bit again during
+staging. Kill by PID. The launcher's own guard uses the pidfile for the same
+reason.
+
+## What to expect in the log
+
+Order is preflight → encode (cache hit) → mix report → weight load (~2 min) →
+sampler → train. Checkpoints land every 50 steps at ~852 MB each, so ~10 GB over
+the run against 764 GB free. First checkpoint ≈ 66 minutes in.
+
+## Not verified
+
+The harness's train loop has not been exercised end to end on this silicon — the
+throughput baseline used a synthetic replica of the geometry, and the staging
+encode was killed before the weight load. If something is going to break, it
+breaks in the first two minutes after the sampler line.

persistent-memory.d/2026-09-03-gx10-run3c-staged.md

@@ -0,0 +1,91 @@
+# `[2026-09-03]` Run 3c STAGED on pfi-gx10 — verified end to end, deliberately NOT launched
+
+The ERP-seat SFT LoRA that died on ana-ml2 at step 24 of 604 to an Anaheim breaker trip is
+now staged on pfi-gx10, unchanged. **The launch is the operator's call and was not taken** —
+he stood this port down once before, so a 13.3 h commitment is not an agent default.
+
+Runbook `docs/runbooks/gx10-run-03c.md`; canonical config + launcher
+`scripts/erp-tune-gx10/`; on the box `/home/infra-ops/erp-tune/`.
+
+    ssh infra-ops@10.100.50.60 '~/erp-tune/launch-run-03c.sh'
+
+## What is on the box
+
+    ~/models/gemma4-26b-a4b-it-bf16   49 GB base, ALREADY THERE from the 09-01 probe
+    ~/erp-tune/eitri-smithy           harness, git 0a6bd2e, tracked tree clean
+    ~/erp-tune/recipe-r3              recipe / survivors / loss-mask
+    ~/erp-tune/datasets/{derived,holdout}   2.4 GB, COPIED (50 s at 49 MB/s from nh3-dev)
+    ~/erp-tune/run-03c/encode-cache   PRE-SEEDED with the verified encode
+    ~/ml/.venv                        + protobuf, pytest (the only two gaps vs ana-ml2)
+
+⚠ **The corpus is copied and the box mounts NO NFS.** `/mnt/smithy` lives on nh3-nas, now on
+the *same subnet* as the racked GX10 — which makes mounting it tempting and still wrong. A
+13 h unattended run is the worst place for a hard NFS dependency
+([[incident_esh_docker_nfs_boot_race]]). 2.4 GB copies in under a minute; there is nothing to
+buy.
+
+## The verification that actually mattered — and it was NOT free reasoning
+
+ana-ml2 ran transformers 5.15.1 / torch 2.13.0 on x86-64. The GX10 runs 5.16.1 / 2.14.0+cu130
+on aarch64. That is precisely the silent backend-delta class CLAUDE.md records as having voided
+two frontier-panel conclusions. So it was **measured**: a full encode was run into a throwaway
+output dir and the encoded corpus compared byte-for-byte.
+
+    ana-ml2   encoded-c16316f1c1bb21da.jsonl   197,360,233 B   sha256 c08bb1fe2ecb0be3...
+    pfi-gx10  encoded-fd8fe1944fb316b2.jsonl   197,360,233 B   sha256 c08bb1fe2ecb0be3...
+
+**Byte-identical.** Every aggregate matched too: 9,504 vs 8,404 ids / 0 overlap, 15 unfittable
+dropped, 9,662 records, ctx 18,600,057 / loss 13,310,930 tok, five mix shares to 4 dp.
+
+⚠ **The cache-key FILENAMES differ and that is correct, not drift.** `base_model_path` is in
+the encode-cache key *by design* (so a different base cannot silently reuse an encode), and
+rehoming the base changes the key while leaving content identical. **The key is an input hash;
+the sha is the output.** Do not read the differing filenames as a mismatch — and do not
+"fix" it by symlinking `/tank/aimodels` onto this box to force a key match. That verified
+artifact was then copied into `run-03c/encode-cache/`, so the run trains on the exact bytes
+compared and will report `[encode] cache hit`.
+
+Also verified rather than assumed: **both 49 GB base shards sha256-match ana-ml2's** (size
+equality was already true and is not the same claim), the harness's own suite is **122 passed**
+on aarch64, and every one of the config's 8 path keys resolves to an existing local file.
+
+## The config is provably the same run
+
+`run-03c-gx10.json` = ana-ml2's `run-03c.json` with 8 path keys rehomed and 2
+`substitute_controls` entries appended (host move; library delta). A generator asserted
+**key-by-key that no non-path value differs** rather than eyeballing a diff — lr 1e-05, rank 64,
+alpha 128, seq 16384, batch 2 x accum 8, save_steps 50, seed 20260824 all intact, and the
+existing 10 substitute_controls are a byte-identical prefix.
+
+## ⚠ I TRIPPED THE pkill SELF-MATCH AGAIN, ~20 MINUTES AFTER READING THE MEMORY ABOUT IT
+
+`ssh gx10 'pkill -f "erp_sft_harness --config .../encode-check.json"'` — the pattern is in the
+remote shell's OWN argv, so it killed my shell alongside the target and the command returned
+nothing. [[feedback_pkill_ssh_self_match]] describes this exactly. Reading the memory did not
+prevent it; **the guard has to be in the artifact, not in recall.**
+
+So the launcher's already-running guard is a **pidfile**, not a pgrep — `pgrep -f
+erp_sft_harness` in a script invoked over ssh matches the invoking shell and would refuse every
+launch. Same root cause, and it would have presented as a mysterious always-refusing launcher.
+
+## The launcher's other guards, each bought with a past failure
+
+    GPU-clear assertion      a stuck orphan held 80 GB while PyTorch reported 0 allocated;
+                             every relaunch was doomed and blamed the NEW run
+    setsid nohup + on-box log a foreground ssh reaped the 09-01 probe: work survived, output did not
+    log-exists refusal       two runs must not share a log
+    >=40 GB free             12 checkpoints x 852 MB (measured off run-03, not estimated)
+
+## Why the slow box is still the right box (unchanged, restated because it is the whole case)
+
+~79.4 s/it here vs 10.8-15.8 on ana-ml2 -> 13.3 h vs ~2.5 h. An Anaheim breaker trip is not
+priced in lost steps: it is a 40-minute drive **each way** on the operator's time, 13 hosts
+down including `pbs-ana` and **three SureFire client machines**. Nothing is waiting on this run,
+so the slowness is close to free.
+
+## NOT verified — the honest gap
+
+The harness's **train loop** has not run end to end on sm_121. The 79.4 s/it baseline used a
+synthetic replica of the geometry, and the staging encode was killed before the weight load.
+If it breaks, it breaks in the first two minutes after the `[sampler]` line — roughly three
+minutes after launch, well before the first checkpoint at ~66 min.

persistent-memory.md

@@ -112,8 +112,13 @@ _As of 2026-09-03 — **nothing is mid-action; the session closed clean.** Every
 item below is a live commitment or a known-open risk, not work in progress._
 
 - **Nothing is running.** No training, no deploys pending, no background jobs.
-  35 commits sit unpushed on `main` — all docs, runbooks, memory and two
+  Commits sit unpushed on `main` — all docs, runbooks, memory and scripts;
-  scripts; push is the operator's call.
+  push is the operator's call.
+
+- **Run 3c is staged on pfi-gx10 and awaiting the operator's go.** Everything is
+  verified and one command away (`ssh infra-ops@10.100.50.60
+  '~/erp-tune/launch-run-03c.sh'`); ~13.3 h once started. Its train loop is the
+  one piece never exercised on sm_121 — watch the first three minutes.
 
 - **`web_search` needs a session restart to appear.** The SearXNG MCP server is
   registered at user scope and `claude mcp list` reports it Connected, but MCP
@@ -145,6 +150,7 @@ item below is a live commitment or a known-open risk, not work in progress._
 
 ## Recent decisions
 
+- `[2026-09-03]` **Run 3c is STAGED on pfi-gx10 and deliberately NOT launched** — the launch is a 13.3 h commitment and the operator stood this port down once already. Base shards AND the encoded corpus sha256-verified identical to ana-ml2's, so the transformers 5.15.1→5.16.1 / x86-64→aarch64 delta is *measured* inert. ⚠ The encode-cache FILENAME differs by design (`base_model_path` is in the key) — input hash, not output. ⚠ Tripped the `pkill -f` ssh self-match again; the launcher guards on a pidfile because of it. → `persistent-memory.d/2026-09-03-gx10-run3c-staged.md`
 - `[2026-09-03]` **SearXNG returned ZERO results for every query while reporting `healthy` for 7 days — 4.5 months stale.** Moved to nh3-docker (residential egress beats the colo's CAPTCHA-gated 38.120.12.42), updated, and exposed to every CC session as the user-scope `web_search` MCP tool. ⚠ `/healthz` cannot tell you whether search works. → `persistent-memory.d/2026-09-03-searxng-nh3-move.md`
 - `[2026-09-03]` **pfi-gx10 racked: VLAN 50 via a DHCP RESERVATION on the UDM, not a host static — operator ruling, so the box stays portable.** ⚠ The racked port arrived on the NATIVE VLAN; ⚠ `port_overrides` is a whole-array PUT; ⚠ prove inter-VLAN routing with `ping -I <wired>` BEFORE downing the Wi-Fi escape hatch. Now single-path. → `persistent-memory.d/2026-09-03-gx10-rack-network.md`
 - `[2026-09-03]` **Three Macs onboarded (mini / Air / Studio) with infra-ops, NOPASSWD sudo, rotated+vaulted passwords and `dsh` on device-scoped keys — and the fourth is `scripts/provision-mac-dsh.sh`, not a fourth hand-run.** ⚠ `sudo -u` keeps the CALLER's `$HOME` and nearly wiped a working install; ⚠ a wrong USERNAME is indistinguishable from a wrong password. → `persistent-memory.d/2026-09-03-mac-fleet-dsh.md`

scripts/erp-tune-gx10/README.md

@@ -0,0 +1,20 @@
+# erp-tune on pfi-gx10
+
+Canonical copies of what is staged on pfi-gx10 for ERP-seat SFT **run 3c** —
+the LoRA that died on ana-ml2 at step 24 of 604 when an Anaheim breaker tripped.
+
+| file | deployed to |
+|---|---|
+| `run-03c-gx10.json` | `/home/infra-ops/erp-tune/run-03c-gx10.json` |
+| `launch-run-03c.sh` | `/home/infra-ops/erp-tune/launch-run-03c.sh` |
+
+`run-03c-gx10.json` is `ana-ml2:/tank/erp-tune/run-03c.json` with its eight path
+keys rehomed to local NVMe and two `substitute_controls` entries appended
+recording the host move and the library delta. **No hyperparameter, corpus,
+base, mask or recipe value differs** — verified key-by-key at generation.
+
+Full context, what was verified and how to launch:
+[`docs/runbooks/gx10-run-03c.md`](../../docs/runbooks/gx10-run-03c.md).
+
+The harness itself (`eitri-smithy`) is not vendored here; it lives on the box at
+`/home/infra-ops/erp-tune/eitri-smithy`, git `0a6bd2e`.

scripts/erp-tune-gx10/launch-run-03c.sh

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# Launch ERP-seat SFT run 3c on pfi-gx10 (NVIDIA GB10, aarch64, sm_121).
+#
+# Run this ON pfi-gx10 as infra-ops. It detaches the job from the invoking
+# shell and logs to the box, so a reaped SSH session cannot take the run with
+# it -- the failure mode that lost the first probe launch on 2026-09-01.
+#
+# Expected: 604 optimizer steps at ~79.4 s/it => ~13.3 h.
+# Checkpoints every 50 steps, ~852 MB each (~10 GB total).
+set -euo pipefail
+
+ROOT=/home/infra-ops/erp-tune
+HARNESS=$ROOT/eitri-smithy
+VENV=/home/infra-ops/ml/.venv/bin/python
+CONFIG=$ROOT/run-03c-gx10.json
+LOG=$ROOT/run-03c.log
+
+# --- Preconditions, asserted rather than assumed -----------------------------
+
+# A stuck orphan holding unified memory while PyTorch reports zero allocated
+# already doomed three relaunches on this box and got blamed on the new run
+# each time. Assert the GPU is clear.
+apps=$(nvidia-smi --query-compute-apps=pid --format=csv,noheader | tr -d '[:space:]')
+if [ -n "$apps" ]; then
+  echo "REFUSING: GPU is not clear -- compute apps still resident:" >&2
+  nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv >&2
+  exit 1
+fi
+
+# Deliberately NOT `pgrep -f erp_sft_harness`: run this over ssh and the
+# pattern appears in the invoking shell's own argv, so the guard matches
+# itself and refuses every launch. Same self-match that makes `pkill -f`
+# unsafe over ssh. The pidfile is exact and cannot self-match; the GPU
+# assertion above catches an orphan under any name.
+if [ -f "$ROOT/run-03c.pid" ] && kill -0 "$(cat "$ROOT/run-03c.pid")" 2>/dev/null; then
+  echo "REFUSING: run-03c.pid names a live process $(cat "$ROOT/run-03c.pid"):" >&2
+  ps -p "$(cat "$ROOT/run-03c.pid")" -o pid,etime,cmd >&2
+  exit 1
+fi
+
+if [ -e "$LOG" ]; then
+  echo "REFUSING: $LOG exists. Move it aside first so two runs cannot share a log." >&2
+  exit 1
+fi
+
+for p in "$HARNESS/erp_sft_harness/__main__.py" "$VENV" "$CONFIG"; do
+  [ -e "$p" ] || { echo "REFUSING: missing $p" >&2; exit 1; }
+done
+
+# Free space for checkpoints: 12 x 852 MB + final adapter, with headroom.
+avail=$(df --output=avail -BG "$ROOT" | tail -1 | tr -dc '0-9')
+if [ "$avail" -lt 40 ]; then
+  echo "REFUSING: only ${avail}G free under $ROOT; want >=40G for checkpoints." >&2
+  exit 1
+fi
+
+# --- Launch ------------------------------------------------------------------
+
+cd "$HARNESS"
+{
+  echo "# launched $(date -Is) on $(hostname) by ${USER}"
+  echo "# harness $(git rev-parse --short HEAD) config $CONFIG"
+} > "$LOG"
+
+setsid nohup "$VENV" -m erp_sft_harness --config "$CONFIG" >> "$LOG" 2>&1 < /dev/null &
+pid=$!
+echo "$pid" > "$ROOT/run-03c.pid"
+
+echo "launched pid $pid -> $LOG"
+echo
+echo "watch:   tail -f $LOG | tr '\\r' '\\n'"
+echo "steps:   grep -ao '[0-9]*/604 \[[^]]*\]' $LOG | tail -1"
+echo "stop:    kill \$(cat $ROOT/run-03c.pid)      # by PID -- never pkill -f over ssh"

scripts/erp-tune-gx10/run-03c-gx10.json

@@ -0,0 +1,47 @@
+{
+  "output_dir": "/home/infra-ops/erp-tune/run-03c",
+  "roots_dir": "/home/infra-ops/erp-tune/datasets/derived",
+  "base_model_path": "/home/infra-ops/models/gemma4-26b-a4b-it-bf16",
+  "base_model_revision": "google/gemma-4-26B-A4B-it (stock instruct, upstream) -- UNCHANGED from runs 2 and 3; run 3c varies the LEARNING RATE alone",
+  "recipe": "/home/infra-ops/erp-tune/recipe-r3/recipe-erp-seat-sft-r3.json",
+  "survivors": "/home/infra-ops/erp-tune/recipe-r3/survivors-r3.jsonl",
+  "chat_template_path": "/home/infra-ops/models/gemma4-26b-a4b-it-bf16/chat_template.jinja",
+  "impersonation_mask_path": "/home/infra-ops/erp-tune/recipe-r3/lossmask-r3.jsonl",
+  "lora_rank": 64,
+  "lora_alpha": 128,
+  "lora_dropout": 0.0,
+  "max_seq_len": 16384,
+  "epochs": 1,
+  "seed": 20260824,
+  "per_device_batch_size": 2,
+  "gradient_accumulation_steps": 8,
+  "learning_rate": 1e-05,
+  "warmup_ratio": 0.1,
+  "lr_scheduler_type": "cosine",
+  "weight_decay": 0.01,
+  "load_in_4bit": false,
+  "gradient_checkpointing": true,
+  "loss_chunk_tokens": 1024,
+  "training_eligibility_override": "operator-2026-08-26-run3-go",
+  "overridden_blockers": [
+    "contamination-scan-not-implemented",
+    "stage-2-csam-detector-inert"
+  ],
+  "substitute_controls": [
+    "pre-training holdout (8,002 samples, work/card/session split)",
+    "stage-A lexical quarantine, RP only (829 records held)",
+    "F1 pippa root excision (target-list change, recipe erp-seat-sft-r3)",
+    "F2/F3/F4 low-quality loss-mask, 5,867 turns (length floor 250w, register cap 20 asterisks/1kw, placeholder leak) -- MASKED not deleted",
+    "impersonation loss-mask unioned into the same file, 16 turns not covered by F2/F3/F4; 5,883 masked turns total, sha-asserted",
+    "kvasir subset to the recipe's stated 38.0% context share (3,498 of 12,003, seeded shuffle at run seed) -- the recipe names the share but stages no selection; see recipe-r3/RUN-03-BUILD-NOTE.md",
+    "DEVIATION RECORDED: recipe token accounting is not the harness tokenizer; bluemoon is NOT the largest loss contributor as trained (32.9% vs c2-logs 35.6%) -- see RUN-03-BUILD-NOTE.md section 4",
+    "SINGLE VARIABLE vs run 3: learning_rate 0.0002 -> 1e-05 (20x cut). Corpus BYTE-IDENTICAL (survivors-r3.jsonl, lossmask-r3.jsonl, same shas); base, max_seq_len 16384, rank 64, alpha 128, dropout 0.0, cosine, warmup 0.1, wd 0.01, batch 2, 1 epoch and target_modules ALL UNCHANGED. Rationale: the zerofata MeroMero v1 card, on this exact base, warns \"Google left the instruct model at the razor's edge of overfitting; finetune it at all and it feels like it'll rapidly lose intelligence\" -- and MeroMero v1 trains at 1e-5. Run 3 used 20x that.",
+    "OPERATIONAL DEVIATION (not scientific): save_steps 100 -> 50. Checkpoint cadence only; it changes nothing about the optimisation, the corpus or the single-variable lr claim above. Recorded as a separate entry so the two deviations cannot be confused. Rationale: an Anaheim site power event on 2026-08-27 killed this run's first launch at step 80 of 604 with save_steps=100, so no checkpoint existed and it restarted from zero.",
+    "relaunch_reason: Anaheim site power event 2026-08-27T01:2x UTC, confirmed by a 1-minute uptime on the host's return. First launch reached step 80 of 604 with save_steps=100, so NO checkpoint existed and this run starts from zero. Configured identically apart from save_steps; NOT a resumption.",
+    "HOST MOVE (operational, not scientific): this run executes on pfi-gx10 (NVIDIA GB10, aarch64, sm_121, 121 GB unified) instead of ana-ml2. Paths are rehomed to local NVMe; NO hyperparameter, corpus, base, mask or recipe value differs from run-03c.json. Base model verified sha256-identical to ana-ml2's copy (both shards, 2026-09-03). Rationale: an Anaheim breaker trip already killed this run once and costs a 40-minute drive plus 13 hosts down, three of them SureFire client machines; the GX10 is a ~240 W appliance at NH3 that can take nothing else down. Measured cost: ~79.4 s/it vs ana-ml2's 10.8-15.8, so 604 steps is ~13.3 h instead of ~2.5 h.",
+    "LIBRARY DELTA (recorded, not assumed harmless): torch 2.13.0 -> 2.14.0+cu130, transformers 5.15.1 -> 5.16.1, triton 3.7.1 -> 3.8.0; peft 0.20.0 and accelerate 1.14.0 unchanged. Attention backend flex_attention on both sides. Tokenisation equivalence across the transformers delta is asserted by comparing this host's encoded corpus against ana-ml2's byte-for-byte before launch, not inferred."
+  ],
+  "unfittable": "drop",
+  "holdout_dir": "/home/infra-ops/erp-tune/datasets/holdout",
+  "save_steps": 50
+}

servers/pfi-gx10/README.md

@@ -11,33 +11,25 @@ Grace-Blackwell desktop supercomputer. Registered 2026-09-01.
 | Kernel | 6.17.0-1031-nvidia |
 | Hostname | `pfi-gx10` (shipped with static `gx10-a745`, corrected) |
 
-## ⚠ The address in `ssh-target` is TEMPORARY
+## Network — racked, and single-path
 
-As of 2026-09-01 this box is **on a desk, on Wi-Fi**, holding a DHCP lease at
+Racked 2026-09-03. `pfi-gx10.nh3.internal` → **10.100.50.60**, wired only on
-`10.100.10.226` on `nh3-userland` (VLAN 10). Ethernet `enP7s7` has no carrier.
+`enP7s7`, VLAN 50 (`nh3-servers`), UniFi switch port 22.
 
-It is going into the rack later. **Target settled (operator, 2026-09-01):
+**The address lives on the switch side, not the host** — a DHCP *reservation*
-`nh3-servers`, VLAN 50, static `10.100.50.60`** — clear of `.40` nh3-docker,
+against the wired MAC `30:c5:99:3d:a7:45`, with the host left on DHCP. Operator
-`.42` nh3-extdev, `.50` nh3-nas, `.90` pbs-nh3, and below the `.150` DHCP pool
+ruling: a reservation moves with the box, a netplan static goes stale the moment
-where fleet statics live.
+it does.
 
-Nothing was configured on the desk — an address that is about to be wrong is
+⚠ **Wi-Fi is deliberately off and there is now exactly ONE path in.** If the
-worse than DHCP. The move is `playbooks/gx10-rack-network.yaml`.
+switch port or the reservation breaks, this is a rack visit. Correct end state
+for a racked server, but it is a posture change from the desk setup.
 
-**Nothing is needed from the operator beyond racking it.** The wired NIC has its
+Full detail, including the order that made the move safe:
-own MAC (`30:c5:99:3d:a7:45`, distinct from the Wi-Fi `50:bb:b5:a2:00:a8`), so
+[`docs/runbooks/gx10-rack-network.md`](../../docs/runbooks/gx10-rack-network.md).
-its post-move address is discoverable from the UDM without being told:
 
-```bash
+⚠ **The box mounts no NFS, on purpose.** Working data is copied to local NVMe —
-curl -sk "https://10.100.0.1/proxy/network/api/s/default/stat/sta" \
+see the training section below.
-  -H "X-API-KEY: $(secret get unifi/pfi-udmse-api-key)" \
-  | python3 -c "import json,sys;[print(c['ip'],c.get('sw_port')) for c in json.load(sys.stdin)['data'] if c['mac']=='30:c5:99:3d:a7:45']"
-```
-
-That also returns the switch port, which must be set to the `nh3-servers`
-network or the box lands back on VLAN 10.
-
-Expect ~65–80 ms RTT until it is wired. That is Wi-Fi power-save, not a fault.
 
 ## Access
 
@@ -62,6 +54,27 @@ trivially while the desktop is still running.
 The playbook will not stop GDM while someone holds a seat session. Override with
 `--var force_dm_stop=true`, or just let the rack-install reboot handle it.
 
+## Training — run 3c is staged and ready
+
+The ERP-seat SFT LoRA (run 3c) that died on ana-ml2 at step 24 of 604 to an
+Anaheim breaker trip is staged here, unchanged, and **not launched** — that call
+is the operator's.
+
+    ssh infra-ops@10.100.50.60 '~/erp-tune/launch-run-03c.sh'
+
+~79.4 s/it measured on this geometry → 604 steps ≈ 13.3 h, peak 75.1 of
+121.6 GiB. Slower than ana-ml2's ~2.5 h and still the right host: this is a
+~240 W appliance at NH3 that cannot take a client's machines dark.
+
+Base model and the encoded corpus were both sha256-verified identical to
+ana-ml2's, so the library delta (transformers 5.15.1 → 5.16.1, x86-64 → aarch64)
+is measured to be inert rather than assumed harmless. Runbook:
+[`docs/runbooks/gx10-run-03c.md`](../../docs/runbooks/gx10-run-03c.md); canonical
+config + launcher in [`scripts/erp-tune-gx10/`](../../scripts/erp-tune-gx10/).
+
+⚠ **Never `pkill -f erp_sft_harness` over SSH** — the pattern is in your own ssh
+argv and you kill your shell with it. Kill by PID from `~/erp-tune/run-03c.pid`.
+
 ## Relevance to Flash-Next
 
 `sm_121`, not `sm_120`. The SGLang fork evaluated for ana-ml2 (henge item 49)