memory: snapshot — run-3c probe in flight; ana-ml2 baseline reduced from its log

The comparison number nobody had written down: ana-ml2's real run-3c step
times, pulled out of run-03c.log before the breaker killed it. About 10.8
to 15.8 s/it over the first 24 steps, so 604 steps lands at roughly 2.2
to 2.7 hours. Anything under about 45 s/it on the GX10 makes it an
overnight run.

Also records the exact geometry from run-03c.json and a real
adapter_config.json, so the probe measures the shape that actually ran
rather than an approximation of it.

Checked the backend-delta trap the playbook warns about before running
anything rather than after: flash-attn is installed on neither box, so
both fall back to sdpa. Library versions do differ -- torch 2.13.0 versus
2.14.0, transformers 5.15.1 versus 5.16.1 -- and that is recorded rather
than assumed harmless. The probe reads the resolved attention
implementation back off the loaded model instead of trusting the request.

The probe discards its first two steps as warmup, which is not optional on
this box: an unwarmed benchmark here already read 27 TFLOP/s when the true
figure was 93, because it was timing the PTX JIT.

persistent-memory.d/2026-09-01-pfi-gx10-onboarding.md

@@ -253,3 +253,62 @@ Still outstanding, all needing root:
     - racking (playbooks/gx10-rack-network.yaml) -- optional, NOT a prerequisite
 
 **Nothing above blocks the run-3c throughput probe**, which needs only the driver and torch.
+
+## `[2026-09-01 23:5x]` Run-3c throughput probe — IN FLIGHT, and the ana-ml2 baseline is now on record
+
+### The baseline nobody had written down: ana-ml2's real s/it for run 3c
+
+Extracted from `/tank/erp-tune/run-03c.log` (the actual run, before the breaker killed it):
+
+    step 10   10.81 s/it       step 20   12.88 s/it       step 24   15.76 s/it
+    => 604 steps on ana-ml2 ~= 2.2-2.7 h
+
+**This is the number the GX10 has to be compared against**, and it was sitting in a log
+nobody had reduced. Anything under ~45 s/it on the GX10 makes 604 steps an overnight run.
+
+### Exact run-3c geometry (from run-03c.json + a real adapter_config.json)
+
+    base            /tank/aimodels/gemma4-26b-a4b-it-bf16   49 GB
+                    gemma4, 30 layers, hidden 2816, 16 heads / 8 KV, MoE (A4B),
+                    vocab 262144, sliding_window 1024, multimodal (vision+audio configs)
+    LoRA            r 64, alpha 128, dropout 0.0, bias none, 205 target modules
+                    (q/k/v/o + gate/up/down across all 30 language-model layers)
+    shape           max_seq_len 16384, per_device_batch 2, grad_accum 8 (effective 16)
+    other           gradient_checkpointing true, bf16, cosine, warmup 0.1, wd 0.01
+    length          604 optimizer steps, 1 epoch, save_steps 50
+
+### ⚠ THE BACKEND-DELTA TRAP — CHECKED, and it is NOT present
+
+CLAUDE.md warns that two frontier-panel conclusions were voided because a benchmark and a
+trainer had silently different attention backends. Enumerated both sides **before** running:
+
+    ana-ml2   flash_attn NOT installed in /tank/erp-tune/venv  -> falls back to sdpa
+    pfi-gx10  flash_attn not available for aarch64/sm_121      -> falls back to sdpa
+
+Same backend both sides. ⚠ **Library versions DO differ and it is recorded rather than
+assumed harmless:**
+
+    ana-ml2   torch 2.13.0        transformers 5.15.1   peft 0.20.0   accelerate 1.14.0
+    pfi-gx10  torch 2.14.0+cu130  transformers 5.16.1   peft 0.20.0   accelerate 1.14.0
+
+The probe reads the **resolved** `_attn_implementation` back off the loaded model rather than
+trusting the request, so the comparison is checkable rather than assumed.
+
+### Probe design (§4-disciplined)
+
+`~/ml/probe_3c.py` on the GX10. Real geometry, synthetic batches — it measures step cost, and
+padding on the real run is 0.0% so the corpus does not change per-step cost materially.
+
+- **12 steps, first 2 discarded as warmup.** Non-negotiable here: an unwarmed benchmark on
+  this box already read 27 TFLOP/s when the true figure was 93, because it was timing the PTX
+  JIT. sm_121 is not in torch's arch list.
+- Reports the **step-time distribution** (median, min/max, every step), not a mean beside a
+  settings dump.
+- Emits **every provenance key always, null when unused** — present-and-null, not absent.
+
+### Transfer
+
+49 GB relayed **ana-ml2 -> nh3-dev -> pfi-gx10**, ~33 MB/s, ~25 min. Relayed rather than
+direct because ana-ml2's key is not authorised for `infra-ops@pfi-gx10` — and relaying costs
+nothing extra, since the Wi-Fi leg is the bottleneck either way. **Deliberately did not add an
+access grant just to save a hop.**