From 265357efb2cb95440c58ee2ebe317401a64d18ed Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 7 Sep 2026 09:33:03 -0700 Subject: [PATCH] ops(erp-tune): stage + launch R47 ERP-seat SFT run 5 on pfi-gx10 Run 5 = the dependency-forcing corpus arm: airoboros-3.2 OUT of the 20% slot, govreport/clean-v1 (496) + qmsum/clean-v1 (97) IN, at run 4's lr 2e-04 with everything else held. kvasir byte-identical (survivors-r5 = survivors-r4 minus airoboros plus the two new roots whole). Operator authorized the launch to infra-ops directly; grant operator-2026-09-07-rnd-run5. Canonical copies of the config, launcher and survivors builder; runbook docs/runbooks/gx10-run-05.md. Launch gates all passed (7/7 root shas + shard hashes, survivor join 8,212 = recipe, holdout disjoint, window_count==1 on all 593 slot rows, realized [mix] slot loss 3.46% vs preregistered 3.4%). 524 steps. persistent-memory current-state updated: run 5 LAUNCHED + training. --- docs/runbooks/gx10-run-05.md | 89 +++++++++++++++++++++ persistent-memory.md | 24 +++--- scripts/erp-tune-gx10/README.md | 47 +++++++---- scripts/erp-tune-gx10/build_r5_survivors.py | 47 +++++++++++ scripts/erp-tune-gx10/launch-run-05.sh | 72 +++++++++++++++++ scripts/erp-tune-gx10/run-05-gx10.json | 45 +++++++++++ 6 files changed, 299 insertions(+), 25 deletions(-) create mode 100644 docs/runbooks/gx10-run-05.md create mode 100644 scripts/erp-tune-gx10/build_r5_survivors.py create mode 100755 scripts/erp-tune-gx10/launch-run-05.sh create mode 100644 scripts/erp-tune-gx10/run-05-gx10.json diff --git a/docs/runbooks/gx10-run-05.md b/docs/runbooks/gx10-run-05.md new file mode 100644 index 0000000..94f7a25 --- /dev/null +++ b/docs/runbooks/gx10-run-05.md @@ -0,0 +1,89 @@ +# pfi-gx10 — ERP-seat SFT run 5 (dependency-forcing slot) + +Launched 2026-09-07 09:05 PDT on pfi-gx10, pid 3311780. Operator authorized the +launch to infra-ops directly (the run-scoped training-eligibility override +`operator-2026-09-07-rnd-run5` was granted to Brokkr; the relay caveat was +closed with the operator directly in-session before launch, as it was for run 4). + +## What run 5 is + +The **third** R47 corpus-gate run. Single variable vs run 4: the 20% context +slot's **composition**. Run 4 put an airoboros-3.2 instruct root in that slot; +run 5 removes it and puts in two **dependency-forcing** roots — +`govreport/clean-v1` (496 reports) + `qmsum/clean-v1` (97 transcripts), where +the loss turn is determined by distant context (document first, query/instruction +last). Everything else is held from run 4: dialogue survivors and the +impersonation loss-mask byte-identical, fireball whole, kvasir at run 4's exact +1,613-sample prefix cut (NOT re-cut), base, lr 2e-04, max_seq_len 16384, rank 64, +alpha 128, batch 2 × accum 8, cosine, warmup 0.1, wd 0.01. + +The slot's ctx share reproduces run 4's 20%, but its **loss** share is 3.4% (run +4's was 20.6%) — preregistered in `SFT-RECIPE-run5-SCOPE.md §4`. **INERT is a +live verdict**, not a build error. + +Machine-readable source of truth: `brokkr-smithy` +`research/R47-premium-corpus-gate/recipe-erp-seat-sft-r5.json` (f49994b, tag R49.4). + +## Layout on the box + + /home/infra-ops/models/gemma4-26b-a4b-it-bf16 49 GB base (byte-identical to run 4) + /home/infra-ops/erp-tune/ + eitri-smithy/ harness, git 0a6bd2e + recipe-r5/ + recipe-erp-seat-sft-r5.json from brokkr-smithy, verbatim + survivors-r5.jsonl 8,212 rows, sha256 a25169a6... + datasets/derived/govreport/clean-v1/ copied from /mnt/smithy (no NFS on box) + datasets/derived/qmsum/clean-v1/ copied from /mnt/smithy + datasets/holdout/ run-1 8,404-sample holdout (disjointness check) + run-05-gx10.json the config + launch-run-05.sh the launcher (pidfile-guarded, GPU-clear-asserting) + build_r5_survivors.py how survivors-r5 was built + run-05/ output: adapter/, checkpoints/, encode-cache/, truncation-report.json + +Canonical copies of the config, launcher and survivors builder are in this repo +under `scripts/erp-tune-gx10/`. `survivors-r5 = survivors-r4 minus airoboros +plus govreport + qmsum whole` — deriving from run 4's survivors is what keeps +kvasir's cut byte-identical. + +## Gates that passed at launch (all before / at encode, no GPU sunk on a bad build) + +- **Preflight** (also run standalone, no GPU): 7/7 root shas verified, each shard + re-hashed against its CLEANROOT (govreport `d533a5cd`, qmsum `c0269d69`, + byte-identical after the NFS→box copy). Survivor→root join = 8,212 samples, + composition exactly the recipe (c2-logs 301 / cwm 183 / bluemoon 126 / + fireball 5396 / kvasir 1613 / govreport 496 / qmsum 97). +- **Holdout disjoint:** 8,212 training vs 8,404 holdout ids, 0 overlap. +- **window_count belt-and-suspenders** (`SFT-RECIPE-run5-SCOPE.md §7.1`): every + slot row must render window_count == 1. `truncation-report.json` showed + **govreport fit_whole=496 / chunked=0 / trunc=0** and **qmsum fit_whole=97 / + chunked=0 / trunc=0**. Any chunked/truncated slot row is a BUILD DEFECT → kill + before training. It passed. +- **Realized `[mix]` loss shares** reproduce the recipe's preregistered shares to + <0.1%: kvasir 0.3369, fireball 0.3042, c2-logs 0.1153, bluemoon 0.1066, + cwm 0.1022, govreport 0.0291, qmsum 0.0055. Slot subtotal 3.46% (pred 3.4%), + slot ctx 0.20. + +Encode: 8,197 samples → 8,370 records (15 unfittable dropped, same as run 4), +ctx 18,598,779 / loss 9,935,076 tok. **524 optimizer steps** (÷ effective batch +16). ~52 s/it on this box → ~7.5 h. Checkpoints every 50 steps. + +## Launch / watch / stop + + ssh infra-ops@10.100.50.60 '~/erp-tune/launch-run-05.sh' + ssh infra-ops@10.100.50.60 "tr '\r' '\n' < ~/erp-tune/run-05.log | tail" + ssh infra-ops@10.100.50.60 'kill $(cat ~/erp-tune/run-05.pid)' # by PID — never pkill -f over ssh + +The launcher refuses if the GPU is not clear, if `run-05.pid` names a live +process, if `run-05.log` exists, if any input is missing, or if under 40 GB free. +It detaches with `setsid nohup` and logs on the box. + +## After the adapter lands — the gate choreography (with brokkr-smithy-dev) + +Same as run 4, coordinated on the althing **cc channel** (Brokkr is armed there): +serve the **base** on this box/stack → Brokkr locks `FLOOR-LOCKED.json` and has +already committed the four-cell verdict evaluator (`tools/verdict_run05.py`: +RESCUED / STILL-COUPLED / OVER-CORRECTED / INERT) before any tuned figure exists +→ send the **swap cue** → serve the **tuned** artifact. T4 is the headline +must-not-harm (per-item, never pooled); refusal is telemetry; CSAM measurement +fail-closed stays. The override id + both blockers + all substitute controls are +in `run-05-gx10.json`, so they ride into the adapter's provenance manifest. diff --git a/persistent-memory.md b/persistent-memory.md index 56e9674..0ad5a09 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -1,6 +1,6 @@ # Persistent memory — eshpfi-management -_Last updated: 2026-09-07 (fleet-ops: irv DNS, ESH outage recovery, backup router-exclusions, booth feature, run-4 gate coord, cc-channel, /mnt/smithy remount, fleet-TLS pattern)_ +_Last updated: 2026-09-07 (fleet-ops: ERP run-5 staged + launched on pfi-gx10, irv DNS, ESH outage recovery, backup router-exclusions, booth feature, run-4 gate coord, cc-channel, /mnt/smithy remount, fleet-TLS pattern)_ > **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its > `Written:` stamp is under an hour old, read it (it carries the in-flight @@ -109,15 +109,21 @@ no longer deployed sidecars here. See Recent decisions.) ## Current state / in-flight _As of 2026-09-07 (fleet-ops session). SUPERSEDES the "run 4 is TRAINING" framing lower down: -run 4 is DONE (gated STILL-COUPLED); run 5 is AUTHORIZED but stalled. Live open items:_ +run 4 is DONE (gated STILL-COUPLED); run 5 is LAUNCHED and TRAINING (09:05 PDT). Live open items:_ -- **⚠ ERP run 5 authorized but NOT running — brokkr sat on it; a full GX10 window wasted - (operator, 2026-09-07).** Operator already gave the third eligibility grant and thought it was - training; brokkr-smithy-dev never built the clean roots / `recipe-r5.json` / launched. GX10 idle - (0% GPU) since the run-4 gate. Raw sources ARE staged (nh3-nas `/volume1/smithy/datasets/raw/{ccdv__govreport-summarization,pszemraj__qmsum-cleaned}`, ~09-06 23:06); clean-v1 derived roots + - `recipe-r5.json` + launch still owed by brokkr. Run 5 = dependency-forcing slot (GovReport+QMSum - replace airoboros; single variable; tests whether long-context-dense data breaks the diversity↔coherence - coupling run 4 left STILL-COUPLED). **FOLLOW-UP: nudge brokkr to build+launch — don't waste the next window.** +- **✅ ERP run 5 LAUNCHED + training on pfi-gx10 (2026-09-07 09:05 PDT, pid 3311780).** Operator + said "yes go, launch run 5" directly to infra-ops (relay caveat on brokkr's third grant + `operator-2026-09-07-rnd-run5` closed in-session, as for run 4). Brokkr delivered the build + (roots on /mnt/smithy 08:13, `recipe-erp-seat-sft-r5.json`, grant); infra-ops staged + launched + it. **524 steps, ~52 s/it → ETA ~16:35 PDT** (~7.5 h). All launch gates passed: 7/7 root shas + + shard hashes, survivor join 8,212 = recipe exactly, holdout disjoint 0 overlap, **window_count + belt-and-suspenders PASS** (govreport 496/496 + qmsum 97/97 all fit_whole), realized `[mix]` slot + loss 3.46% (pred 3.4%) at ctx 0.20 — single variable landed; INERT is a live verdict. Dependency- + forcing slot (GovReport+QMSum replace airoboros); kvasir byte-identical (survivors-r5 = survivors-r4 + − airoboros + govreport+qmsum, sha a25169a6). Canonical artifacts committed to eshpfi + `scripts/erp-tune-gx10/` + runbook `docs/runbooks/gx10-run-05.md`. **Completion poller `bhrkre8p4` + armed → on adapter-land, coordinate the base-serve→floor-lock→swap-cue→tuned-serve gate with brokkr + on the cc channel** (he's scaffolded run05-gate/ + committed the 4-cell verdict evaluator). - **run-4 gate = STILL-COUPLED** (RESULT-run04-gate.md, brokkr-smithy). Corpus dilution kept the diversity gain, did NOT remove the safety/coherence regression. **Base-abliteration LABEL unresolved — operator's ruling owed** (recipe says `-heretic`, provenance says stock; telemetry leans STOCK: base diff --git a/scripts/erp-tune-gx10/README.md b/scripts/erp-tune-gx10/README.md index 2b35223..ec02989 100644 --- a/scripts/erp-tune-gx10/README.md +++ b/scripts/erp-tune-gx10/README.md @@ -1,20 +1,35 @@ # erp-tune on pfi-gx10 -Canonical copies of what is staged on pfi-gx10 for ERP-seat SFT **run 3c** — -the LoRA that died on ana-ml2 at step 24 of 604 when an Anaheim breaker tripped. - -| file | deployed to | -|---|---| -| `run-03c-gx10.json` | `/home/infra-ops/erp-tune/run-03c-gx10.json` | -| `launch-run-03c.sh` | `/home/infra-ops/erp-tune/launch-run-03c.sh` | - -`run-03c-gx10.json` is `ana-ml2:/tank/erp-tune/run-03c.json` with its eight path -keys rehomed to local NVMe and two `substitute_controls` entries appended -recording the host move and the library delta. **No hyperparameter, corpus, -base, mask or recipe value differs** — verified key-by-key at generation. - -Full context, what was verified and how to launch: -[`docs/runbooks/gx10-run-03c.md`](../../docs/runbooks/gx10-run-03c.md). - +Canonical copies of what is staged on pfi-gx10 for the R47 ERP-seat SFT runs. The harness itself (`eitri-smithy`) is not vendored here; it lives on the box at `/home/infra-ops/erp-tune/eitri-smithy`, git `0a6bd2e`. + +| file | deployed to | run | +|---|---|---| +| `run-03c-gx10.json` | `/home/infra-ops/erp-tune/run-03c-gx10.json` | 3c | +| `launch-run-03c.sh` | `/home/infra-ops/erp-tune/launch-run-03c.sh` | 3c | +| `run-05-gx10.json` | `/home/infra-ops/erp-tune/run-05-gx10.json` | 5 | +| `launch-run-05.sh` | `/home/infra-ops/erp-tune/launch-run-05.sh` | 5 | +| `build_r5_survivors.py` | `/home/infra-ops/erp-tune/build_r5_survivors.py` | 5 | + +Runbooks: [`docs/runbooks/gx10-run-03c.md`](../../docs/runbooks/gx10-run-03c.md), +[`docs/runbooks/gx10-run-05.md`](../../docs/runbooks/gx10-run-05.md). + +**Run 3c** — the LoRA that died on ana-ml2 at step 24 when an Anaheim breaker +tripped, rehomed here unchanged (eight path keys rehomed to local NVMe, two host/ +library `substitute_controls` appended; no hyperparameter/corpus/base/mask/recipe +value differs, verified key-by-key). + +**Run 5** — third R47 corpus-gate run: the 20% slot's composition changes +(airoboros OUT, dependency-forcing govreport + qmsum IN) at run 4's lr 2e-04, +everything else held; kvasir byte-identical (survivors reused from run 4). +`run-05-gx10.json` is run 4's config with recipe/survivors/override swapped. + +> **Run 4 is not vendored here.** It ran on the box (config `run-04-gx10.json`, +> gated STILL-COUPLED) but its canonical copies were never committed; run 5's +> `build_r5_survivors.py` derives from `survivors-r4.jsonl` on the box, so run 4 +> is a live dependency of the run-5 build even though it is absent from this repo. + +Each `run-*-gx10.json` carries its run-scoped `training_eligibility_override`, +`overridden_blockers` and `substitute_controls` so the adapter's provenance +manifest records the grant, the two open blockers and their substitute controls. diff --git a/scripts/erp-tune-gx10/build_r5_survivors.py b/scripts/erp-tune-gx10/build_r5_survivors.py new file mode 100644 index 0000000..5eb5b20 --- /dev/null +++ b/scripts/erp-tune-gx10/build_r5_survivors.py @@ -0,0 +1,47 @@ +import json, hashlib, collections + +R4 = "/home/infra-ops/erp-tune/recipe-r4/survivors-r4.jsonl" +GOV = "/home/infra-ops/erp-tune/datasets/derived/govreport/clean-v1/govreport-clean-v1.jsonl" +QMS = "/home/infra-ops/erp-tune/datasets/derived/qmsum/clean-v1/qmsum-clean-v1.jsonl" +OUT = "/home/infra-ops/erp-tune/recipe-r5/survivors-r5.jsonl" + +# Start from run-4's survivors: they already carry the EXACT held dialogue +# selection (c2-logs 301 / cwm 183 / bluemoon 126), fireball whole (5,396) and +# kvasir's exact 1,613-sample prefix cut. Reusing them is what guarantees kvasir +# is byte-identical to run 4 -- NOT re-cut (run-5 recipe held clause). +r4 = [json.loads(l) for l in open(R4)] +rows = [r for r in r4 if r["dataset_id"] != "airoboros-3.2"] + +c = collections.Counter(r["dataset_id"] for r in rows) +held = {"c2-logs-32k-llama3": 301, "creative-writing-multiturn": 183, + "bluemoon": 126, "fireball": 5396, "kvasir": 1613} +assert set(c) == set(held), "unexpected dataset_id set after airoboros removal: %s" % dict(c) +for k, v in held.items(): + assert c[k] == v, "HELD ROOT CHANGED: %s %d != %d" % (k, c[k], v) +print(" held (dialogue+fireball+kvasir) unchanged:", dict(c)) + +def append_root(path, dsid, expect): + n = 0 + seen = set() + for l in open(path): + d = json.loads(l) + assert d["id"] not in seen, "dup id in %s: %s" % (dsid, d["id"]) + seen.add(d["id"]) + rows.append({"dataset_id": dsid, "id": d["id"]}) + n += 1 + assert n == expect, "%s: %d != %d" % (dsid, n, expect) + print(" %s appended: %d" % (dsid, n)) + +append_root(GOV, "govreport", 496) +append_root(QMS, "qmsum", 97) + +expected_total = 301 + 183 + 126 + 5396 + 1613 + 496 + 97 # 8212 +assert len(rows) == expected_total, "%d != %d" % (len(rows), expected_total) + +with open(OUT, "w") as f: + for r in rows: + f.write(json.dumps(r, separators=(",", ":"), sort_keys=True) + "\n") +sha = hashlib.sha256(open(OUT, "rb").read()).hexdigest() +print(" wrote %s survivors -> %s" % (format(len(rows), ","), OUT)) +print(" final composition:", dict(collections.Counter(r["dataset_id"] for r in rows))) +print(" sha256 %s" % sha) diff --git a/scripts/erp-tune-gx10/launch-run-05.sh b/scripts/erp-tune-gx10/launch-run-05.sh new file mode 100755 index 0000000..a8b6d3d --- /dev/null +++ b/scripts/erp-tune-gx10/launch-run-05.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# Launch ERP-seat SFT run 5 on pfi-gx10 (NVIDIA GB10, aarch64, sm_121). +# +# Run this ON pfi-gx10 as infra-ops. It detaches the job from the invoking +# shell and logs to the box, so a reaped SSH session cannot take the run with +# it -- the failure mode that lost the first probe launch on 2026-09-01. +# +# Run 5 = the dependency-forcing slot: airoboros OUT, govreport + qmsum IN, +# everything else held. ~8,212 survivors -> ~520-525 optimizer steps. +# Checkpoints every 50 steps. +set -euo pipefail + +ROOT=/home/infra-ops/erp-tune +HARNESS=$ROOT/eitri-smithy +VENV=/home/infra-ops/ml/.venv/bin/python +CONFIG=$ROOT/run-05-gx10.json +LOG=$ROOT/run-05.log + +# --- Preconditions, asserted rather than assumed ----------------------------- + +# A stuck orphan holding unified memory while PyTorch reports zero allocated +# already doomed three relaunches on this box and got blamed on the new run +# each time. Assert the GPU is clear. +apps=$(nvidia-smi --query-compute-apps=pid --format=csv,noheader | tr -d '[:space:]') +if [ -n "$apps" ]; then + echo "REFUSING: GPU is not clear -- compute apps still resident:" >&2 + nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv >&2 + exit 1 +fi + +# Deliberately NOT `pgrep -f erp_sft_harness`: run this over ssh and the +# pattern appears in the invoking shell's own argv, so the guard matches +# itself and refuses every launch. The pidfile is exact and cannot self-match; +# the GPU assertion above catches an orphan under any name. +if [ -f "$ROOT/run-05.pid" ] && kill -0 "$(cat "$ROOT/run-05.pid")" 2>/dev/null; then + echo "REFUSING: run-05.pid names a live process $(cat "$ROOT/run-05.pid"):" >&2 + ps -p "$(cat "$ROOT/run-05.pid")" -o pid,etime,cmd >&2 + exit 1 +fi + +if [ -e "$LOG" ]; then + echo "REFUSING: $LOG exists. Move it aside first so two runs cannot share a log." >&2 + exit 1 +fi + +for p in "$HARNESS/erp_sft_harness/__main__.py" "$VENV" "$CONFIG"; do + [ -e "$p" ] || { echo "REFUSING: missing $p" >&2; exit 1; } +done + +# Free space for checkpoints, with headroom. +avail=$(df --output=avail -BG "$ROOT" | tail -1 | tr -dc '0-9') +if [ "$avail" -lt 40 ]; then + echo "REFUSING: only ${avail}G free under $ROOT; want >=40G for checkpoints." >&2 + exit 1 +fi + +# --- Launch ------------------------------------------------------------------ + +cd "$HARNESS" +{ + echo "# launched $(date -Is) on $(hostname) by ${USER}" + echo "# harness $(git rev-parse --short HEAD) config $CONFIG" +} > "$LOG" + +setsid nohup "$VENV" -m erp_sft_harness --config "$CONFIG" >> "$LOG" 2>&1 < /dev/null & +pid=$! +echo "$pid" > "$ROOT/run-05.pid" + +echo "launched pid $pid -> $LOG" +echo +echo "watch: tail -f $LOG | tr '\\r' '\\n'" +echo "stop: kill \$(cat $ROOT/run-05.pid) # by PID -- never pkill -f over ssh" diff --git a/scripts/erp-tune-gx10/run-05-gx10.json b/scripts/erp-tune-gx10/run-05-gx10.json new file mode 100644 index 0000000..80b4df6 --- /dev/null +++ b/scripts/erp-tune-gx10/run-05-gx10.json @@ -0,0 +1,45 @@ +{ + "output_dir": "/home/infra-ops/erp-tune/run-05", + "roots_dir": "/home/infra-ops/erp-tune/datasets/derived", + "base_model_path": "/home/infra-ops/models/gemma4-26b-a4b-it-bf16", + "base_model_revision": "google/gemma-4-26B-A4B-it (stock instruct, upstream) -- UNCHANGED from runs 2, 3, 3c and 4. Run 5 varies the 20% slot's COMPOSITION only: airoboros-3.2 OUT, govreport + qmsum IN, at run 4's lr 2e-04. Base byte-identical to run 4 (index sha 907826a6...).", + "recipe": "/home/infra-ops/erp-tune/recipe-r5/recipe-erp-seat-sft-r5.json", + "survivors": "/home/infra-ops/erp-tune/recipe-r5/survivors-r5.jsonl", + "chat_template_path": "/home/infra-ops/models/gemma4-26b-a4b-it-bf16/chat_template.jinja", + "impersonation_mask_path": "/home/infra-ops/erp-tune/recipe-r3/lossmask-r3.jsonl", + "lora_rank": 64, + "lora_alpha": 128, + "lora_dropout": 0.0, + "max_seq_len": 16384, + "epochs": 1, + "seed": 20260824, + "per_device_batch_size": 2, + "gradient_accumulation_steps": 8, + "learning_rate": 0.0002, + "warmup_ratio": 0.1, + "lr_scheduler_type": "cosine", + "weight_decay": 0.01, + "load_in_4bit": false, + "gradient_checkpointing": true, + "loss_chunk_tokens": 1024, + "training_eligibility_override": "operator-2026-09-07-rnd-run5", + "overridden_blockers": [ + "contamination-scan-not-implemented", + "stage-2-csam-detector-inert" + ], + "substitute_controls": [ + "pre-training holdout, run-1 (8,404 samples, work/card/session split)", + "pre-training holdout, govreport/holdout-v1 (416 reports, sha256-ranked, never_trained_on)", + "pre-training holdout, qmsum/holdout-v1 (5 transcripts, sha256-ranked, never_trained_on)", + "stage-A lexical quarantine, RP (829 records held unread)", + "stage-A lexical quarantine, run-5 slot (133 records held unread, /mnt/smithy/datasets/quarantine/r47-run5-longdep-screen/)", + "SCROLLS-membership disclosure on both slot sources (avoidance, NOT a scan): govreport + qmsum are SCROLLS/ZeroSCROLLS members, in no hoard/default-benchmarks.yaml entry and used by no R47 instrument", + "SINGLE VARIABLE vs run 4: the 20% slot's COMPOSITION changes -- airoboros-3.2 is REMOVED and govreport (496 reports) + qmsum (97 transcripts) are ADDED at run 4's realized slot ctx (3,723,090 vs 3,720,956, +0.06%). Dialogue survivors (survivors-r3 selection) and the impersonation loss-mask (lossmask-r3) are held BYTE-IDENTICAL to runs 3/4; fireball whole; kvasir at run 4's EXACT 1,613-sample prefix cut (survivors reused from survivors-r4, NOT re-cut); base, lr 2e-04, max_seq_len 16384, rank 64, alpha 128, dropout 0.0, cosine, warmup 0.1, wd 0.01, batch 2 x accum 8, 1 epoch all UNCHANGED.", + "kvasir is HELD, not re-cut: the 1,613 kvasir survivors are reused verbatim from survivors-r4.jsonl (which cut run-3's seed-20260824 prefix at 3,347,622 ctx). survivors-r5.jsonl = survivors-r4 minus airoboros plus the govreport + qmsum roots whole; sha256 a25169a6258cd4abb0cb494a176a921c0e98eb73d65c53d033b6ee18293a43ae.", + "window_count belt-and-suspenders (SFT-RECIPE-run5-SCOPE.md 7.1): every govreport + qmsum row renders <= 14,000 tokens (max 9,385 / 13,700) and the harness never packs across samples, so window_count MUST be 1 on every slot row; a chunked_into_2 or single_window_truncated on either new root in truncation-report.json is a BUILD DEFECT and the run is killed before training.", + "HOST: pfi-gx10 (GB10, aarch64, sm_121, 121 GB unified). Base model sha256-verified identical to ana-ml2's copy; new-root shard sha256 verified against CLEANROOT after transfer (govreport d533a5cd, qmsum c0269d69); harness eitri-smithy 0a6bd2e; corpus COPIED, box mounts no NFS." + ], + "unfittable": "drop", + "holdout_dir": "/home/infra-ops/erp-tune/datasets/holdout", + "save_steps": 50 +}