# ERP/RP tune run-01 COMPLETE — 7.36h, gate passed on the axis it was built for

`[2026-08-25]`

## The run

    1312/1312 in 7:21:52   train_loss 2.793   epoch 1.0
    20.1 s/it FLAT across every 100-step window   (round 1: 35-46.5 s/it)
    adapter: /tank/erp-tune/run-01/adapter/  410 tensors, provenance.json

**47% faster than the round-1 projection of 13.85h**, from two changes: the
bucketed sampler and flex attention. Rate was flat — 19.7 / 19.8 / 20.4 / 20.3
across the four 100-step windows — which means the 35-46.5 spread in round 1 was
*entirely padding*, and removing padding removed the variance rather than just
the mean.

⚠ **I quoted three different ETAs (6.9h, 8h, 7.3h) before I started using a
rolling average.** The first two were instantaneous tqdm readings off a number
that swings 17-25 s/it with batch width. Only the rolling rate was honest. Same
measure-don't-sample discipline I wrote into the throughput playbook, violated on
the one metric I kept reporting.

## lora_B gate — PASSED, twice

    checkpoint-100   205/205 non-zero, median norm 0.829
    final adapter    205/205 non-zero, median norm 1.708
    vision_tower tensors: 0 on both

Median norm rising 0.829 -> 1.708 means it kept learning through the whole run
rather than saturating early. This check **never ran in round 1** (died at step
19, first checkpoint was 100) and it is the only failure mode that stays
invisible until the acceptance gate reports base-identical numbers.

## The gate — brokkr-smithy-dev

**It did the thing it was built to do:**

    metric                    base A/B        tuned      delta      floor
    attractor hit rate    94.8% / 96.8%       84.5%     -11.3pt     2.0pt
    diversity (pairwise)  0.213 / 0.221      0.3948     +0.178     0.008

Diversity moved **22x its own noise floor**. Attractor rate (how often the model
reaches for the same names and phrasings) fell 11 points against a 2-point floor.

    T1 100 · T2 95 · T3 96-97 · T4 98 · T5 100 · T6 81-82 · core ~94.2
    memorisation: 0.0000 on BOTH arms, all three corpora

**Zero memorisation closes the R20 licensed-prose exposure on measurement rather
than argument.**

⚠ **Caveat brokkr volunteered rather than buried:** the tuned arm lost 18 of 192
generations to truncation/degeneracy against base's 1-2. Lopsided exclusions
plausibly flatter the diversity magnitude. Direction is unambiguous at 22x floor;
the number carries an asterisk.

## The noise-floor near-miss — the methodology lesson

brokkr was one step from reporting a 13-point T6 regression **that sat inside
twice his instrument's own variance.**

    --per-type 32     max swing across tasks:  9 points
    --per-type 128    max swing across tasks:  1 point

His gate criterion is "no task regresses by more than one item" = 3.1 points at
n=32. **The instrument's own run-to-run noise was 3 items.** He was scoring a
preregistered gate at 4x finer resolution than it could resolve, and caught it by
running a control he did not strictly need. Quadrupling n collapsed the noise
exactly as binomial statistics predicts.

⚠ **Root cause of the noise is a property of the SEAT:** `max-num-seqs` is unset,
so with a 218,625-token KV cache the scheduler batches freely up to vLLM's
default of 256. Continuous batching changes reduction order and borderline items
flip. Temperature 0 buys deterministic *sampling*, not deterministic
*arithmetic*. He declined a `--max-num-seqs 1` determinism control for the right
reason: a floor measured on a seat serving one request at a time is not the floor
that applies to the seat we ship.

## The confound I built and he caught

I optimised a pipeline for production and then handed him its output as an eval
instrument **without asking whether those were the same job.** The tuned arm
would have reached the seat as NVFP4A16 while his base arm was bf16 — any
regression would have been tuning-damage OR quantization-damage with no way to
separate them, and the gate's whole question is "did the tune cost us
capability."

**Both arms now bf16, same seat, same port, argv differing in exactly two
lines** (weights path, served name), template sha256 identical
(`ae53464bf3be2580`), KV cache identical to the digit (218,625 tokens across all
three launches). Quantization moved *downstream* of the gate.

See [[2026-08-25-refusal-retention-probe]] for the axis his gate did not have.