The output-side test ran on the live seat. There is no shoulder at 123: the
120-139 bin holds three of ninety-six and is a TROUGH, and 17.7% of generations
cross a cap PIPPA can never cross. The clip-as-boundary reading is dead, killed
by the test that could have confirmed it.
Corrects this document's own earlier read, which compared the tuned MEAN (88.5)
to PIPPA's MEDIAN (67) and concluded 'comfortably inside the upper body'. Median
to median it is 62 against 67. Mixing statistics across a comparison produced a
more reassuring answer than the data supports.
What the data shows instead is bimodality -- a mode at 20-39, a trough, a second
mode astride PIPPA's centre, a tail to 505, against a base with no such shape.
The tune changed rp length's SHAPE rather than its centre: roots whose length
distributions do not overlap learned as distinct modes rather than blended into
an average. And the skew is rp-ONLY, which localises it to the family the
clipped root lives in and is the strongest support the turn-share mechanism gets
from the output side.
Consequence for pair generation: chosen/rejected sampled from a bimodal
generator inherit the mixture, not a mean, and naive sampling over-draws the
short mode.
Also records that the degeneracy rate is NOT yet a usable baseline -- same arm,
same seed, VOID flipped no->YES across a re-run because the 10% budget sits at
the noise boundary. A guard whose trip point is at the noise floor produces
disagreement between honest observers rather than silence. Replicates running.
The run-2 gate found tuned rp turns 36% shorter than base. brokkr hypothesised
the mix was teaching PIPPA's 2023 Character.AI product clip; the corpus side is
now measured and confirmed. PIPPA's max is 123 words EXACTLY, 100% at or under
it, and 0.00% in the 124-130 band -- a wall, not a preference. Every other root
crosses its own p99 smoothly.
The mechanism is sharper than 'PIPPA is in the mix'. PIPPA is 70.3% of bot TURNS
but only 37.5% of bot WORDS, precisely because its turns are clipped -- and
length is learned per turn, not per token. By loss tokens it looks like a third
of the dialogue signal; by end-of-turn demonstrations it is seven in ten from a
source that cannot exceed 123 words. Generalises: a length-clipped root is
over-represented in the length signal by exactly the ratio its clipping creates.
Counter-evidence recorded too: the tune landed near PIPPA's MEDIAN (67), not its
CAP, which is central tendency rather than learning the boundary. Weaker claim
than the hypothesis, and not demonstrated either way.
Filed here rather than only in the gate record because preference pairs
generated FROM this tune inherit its length distribution in both chosen and
rejected -- DPO would train an artifact in as an explicit objective. Settle the
length question before generating pairs.
Run 2 is an SFT on the official instruct base, so it will refuse at near-stock
rates by design; targeted DPO is where refusals get pruned on chosen axes. That
was the trade accepted when the stock base was picked over a third-party
abliteration.
Surveys what is on disk against what the stage needs. Ready: the merged tune,
the SFT adapter, GPU0 once the eval seat comes down, the whole non-loss half of
the SFT harness, two unvetted Gutenberg preference sets, and the LitBench-RM
judge.
Missing, in order of pain: preference data for the refusal axes (nothing on
disk targets it -- the Gutenberg sets are prose-quality), the axis list itself,
and a DPO trainer (trl is not installed).
The gating item is not technical: WHICH refusal axes are in scope and which are
explicitly kept. Data generation, pair counts, the held-out split and the
success probe are all functions of that list, so nobody should generate a pair
before it is written down. Flags that the domain-compliance probe should
measure run 2 BEFORE pruning, since the pre-number is the only baseline that
will ever exist.
Also records the operational trap: do the trl install AFTER a run finishes,
never during one -- a resolution that upgrades transformers under a live
process can break its save path.
vh