docs(erp-dpo): the PIPPA length clip, measured — DPO pairs would inherit it

The run-2 gate found tuned rp turns 36% shorter than base. brokkr hypothesised
the mix was teaching PIPPA's 2023 Character.AI product clip; the corpus side is
now measured and confirmed. PIPPA's max is 123 words EXACTLY, 100% at or under
it, and 0.00% in the 124-130 band -- a wall, not a preference. Every other root
crosses its own p99 smoothly.

The mechanism is sharper than 'PIPPA is in the mix'. PIPPA is 70.3% of bot TURNS
but only 37.5% of bot WORDS, precisely because its turns are clipped -- and
length is learned per turn, not per token. By loss tokens it looks like a third
of the dialogue signal; by end-of-turn demonstrations it is seven in ten from a
source that cannot exceed 123 words. Generalises: a length-clipped root is
over-represented in the length signal by exactly the ratio its clipping creates.

Counter-evidence recorded too: the tune landed near PIPPA's MEDIAN (67), not its
CAP, which is central tendency rather than learning the boundary. Weaker claim
than the hypothesis, and not demonstrated either way.

Filed here rather than only in the gate record because preference pairs
generated FROM this tune inherit its length distribution in both chosen and
rejected -- DPO would train an artifact in as an explicit objective. Settle the
length question before generating pairs.
This commit is contained in:
vh
2026-08-26 02:25:08 -07:00
parent 0bb9ee7777
commit 5171f19e16
+52
View File
@@ -92,6 +92,58 @@ as everything else — see playbook 4.3's audit table. **A DPO run that does not
record which axes it pruned is unauditable**, and this stage is the one where
that matters most.
## ⚠ Measured 2026-08-26 — a length artifact the DPO pairs would inherit
The run-2 gate found tuned rp turns at **88.5 / 90.9 words against the base's
137.1 / 145.4** — a 36% reduction, consistent across blocks. brokkr-smithy-dev
hypothesised the mix was teaching PIPPA's product clip. The corpus side is now
measured and it is confirmed, with a mechanism sharper than "PIPPA is in the
mix":
root bot turns p50 p90 p99 max <=123 124-130
pippa 79,414 67 95 108 123 100.0% 0.00%
bluemoon 1,170 563 908 1432 1924 0.0% 0.00%
c2-logs-32k-llama3 23,704 234 394 686 2000 6.6% 1.44%
creative-writing-multiturn 8,621 204 485 858 1837 24.0% 2.16%
**PIPPA's max is 123 exactly, 100% at or under it, 0.00% in the 124-130 band.**
That is a product clip (2023 Character.AI), not a length preference — every
other root crosses its own p99 smoothly.
The exposure asymmetry is the finding:
root turn share word share mean len
pippa 70.3% 37.5% 67
c2-logs-32k-llama3 21.0% 42.5% 256
creative-writing-multiturn 7.6% 15.1% 251
bluemoon 1.0% 4.9% 601
**PIPPA is 70.3% of bot TURNS but 37.5% of bot WORDS — and length is learned per
turn, not per token.** Each of those 79,414 turns is a separate demonstration of
where a bot reply ends. By loss tokens PIPPA looks like a third of the dialogue
signal; by end-of-turn demonstrations it is seven in ten, from a source that
cannot exceed 123 words.
⚠ **Generalises past PIPPA: a length-clipped root is over-represented in the
length signal by exactly the ratio its clipping creates.** The shorter the clip,
the more turns per token of budget, the harder it teaches the clip. Check the
turn-share/word-share split of every root before assuming a mix is balanced.
⚠ **The tune landed near PIPPA's MEDIAN (67), not its CAP (123).** If it were
learning the clip as a boundary you would expect mass piling toward 123. It
looks more like central tendency, which is a weaker and less alarming claim than
"it learned the limit". Not demonstrated either way — the closing test is the
tuned arm's own rp length distribution against PIPPA's and bluemoon's, looking
for a shoulder near 123.
**Why this belongs in the DPO prep and not just the gate record:** preference
pairs generated *from this tune* inherit its length distribution in both
`chosen` and `rejected`. If short-rp is an artifact rather than a preference,
DPO on those pairs trains it in as an explicit objective rather than leaving it
as an incidental SFT bias. Settle the length question before generating pairs.
Scripts: `/tank/erp-tune/pippa_clip.py`, `/tank/erp-tune/clip_share.py`.
## The decision that gates the rest
**Which refusal axes are in scope, and which are explicitly kept?**