docs(erp-dpo): the PIPPA length clip, measured — DPO pairs would inherit it
The run-2 gate found tuned rp turns 36% shorter than base. brokkr hypothesised the mix was teaching PIPPA's 2023 Character.AI product clip; the corpus side is now measured and confirmed. PIPPA's max is 123 words EXACTLY, 100% at or under it, and 0.00% in the 124-130 band -- a wall, not a preference. Every other root crosses its own p99 smoothly. The mechanism is sharper than 'PIPPA is in the mix'. PIPPA is 70.3% of bot TURNS but only 37.5% of bot WORDS, precisely because its turns are clipped -- and length is learned per turn, not per token. By loss tokens it looks like a third of the dialogue signal; by end-of-turn demonstrations it is seven in ten from a source that cannot exceed 123 words. Generalises: a length-clipped root is over-represented in the length signal by exactly the ratio its clipping creates. Counter-evidence recorded too: the tune landed near PIPPA's MEDIAN (67), not its CAP, which is central tendency rather than learning the boundary. Weaker claim than the hypothesis, and not demonstrated either way. Filed here rather than only in the gate record because preference pairs generated FROM this tune inherit its length distribution in both chosen and rejected -- DPO would train an artifact in as an explicit objective. Settle the length question before generating pairs.
This commit is contained in:
@@ -92,6 +92,58 @@ as everything else — see playbook 4.3's audit table. **A DPO run that does not
|
||||
record which axes it pruned is unauditable**, and this stage is the one where
|
||||
that matters most.
|
||||
|
||||
## ⚠ Measured 2026-08-26 — a length artifact the DPO pairs would inherit
|
||||
|
||||
The run-2 gate found tuned rp turns at **88.5 / 90.9 words against the base's
|
||||
137.1 / 145.4** — a 36% reduction, consistent across blocks. brokkr-smithy-dev
|
||||
hypothesised the mix was teaching PIPPA's product clip. The corpus side is now
|
||||
measured and it is confirmed, with a mechanism sharper than "PIPPA is in the
|
||||
mix":
|
||||
|
||||
root bot turns p50 p90 p99 max <=123 124-130
|
||||
pippa 79,414 67 95 108 123 100.0% 0.00%
|
||||
bluemoon 1,170 563 908 1432 1924 0.0% 0.00%
|
||||
c2-logs-32k-llama3 23,704 234 394 686 2000 6.6% 1.44%
|
||||
creative-writing-multiturn 8,621 204 485 858 1837 24.0% 2.16%
|
||||
|
||||
**PIPPA's max is 123 exactly, 100% at or under it, 0.00% in the 124-130 band.**
|
||||
That is a product clip (2023 Character.AI), not a length preference — every
|
||||
other root crosses its own p99 smoothly.
|
||||
|
||||
The exposure asymmetry is the finding:
|
||||
|
||||
root turn share word share mean len
|
||||
pippa 70.3% 37.5% 67
|
||||
c2-logs-32k-llama3 21.0% 42.5% 256
|
||||
creative-writing-multiturn 7.6% 15.1% 251
|
||||
bluemoon 1.0% 4.9% 601
|
||||
|
||||
**PIPPA is 70.3% of bot TURNS but 37.5% of bot WORDS — and length is learned per
|
||||
turn, not per token.** Each of those 79,414 turns is a separate demonstration of
|
||||
where a bot reply ends. By loss tokens PIPPA looks like a third of the dialogue
|
||||
signal; by end-of-turn demonstrations it is seven in ten, from a source that
|
||||
cannot exceed 123 words.
|
||||
|
||||
⚠ **Generalises past PIPPA: a length-clipped root is over-represented in the
|
||||
length signal by exactly the ratio its clipping creates.** The shorter the clip,
|
||||
the more turns per token of budget, the harder it teaches the clip. Check the
|
||||
turn-share/word-share split of every root before assuming a mix is balanced.
|
||||
|
||||
⚠ **The tune landed near PIPPA's MEDIAN (67), not its CAP (123).** If it were
|
||||
learning the clip as a boundary you would expect mass piling toward 123. It
|
||||
looks more like central tendency, which is a weaker and less alarming claim than
|
||||
"it learned the limit". Not demonstrated either way — the closing test is the
|
||||
tuned arm's own rp length distribution against PIPPA's and bluemoon's, looking
|
||||
for a shoulder near 123.
|
||||
|
||||
**Why this belongs in the DPO prep and not just the gate record:** preference
|
||||
pairs generated *from this tune* inherit its length distribution in both
|
||||
`chosen` and `rejected`. If short-rp is an artifact rather than a preference,
|
||||
DPO on those pairs trains it in as an explicit objective rather than leaving it
|
||||
as an incidental SFT bias. Settle the length question before generating pairs.
|
||||
|
||||
Scripts: `/tank/erp-tune/pippa_clip.py`, `/tank/erp-tune/clip_share.py`.
|
||||
|
||||
## The decision that gates the rest
|
||||
|
||||
**Which refusal axes are in scope, and which are explicitly kept?**
|
||||
|
||||
Reference in New Issue
Block a user