diff --git a/docs/pfi/erp-dpo-stage-prep.md b/docs/pfi/erp-dpo-stage-prep.md index 56a0d5f..6f6b854 100644 --- a/docs/pfi/erp-dpo-stage-prep.md +++ b/docs/pfi/erp-dpo-stage-prep.md @@ -92,6 +92,58 @@ as everything else — see playbook 4.3's audit table. **A DPO run that does not record which axes it pruned is unauditable**, and this stage is the one where that matters most. +## ⚠ Measured 2026-08-26 — a length artifact the DPO pairs would inherit + +The run-2 gate found tuned rp turns at **88.5 / 90.9 words against the base's +137.1 / 145.4** — a 36% reduction, consistent across blocks. brokkr-smithy-dev +hypothesised the mix was teaching PIPPA's product clip. The corpus side is now +measured and it is confirmed, with a mechanism sharper than "PIPPA is in the +mix": + + root bot turns p50 p90 p99 max <=123 124-130 + pippa 79,414 67 95 108 123 100.0% 0.00% + bluemoon 1,170 563 908 1432 1924 0.0% 0.00% + c2-logs-32k-llama3 23,704 234 394 686 2000 6.6% 1.44% + creative-writing-multiturn 8,621 204 485 858 1837 24.0% 2.16% + +**PIPPA's max is 123 exactly, 100% at or under it, 0.00% in the 124-130 band.** +That is a product clip (2023 Character.AI), not a length preference — every +other root crosses its own p99 smoothly. + +The exposure asymmetry is the finding: + + root turn share word share mean len + pippa 70.3% 37.5% 67 + c2-logs-32k-llama3 21.0% 42.5% 256 + creative-writing-multiturn 7.6% 15.1% 251 + bluemoon 1.0% 4.9% 601 + +**PIPPA is 70.3% of bot TURNS but 37.5% of bot WORDS — and length is learned per +turn, not per token.** Each of those 79,414 turns is a separate demonstration of +where a bot reply ends. By loss tokens PIPPA looks like a third of the dialogue +signal; by end-of-turn demonstrations it is seven in ten, from a source that +cannot exceed 123 words. + +⚠ **Generalises past PIPPA: a length-clipped root is over-represented in the +length signal by exactly the ratio its clipping creates.** The shorter the clip, +the more turns per token of budget, the harder it teaches the clip. Check the +turn-share/word-share split of every root before assuming a mix is balanced. + +⚠ **The tune landed near PIPPA's MEDIAN (67), not its CAP (123).** If it were +learning the clip as a boundary you would expect mass piling toward 123. It +looks more like central tendency, which is a weaker and less alarming claim than +"it learned the limit". Not demonstrated either way — the closing test is the +tuned arm's own rp length distribution against PIPPA's and bluemoon's, looking +for a shoulder near 123. + +**Why this belongs in the DPO prep and not just the gate record:** preference +pairs generated *from this tune* inherit its length distribution in both +`chosen` and `rejected`. If short-rp is an artifact rather than a preference, +DPO on those pairs trains it in as an explicit objective rather than leaving it +as an incidental SFT bias. Settle the length question before generating pairs. + +Scripts: `/tank/erp-tune/pippa_clip.py`, `/tank/erp-tune/clip_share.py`. + ## The decision that gates the rest **Which refusal axes are in scope, and which are explicitly kept?**