docs(erp-dpo): the clip hypothesis is falsified — the distribution is bimodal
The output-side test ran on the live seat. There is no shoulder at 123: the 120-139 bin holds three of ninety-six and is a TROUGH, and 17.7% of generations cross a cap PIPPA can never cross. The clip-as-boundary reading is dead, killed by the test that could have confirmed it. Corrects this document's own earlier read, which compared the tuned MEAN (88.5) to PIPPA's MEDIAN (67) and concluded 'comfortably inside the upper body'. Median to median it is 62 against 67. Mixing statistics across a comparison produced a more reassuring answer than the data supports. What the data shows instead is bimodality -- a mode at 20-39, a trough, a second mode astride PIPPA's centre, a tail to 505, against a base with no such shape. The tune changed rp length's SHAPE rather than its centre: roots whose length distributions do not overlap learned as distinct modes rather than blended into an average. And the skew is rp-ONLY, which localises it to the family the clipped root lives in and is the strongest support the turn-share mechanism gets from the output side. Consequence for pair generation: chosen/rejected sampled from a bimodal generator inherit the mixture, not a mean, and naive sampling over-draws the short mode. Also records that the degeneracy rate is NOT yet a usable baseline -- same arm, same seed, VOID flipped no->YES across a re-run because the 10% budget sits at the noise boundary. A guard whose trip point is at the noise floor produces disagreement between honest observers rather than silence. Replicates running.
This commit is contained in:
@@ -129,12 +129,76 @@ length signal by exactly the ratio its clipping creates.** The shorter the clip,
|
||||
the more turns per token of budget, the harder it teaches the clip. Check the
|
||||
turn-share/word-share split of every root before assuming a mix is balanced.
|
||||
|
||||
⚠ **The tune landed near PIPPA's MEDIAN (67), not its CAP (123).** If it were
|
||||
learning the clip as a boundary you would expect mass piling toward 123. It
|
||||
looks more like central tendency, which is a weaker and less alarming claim than
|
||||
"it learned the limit". Not demonstrated either way — the closing test is the
|
||||
tuned arm's own rp length distribution against PIPPA's and bluemoon's, looking
|
||||
for a shoulder near 123.
|
||||
### The output-side test ran. The clip hypothesis is FALSIFIED — and the truth is more interesting
|
||||
|
||||
Measured on the live tuned seat, n=96 rp generations (brokkr-smithy-dev,
|
||||
2026-08-26). **There is no shoulder at 123:**
|
||||
|
||||
0-19 18 ██████████████████
|
||||
20-39 22 ██████████████████████ <- MODE
|
||||
40-59 5 █████ <- trough
|
||||
60-79 15 ███████████████ <- PIPPA median 67
|
||||
80-99 10 ██████████
|
||||
100-119 9 █████████
|
||||
120-139 3 ███ <- PIPPA CAP 123. Three. A trough.
|
||||
140-159 5 █████
|
||||
160-179 4 ████
|
||||
300+ 3 ███ max 505
|
||||
|
||||
<=123 79/96 = 82.3% (PIPPA itself: 100.0%)
|
||||
median 62 · mean 77.9 · p90 152 · p99 367
|
||||
|
||||
**The clip-as-boundary reading is dead.** Mass would pile *under* a learned wall;
|
||||
the 120-139 bin holds three of ninety-six and is a trough, and **17.7% of
|
||||
generations cross a cap PIPPA can never cross.**
|
||||
|
||||
⚠ **A correction to this document's earlier read.** It compared the tuned *mean*
|
||||
(88.5) to PIPPA's *median* (67) and p90 (95) and concluded "central tendency,
|
||||
comfortably inside the upper body." **Median to median it is 62 against 67** — at
|
||||
or slightly below PIPPA's centre. Mixing statistics across a comparison produced
|
||||
a more reassuring answer than the data supports; the 88.5 was a mean dragged up
|
||||
by the tail.
|
||||
|
||||
**What the data shows instead: the distribution is BIMODAL.** A mode at 20-39, a
|
||||
trough at 40-59, a second mode at 60-79 astride PIPPA's centre, then a long tail
|
||||
to 505. The base arm has no such shape (tuned mean/median ratio 1.45 and 1.23
|
||||
against the base's 1.09 and 1.14).
|
||||
|
||||
So the tune did not change rp length so much as change its **shape** — replacing
|
||||
one symmetric distribution with a mixture. That is what you would expect from
|
||||
roots whose length distributions do not overlap (PIPPA 67, c2-logs 256, bluemoon
|
||||
601) being learned as **distinct modes rather than blended into an average**.
|
||||
|
||||
⚠ **And the skew is rp-ONLY.** Story sits at mean/median 0.98 / 0.95 tuned
|
||||
against 0.98 / 1.00 base. The shape change appears exactly in the family the
|
||||
clipped root lives in — the strongest support the turn-share mechanism gets from
|
||||
the output side.
|
||||
|
||||
**What this means for pair generation:** `chosen`/`rejected` sampled from a
|
||||
bimodal generator inherit the mixture, not a mean. Sampling naively will
|
||||
over-draw from the dominant short mode. If length is not the axis being tuned,
|
||||
control for it explicitly rather than assuming a representative sample.
|
||||
|
||||
### ⚠ The degeneracy rate is NOT yet a usable baseline
|
||||
|
||||
Same arm, same seed block, re-run:
|
||||
|
||||
scored 162/192 -> 163/192
|
||||
degenerate 7 -> 11
|
||||
truncated 23 -> 18
|
||||
rp VOID no -> YES
|
||||
|
||||
Distance moved 0.003, well inside the 0.0125 floor, so the gate result is not
|
||||
threatened. But **degeneracy straddles the 10% budget**, so the same arm on the
|
||||
same seed VOIDs or does not depending on the run. **A guard whose trip point sits
|
||||
at the noise boundary flips**, and it produces disagreement between honest
|
||||
observers rather than silence — a distinct defect from a gate that cannot fail.
|
||||
|
||||
The honest statement is that **the tuned arm's rp degeneracy is AT the budget**,
|
||||
not clearly over or under it. Replicates are being run to pin it. **Do not build
|
||||
DPO pairs on the single-sample figure** — the degeneracy rate decides how much of
|
||||
the tune's own output is usable as `chosen`, so it is load-bearing for pair
|
||||
generation and much cheaper to pin now than to discover after pairs exist.
|
||||
|
||||
**Why this belongs in the DPO prep and not just the gate record:** preference
|
||||
pairs generated *from this tune* inherit its length distribution in both
|
||||
|
||||
Reference in New Issue
Block a user