docs(erp-dpo): the clip hypothesis is falsified — the distribution is bimodal

The output-side test ran on the live seat. There is no shoulder at 123: the
120-139 bin holds three of ninety-six and is a TROUGH, and 17.7% of generations
cross a cap PIPPA can never cross. The clip-as-boundary reading is dead, killed
by the test that could have confirmed it.

Corrects this document's own earlier read, which compared the tuned MEAN (88.5)
to PIPPA's MEDIAN (67) and concluded 'comfortably inside the upper body'. Median
to median it is 62 against 67. Mixing statistics across a comparison produced a
more reassuring answer than the data supports.

What the data shows instead is bimodality -- a mode at 20-39, a trough, a second
mode astride PIPPA's centre, a tail to 505, against a base with no such shape.
The tune changed rp length's SHAPE rather than its centre: roots whose length
distributions do not overlap learned as distinct modes rather than blended into
an average. And the skew is rp-ONLY, which localises it to the family the
clipped root lives in and is the strongest support the turn-share mechanism gets
from the output side.

Consequence for pair generation: chosen/rejected sampled from a bimodal
generator inherit the mixture, not a mean, and naive sampling over-draws the
short mode.

Also records that the degeneracy rate is NOT yet a usable baseline -- same arm,
same seed, VOID flipped no->YES across a re-run because the 10% budget sits at
the noise boundary. A guard whose trip point is at the noise floor produces
disagreement between honest observers rather than silence. Replicates running.
This commit is contained in:
2026-08-26 06:22:21 -07:00
parent 1e4d827c5d
commit 37d3189622
+70 -6
View File
@@ -129,12 +129,76 @@ length signal by exactly the ratio its clipping creates.** The shorter the clip,
the more turns per token of budget, the harder it teaches the clip. Check the
turn-share/word-share split of every root before assuming a mix is balanced.
**The tune landed near PIPPA's MEDIAN (67), not its CAP (123).** If it were
learning the clip as a boundary you would expect mass piling toward 123. It
looks more like central tendency, which is a weaker and less alarming claim than
"it learned the limit". Not demonstrated either way — the closing test is the
tuned arm's own rp length distribution against PIPPA's and bluemoon's, looking
for a shoulder near 123.
### The output-side test ran. The clip hypothesis is FALSIFIED — and the truth is more interesting
Measured on the live tuned seat, n=96 rp generations (brokkr-smithy-dev,
2026-08-26). **There is no shoulder at 123:**
0-19 18 ██████████████████
20-39 22 ██████████████████████ <- MODE
40-59 5 █████ <- trough
60-79 15 ███████████████ <- PIPPA median 67
80-99 10 ██████████
100-119 9 █████████
120-139 3 ███ <- PIPPA CAP 123. Three. A trough.
140-159 5 █████
160-179 4 ████
300+ 3 ███ max 505
<=123 79/96 = 82.3% (PIPPA itself: 100.0%)
median 62 · mean 77.9 · p90 152 · p99 367
**The clip-as-boundary reading is dead.** Mass would pile *under* a learned wall;
the 120-139 bin holds three of ninety-six and is a trough, and **17.7% of
generations cross a cap PIPPA can never cross.**
**A correction to this document's earlier read.** It compared the tuned *mean*
(88.5) to PIPPA's *median* (67) and p90 (95) and concluded "central tendency,
comfortably inside the upper body." **Median to median it is 62 against 67** — at
or slightly below PIPPA's centre. Mixing statistics across a comparison produced
a more reassuring answer than the data supports; the 88.5 was a mean dragged up
by the tail.
**What the data shows instead: the distribution is BIMODAL.** A mode at 20-39, a
trough at 40-59, a second mode at 60-79 astride PIPPA's centre, then a long tail
to 505. The base arm has no such shape (tuned mean/median ratio 1.45 and 1.23
against the base's 1.09 and 1.14).
So the tune did not change rp length so much as change its **shape** — replacing
one symmetric distribution with a mixture. That is what you would expect from
roots whose length distributions do not overlap (PIPPA 67, c2-logs 256, bluemoon
601) being learned as **distinct modes rather than blended into an average**.
**And the skew is rp-ONLY.** Story sits at mean/median 0.98 / 0.95 tuned
against 0.98 / 1.00 base. The shape change appears exactly in the family the
clipped root lives in — the strongest support the turn-share mechanism gets from
the output side.
**What this means for pair generation:** `chosen`/`rejected` sampled from a
bimodal generator inherit the mixture, not a mean. Sampling naively will
over-draw from the dominant short mode. If length is not the axis being tuned,
control for it explicitly rather than assuming a representative sample.
### ⚠ The degeneracy rate is NOT yet a usable baseline
Same arm, same seed block, re-run:
scored 162/192 -> 163/192
degenerate 7 -> 11
truncated 23 -> 18
rp VOID no -> YES
Distance moved 0.003, well inside the 0.0125 floor, so the gate result is not
threatened. But **degeneracy straddles the 10% budget**, so the same arm on the
same seed VOIDs or does not depending on the run. **A guard whose trip point sits
at the noise boundary flips**, and it produces disagreement between honest
observers rather than silence — a distinct defect from a gate that cannot fail.
The honest statement is that **the tuned arm's rp degeneracy is AT the budget**,
not clearly over or under it. Replicates are being run to pin it. **Do not build
DPO pairs on the single-sample figure** — the degeneracy rate decides how much of
the tune's own output is usable as `chosen`, so it is load-bearing for pair
generation and much cheaper to pin now than to discover after pairs exist.
**Why this belongs in the DPO prep and not just the gate record:** preference
pairs generated *from this tune* inherit its length distribution in both