lv-bronte: the voice axis passes under the corrected floor rule — amended, not rewritten
lv-bronte shipped 2026-09-17 with a FAILED voice axis written into its compose comment,
its NFS README and its gate record. That verdict no longer stands, and this records the
correction in all three places without deleting what they said.
The floor rule is now pairwise (commit 0bb4938, pre-registered for lv-hemingway before
any Hemingway number existed). Re-scoring the SAME 360 generations, no re-run, no changed
delta_cb:
ckpt475 (shipped) +0.193 vs pairwise floor 0.091 -> PASS, 2.1x
ckpt925 (not shipped) +0.210 vs its own spread 0.251 -> still fails
As run, the floor was 0.251 for every candidate, contributed entirely by ckpt925's single
outlier seed — a candidate nobody was shipping failed the one that was.
Why this is not a threshold chosen to produce a verdict: the previous session found the
defect, wrote it into this very file, and deliberately declined to act on it. The rule was
changed prospectively on an argument independent of the answer — the sampling variability
of a difference A-B depends on A and B, not on a third arm C. voice_distance.py now prints
both floors and flags disagreement so neither can be quoted without the other.
What changes for a reader: the sensitivity floor is 0.091 rather than 0.251, and "do not
cite lv-bronte as evidence pair-SFT works for this author" is withdrawn. What does not
change: NOT-COPIED and NO-DAMAGE as recorded, ckpt475 over ckpt925 for the same reasons,
and the two-epoch recipe still not transferring to Brontë.
Amendments are append-only in all three artifacts. The on-host compose is unchanged so far
— this edit is comment-only and will ride with the next real deploy rather than triggering
a model reload for a comment.
This commit is contained in:
@@ -85,3 +85,51 @@ Commits `fc834a8` `533cc0c` `7964d07` `e9e8c40` `8bb7686`.
|
||||
NUMBERS are Brontë's; those two labels are not. Not yet fixed.
|
||||
|
||||
Related: [[2026-09-16-lv-voices-line]], [[2026-09-16-lv-hemingway-corpus]], [[2026-09-16-voices-seat-lora]].
|
||||
|
||||
---
|
||||
|
||||
## ⚠⚠ AMENDED 2026-09-17 — THE VOICE AXIS PASSES UNDER THE CORRECTED FLOOR RULE
|
||||
|
||||
Everything above is left verbatim; it is what was believed at ship time. This section is
|
||||
the correction, not a rewrite.
|
||||
|
||||
**The defect this file itself named was fixed, and fixing it flips ckpt475's verdict.**
|
||||
The section "⚠ A DEFECT IN THE v2 RULE ITSELF" above says the rule should state whether
|
||||
the floor is computed over the compared pair or over every arm present. It is now
|
||||
**pairwise**, pre-registered in `scripts/hemingway-corpus/GATE-PREREG.md` before a single
|
||||
lv-hemingway number existed. Re-scoring the SAME 360 generations — no re-run, no changed
|
||||
delta_cb:
|
||||
|
||||
```
|
||||
arm delta_cb per-seed spread
|
||||
ckpt925 0.531 (0.776 0.584 0.525 0.620) 0.251
|
||||
ckpt475 0.548 (0.670 0.631 0.604 0.578) 0.091
|
||||
base-unadapted 0.741 (0.772 0.813 0.751 0.772) 0.062
|
||||
|
||||
all-arms floor (as run) 0.251
|
||||
ckpt475 +0.193 vs pairwise floor 0.091 -> MOVED toward Brontë, 2.1x <- the two rules DISAGREE
|
||||
ckpt925 +0.210 vs pairwise floor 0.251 -> within the floor, NOT a finding
|
||||
```
|
||||
|
||||
⭐ **The sequence matters and is the reason this is not threshold-shopping.** The previous
|
||||
session found the defect, recorded it, and explicitly declined to exploit it. The rule was
|
||||
then changed prospectively on a structural argument independent of the answer it produces —
|
||||
the sampling variability of a difference A−B depends on A and B, never on a third arm C, so
|
||||
a candidate's verdict must not depend on which other arms were generated. `voice_distance.py`
|
||||
prints both floors and flags disagreement, so neither number can be quoted alone.
|
||||
|
||||
**Consequences:**
|
||||
- lv-bronte's voice axis is a **PASS at 2.1x**, not a fail. The caveat is amended in place
|
||||
(append-only) in `stacks/voices-seat/compose.yaml` and
|
||||
`/tank/aimodels/voice-adapters/lv-bronte-4b-v1/README.md`.
|
||||
- The sensitivity floor for that measurement is **0.091**, not 0.251.
|
||||
- "Do not cite lv-bronte as evidence pair-SFT works for this author" is **WITHDRAWN**.
|
||||
- ckpt475 over ckpt925 is unchanged and for unchanged reasons (no verbatim 8-gram hit,
|
||||
2.7x tighter seed variance).
|
||||
- The "no cheap fix for the underpowered result" analysis above is superseded for Brontë:
|
||||
it was underpowered against an inflated floor, not against its own.
|
||||
|
||||
**Also amended:** the two hardcoded Yarros labels flagged at the end of this file are fixed.
|
||||
`voice_distance.py --author` is now REQUIRED — the committed Brontë output literally reads
|
||||
"reference: held-out Yarros" over Brontë's numbers — and the stale "one seed-pair per arm /
|
||||
corroborates Base < Instruct" footer now reports what the run actually carries.
|
||||
|
||||
Reference in New Issue
Block a user