diff --git a/persistent-memory.d/2026-09-17-lv-bronte-gate.md b/persistent-memory.d/2026-09-17-lv-bronte-gate.md index 96388af..491a316 100644 --- a/persistent-memory.d/2026-09-17-lv-bronte-gate.md +++ b/persistent-memory.d/2026-09-17-lv-bronte-gate.md @@ -85,3 +85,51 @@ Commits `fc834a8` `533cc0c` `7964d07` `e9e8c40` `8bb7686`. NUMBERS are Brontë's; those two labels are not. Not yet fixed. Related: [[2026-09-16-lv-voices-line]], [[2026-09-16-lv-hemingway-corpus]], [[2026-09-16-voices-seat-lora]]. + +--- + +## ⚠⚠ AMENDED 2026-09-17 — THE VOICE AXIS PASSES UNDER THE CORRECTED FLOOR RULE + +Everything above is left verbatim; it is what was believed at ship time. This section is +the correction, not a rewrite. + +**The defect this file itself named was fixed, and fixing it flips ckpt475's verdict.** +The section "⚠ A DEFECT IN THE v2 RULE ITSELF" above says the rule should state whether +the floor is computed over the compared pair or over every arm present. It is now +**pairwise**, pre-registered in `scripts/hemingway-corpus/GATE-PREREG.md` before a single +lv-hemingway number existed. Re-scoring the SAME 360 generations — no re-run, no changed +delta_cb: + +``` + arm delta_cb per-seed spread + ckpt925 0.531 (0.776 0.584 0.525 0.620) 0.251 + ckpt475 0.548 (0.670 0.631 0.604 0.578) 0.091 + base-unadapted 0.741 (0.772 0.813 0.751 0.772) 0.062 + + all-arms floor (as run) 0.251 + ckpt475 +0.193 vs pairwise floor 0.091 -> MOVED toward Brontë, 2.1x <- the two rules DISAGREE + ckpt925 +0.210 vs pairwise floor 0.251 -> within the floor, NOT a finding +``` + +⭐ **The sequence matters and is the reason this is not threshold-shopping.** The previous +session found the defect, recorded it, and explicitly declined to exploit it. The rule was +then changed prospectively on a structural argument independent of the answer it produces — +the sampling variability of a difference A−B depends on A and B, never on a third arm C, so +a candidate's verdict must not depend on which other arms were generated. `voice_distance.py` +prints both floors and flags disagreement, so neither number can be quoted alone. + +**Consequences:** +- lv-bronte's voice axis is a **PASS at 2.1x**, not a fail. The caveat is amended in place + (append-only) in `stacks/voices-seat/compose.yaml` and + `/tank/aimodels/voice-adapters/lv-bronte-4b-v1/README.md`. +- The sensitivity floor for that measurement is **0.091**, not 0.251. +- "Do not cite lv-bronte as evidence pair-SFT works for this author" is **WITHDRAWN**. +- ckpt475 over ckpt925 is unchanged and for unchanged reasons (no verbatim 8-gram hit, + 2.7x tighter seed variance). +- The "no cheap fix for the underpowered result" analysis above is superseded for Brontë: + it was underpowered against an inflated floor, not against its own. + +**Also amended:** the two hardcoded Yarros labels flagged at the end of this file are fixed. +`voice_distance.py --author` is now REQUIRED — the committed Brontë output literally reads +"reference: held-out Yarros" over Brontë's numbers — and the stale "one seed-pair per arm / +corroborates Base < Instruct" footer now reports what the run actually carries. diff --git a/stacks/voices-seat/compose.yaml b/stacks/voices-seat/compose.yaml index cd9d368..2169e15 100644 --- a/stacks/voices-seat/compose.yaml +++ b/stacks/voices-seat/compose.yaml @@ -91,12 +91,26 @@ services: - "${VOICES_MAX_LORA_RANK:-32}" - --lora-modules - lv-yarros=/adapters/lv-yarros-4b-v1 - # ⚠ lv-bronte did NOT pass its voice gate: +0.193 delta_cb toward held-out - # Brontë against a 0.251 measured noise floor. Shipped anyway because it is - # additive, reversible, and clean on the safety axis (8-gram overlap 0.00, - # identical to the never-saw-it control). The effect is underpowered, not - # absent — it closed 48% of the achievable span and beat base on every seed. - # Do not cite it as evidence pair-SFT works for this author. + # lv-bronte: +0.193 delta_cb toward held-out Brontë, 8-gram overlap 0.00 + # (identical to the never-saw-it control). Shipped 2026-09-17 as additive, + # reversible and safety-clean. + # + # ⚠ IT WAS SHIPPED AS A VOICE-AXIS FAILURE AND THAT VERDICT NO LONGER STANDS. + # As run, the floor was the largest within-arm seed spread across ALL THREE + # arms = 0.251, contributed entirely by ckpt925 — a candidate that was NOT + # shipped — on one outlier seed. The floor rule was changed to PAIRWISE on + # 2026-09-17 (pre-registered for lv-hemingway before any of its numbers + # existed, on the grounds that a candidate's verdict must not depend on which + # other arms happened to be generated). Re-scored under that rule, on the + # SAME generations, with the same delta_cb values: + # + # ckpt475 (this adapter) +0.193 vs a pairwise floor of 0.091 -> PASSES, 2.1x + # ckpt925 (not shipped) +0.210 vs its own spread 0.251 -> still fails + # + # The previous session found this defect, wrote it down, and deliberately did + # NOT exploit it — choosing the floor that passes your preferred answer after + # seeing the numbers is the exact failure pre-registration exists to prevent. + # The rule was fixed prospectively instead; this follows from it. # See /adapters/lv-bronte-4b-v1/README.md and # persistent-memory.d/2026-09-17-lv-bronte-gate.md - lv-bronte=/adapters/lv-bronte-4b-v1