3.8 KiB
[2026-09-16] The lv-* voice line: Option C proved, lv-yarros shipped, lv-hemingway training
⭐⭐ INSTRUCTION-PAIR SFT BEATS RAW-TEXT TRAINING FOR AUTHOR VOICE, AND THE INCUMBENT NEVER CLEARED ITS OWN CONTROL. Measured n=120 per arm, 30 in-genre beats from HELD-OUT val passages × 4 seeds, all arms re-measured in one session on one box:
| arm | delta_cb (lower = more Yarros) | vs base control | 8-gram overlap |
|---|---|---|---|
| pairs 2ep ckpt-1650 | 0.410 | +0.289 ✅ | 0.12 |
| pairs 3ep ckpt-1650 (shipped) | 0.438 | +0.262 ✅ | 0.09 |
| raw-text instruct (incumbent) | 0.558 | +0.141 ❌ inside the 0.153 floor | 0.14 |
| base-unadapted (control) | 0.700 | — | 0.07 |
Same-author target 0.463 (held-out Yarros vs itself). ⚠ The two pair arms are NOT distinguishable on voice — 0.028 against a 0.153 floor. The 3ep checkpoint was chosen on the axes that ARE resolvable: better held-out fit (2.3126 vs 2.3264), less overshoot (0.06 vs 0.10), and verbatim overlap nearest the never-saw-it control.
⭐ THE RECIPE IS TWO EPOCHS ON A THREE-EPOCH SCHEDULE, not three epochs. Launch --epochs 3;
the minimum lands at step 1650 inside epoch two and epoch three overfits (2.3126 → 2.3882,
flat). The entire gain over a 2-epoch run came from the stretched cosine keeping the LR alive —
at step 1600 the 3ep run was at 2.9e-05 where the 2ep run had annealed to 2e-07. ⚠⚠ A
resume-and-append-one-epoch is a NO-OP for exactly that reason (lr 2.3e-09 at step 1670): it
must be a fresh run with the longer schedule.
⭐ THE SAFETY PROPERTY: the model writes the INSTRUCTION, never the RESPONSE. Every response is real renamed prose; only the beat is machine-written, so voice is inherited rather than synthesised. Memorisation checked with both controls (positive control saturates at 160): the shipped arm sits at 0.09 against a 0.07 never-saw-it baseline and BELOW the raw-text arm's 0.14.
⚠ THE v1 DECISION RULE WAS WELL-FORMED AND MEASURED THE WRONG THING, and the amendment is
recorded in scripts/yarros-corpus/score_beats.py with v1 retained verbatim. It gated on
in-band / on-beat / ran-on — and base-unadapted scores in-band 0.96. Instruction-following is
something Qwen3-4B-Instruct ships with, so those axes detect only DAMAGE, never the benefit an
adapter exists to buy. v2 gates on voice (delta_cb vs control beyond the floor) + not-copied
(8-gram overlap near control) + no-damage (overshoot). ⚠ on-beat's −0.27 was outside the floor
and is dropped from the gate, not explained away — the keyword proxy punishes prose that
DRAMATISES "she mocks him" rather than echoing the word, but three read samples is an anecdote.
⭐ A 5-BEAT FIXTURE HAD A NOISE FLOOR OF 0.800 AND MANUFACTURED A +0.45 RESULT. At n=20 the
pilot looked like a clear in-band win; at n=120 the same gap was +0.08, inside a 0.233 floor.
One sample moves a rate by 0.2 when there are five. The 30-beat in-genre fixture (built from
held-out val pairs, ~/beats-yarros-30.json) is the instrument; the Brontë stray-dog/kitten
fixture was also the wrong GENRE — "He licked her clean" came back as explicit sex.
⚠ THE HARNESS TRUNCATES AT THE FIRST BLANK LINE and that surface reported the pair arm as
"19 words, off-beat 0.10" when the untruncated output was 90–132 words with the beat rendered
in a later block. score_beats.py --metric-source raw|paragraph keeps both views and the verdict
names which it used. Same family as feedback_filters_that_silently_narrow_the_window.
Artefacts. scripts/yarros-corpus/{build_sft_pairs,train_pairs_lora,score_beats, memorization_check}.py; commits 9b3d3c8 90ed506 713e83d efb7345 7505124. Booth
(24h TTL) was http://10.100.10.50:8090/b/babyyarros-beats/ — six beats × four arms, blind-labelled.