54 lines
3.8 KiB
Markdown
54 lines
3.8 KiB
Markdown
# `[2026-09-16]` The lv-* voice line: Option C proved, lv-yarros shipped, lv-hemingway training
|
||
|
||
⭐⭐ **INSTRUCTION-PAIR SFT BEATS RAW-TEXT TRAINING FOR AUTHOR VOICE, AND THE INCUMBENT NEVER
|
||
CLEARED ITS OWN CONTROL.** Measured n=120 per arm, 30 in-genre beats from HELD-OUT val passages
|
||
× 4 seeds, all arms re-measured in one session on one box:
|
||
|
||
| arm | delta_cb (lower = more Yarros) | vs base control | 8-gram overlap |
|
||
|---|---|---|---|
|
||
| pairs 2ep ckpt-1650 | **0.410** | +0.289 ✅ | 0.12 |
|
||
| pairs 3ep ckpt-1650 (**shipped**) | 0.438 | +0.262 ✅ | **0.09** |
|
||
| raw-text instruct (incumbent) | 0.558 | +0.141 ❌ **inside the 0.153 floor** | 0.14 |
|
||
| base-unadapted (control) | 0.700 | — | 0.07 |
|
||
|
||
Same-author target 0.463 (held-out Yarros vs itself). ⚠ **The two pair arms are NOT
|
||
distinguishable on voice** — 0.028 against a 0.153 floor. The 3ep checkpoint was chosen on the
|
||
axes that ARE resolvable: better held-out fit (2.3126 vs 2.3264), less overshoot (0.06 vs 0.10),
|
||
and verbatim overlap nearest the never-saw-it control.
|
||
|
||
⭐ **THE RECIPE IS TWO EPOCHS ON A THREE-EPOCH SCHEDULE, not three epochs.** Launch `--epochs 3`;
|
||
the minimum lands at step 1650 **inside epoch two** and epoch three overfits (2.3126 → 2.3882,
|
||
flat). The entire gain over a 2-epoch run came from the stretched cosine keeping the LR alive —
|
||
at step 1600 the 3ep run was at 2.9e-05 where the 2ep run had annealed to 2e-07. ⚠⚠ **A
|
||
resume-and-append-one-epoch is a NO-OP for exactly that reason** (lr 2.3e-09 at step 1670): it
|
||
must be a fresh run with the longer schedule.
|
||
|
||
⭐ **THE SAFETY PROPERTY: the model writes the INSTRUCTION, never the RESPONSE.** Every response
|
||
is real renamed prose; only the beat is machine-written, so voice is inherited rather than
|
||
synthesised. Memorisation checked with both controls (positive control saturates at 160): the
|
||
shipped arm sits at 0.09 against a 0.07 never-saw-it baseline and BELOW the raw-text arm's 0.14.
|
||
|
||
⚠ **THE v1 DECISION RULE WAS WELL-FORMED AND MEASURED THE WRONG THING**, and the amendment is
|
||
recorded in `scripts/yarros-corpus/score_beats.py` with v1 retained verbatim. It gated on
|
||
in-band / on-beat / ran-on — and **base-unadapted scores in-band 0.96**. Instruction-following is
|
||
something Qwen3-4B-Instruct ships with, so those axes detect only DAMAGE, never the benefit an
|
||
adapter exists to buy. v2 gates on voice (delta_cb vs control beyond the floor) + not-copied
|
||
(8-gram overlap near control) + no-damage (overshoot). ⚠ on-beat's −0.27 was outside the floor
|
||
and is dropped from the gate, **not explained away** — the keyword proxy punishes prose that
|
||
DRAMATISES "she mocks him" rather than echoing the word, but three read samples is an anecdote.
|
||
|
||
⭐ **A 5-BEAT FIXTURE HAD A NOISE FLOOR OF 0.800 AND MANUFACTURED A +0.45 RESULT.** At n=20 the
|
||
pilot looked like a clear in-band win; at n=120 the same gap was +0.08, inside a 0.233 floor.
|
||
One sample moves a rate by 0.2 when there are five. The 30-beat in-genre fixture (built from
|
||
held-out val pairs, `~/beats-yarros-30.json`) is the instrument; the Brontë stray-dog/kitten
|
||
fixture was also the wrong GENRE — "He licked her clean" came back as explicit sex.
|
||
|
||
⚠ **THE HARNESS TRUNCATES AT THE FIRST BLANK LINE and that surface reported the pair arm as
|
||
"19 words, off-beat 0.10"** when the untruncated output was 90–132 words with the beat rendered
|
||
in a later block. `score_beats.py --metric-source raw|paragraph` keeps both views and the verdict
|
||
names which it used. Same family as `feedback_filters_that_silently_narrow_the_window`.
|
||
|
||
**Artefacts.** `scripts/yarros-corpus/{build_sft_pairs,train_pairs_lora,score_beats,
|
||
memorization_check}.py`; commits `9b3d3c8` `90ed506` `713e83d` `efb7345` `7505124`. Booth
|
||
(24h TTL) was `http://10.100.10.50:8090/b/babyyarros-beats/` — six beats × four arms, blind-labelled.
|