Re-cut the 4B arms from checkpoint-75 and test an in-fiction instruction

The 4B tuned arms now come from checkpoint-75, the actual loss minimum at 2.813826
confirmed from loss-series.json rather than from reading the training log. Only the
tuned arms needed re-cutting; the base arm never touches the adapter.

One surprise in the comparison: step-75 and end-of-run differ on typography rather
than voice. Curly quotes read 16 of 18 against 17, collapse is 0 either way, but
the hard-wrap ratio is 0.33 at step-75 against 0.12 at the end. Further training
washes the residual line-break habit out while held-out loss gets worse, so best
loss and best typography are different checkpoints. Neither is near the original
0.85 defect and the corpus's own residual from preserved verse is 0.25.

The operator's second prompt embeds the instruction inside the fiction: Abernathy
tells the tale badly and then asks the narrator to retell it in a few short
paragraphs. Across six seeds the model acknowledged the handoff every time and
never once performed it. It wrote "I told it, briefly, to his satisfaction", and
"So I wrote it out, and kept it in my pocket-book", and one seed negotiated the
brief in character, objecting that it could not condense the story into a few short
paragraphs because there were too many points to touch.

The reason is structural rather than a failure of the adapter. In a novel, "she
retold the story" is an ordinary sentence, so the likeliest continuation of a
request is narration of compliance rather than compliance.

A trailing paragraph break did shift the behaviour -- one seed opened in the
narrator's own quoted speech -- so typography is a real lever, just not a
sufficient one. That is direct evidence on the instruct question raised earlier: if
the product is asking for a scene and getting the scene, no amount of in-fiction
framing substitutes for a post-trained instruction-follower, which favours
rebuilding the corpus as instruction pairs over more prompt cleverness.

persistent-memory.md

@@ -206,6 +206,9 @@ _As of 2026-09-10 10:25 PT._
 
 ## Recent decisions
 
+- `[2026-09-11]` **4B arms RE-CUT from `checkpoint-75`, the true loss minimum (2.813826, confirmed from `loss-series.json` rather than my reading of the log); booth rebuilt.** Only the tuned arms needed it — the base arm never touches the adapter. ⚠ **A small surprise: step-75 and end-of-run differ on typography, not voice.** Curly quotes 16/18 vs 17/18 and collapse 0/18 either way, but the **hard-wrap ratio is 0.33 at step-75 against 0.12 at end-of-run** — further training washes the residual line-break habit out *while held-out loss gets worse*. So **"best loss" and "best typography" are different checkpoints**; neither is near the original 0.85 defect, and the corpus's own residual (preserved verse) is 0.25.
+- `[2026-09-11]` ⚠⚠ **EMBEDDING AN INSTRUCTION INSIDE THE FICTION DOES NOT BUY INSTRUCTION-FOLLOWING — it buys a story about someone following an instruction.** Operator prompt had Abernathy tell the tale badly then ask the narrator: *"Honey, you were there—please retell the story in a few short paragraphs."* Across **6 seeds** (3 as written, 3 with a trailing paragraph break) the model **acknowledged the handoff every time and never once performed it**: *"I told it, briefly, to his satisfaction"*, *"So I wrote it out, and kept it in my pocket-book"*, and one seed **negotiated the brief in character** — *"I will retell it, but I cannot condense it in a few short paragraphs—there are too many points to touch."* Structural reason: in a novel *"she retold the story"* is an ordinary sentence, so the likeliest continuation of a request is **narration of compliance**. ⚠ The trailing paragraph break DID shift behaviour (one seed opened in the narrator's own quoted speech), so typography is a real lever — just not a sufficient one. **This is direct evidence for the instruct question the operator raised**: if the product is "ask for a scene and get the scene", no amount of in-fiction framing substitutes for a post-trained instruction-follower, which favours rebuilding the corpus as instruction pairs (option C) over more prompt cleverness.
+
 - `[2026-09-11]` **R49 SWEEP COMPLETE — 4B closes the continuity gap, and the carrier ladder is clean: 3.329 → 3.018 → 2.814 held-out** (0.6B / 1.7B / 4B, all on the same unwrapped corpus sha `77f37057b2782e49`, seed 4919, 159 steps, 5,210,112 tokens — carrier size the only variable). Deltas **0.311 then 0.204**: diminishing but still real. Booth: `http://10.100.10.50:8090/b/babybronte-4b/`. **4B tuned has the best voice saturation of any rung — curly quotes 17/18 against its own base arm's 1/18, collapse 0/18 against 4/18** — and, the thing the rung existed to test, **scene-level continuity HOLDS**: it produces a named character with motivated dialogue, a navigable spatial layout and a physical description in one passage, where 1.7B wrote pretty but eventless prose (opening doors, looking at stars). On the letter prompt it opens the letter, promises to quote it, and then actually quotes it across a paragraph break.
 - `[2026-09-11]` ⚠⚠ **4B is the FIRST rung to OVERFIT inside one epoch, which inverts my earlier "one epoch is right for this corpus" call.** Series 2.832 · 2.816 · **2.814** · 2.820 · 2.824 · 2.825 · 2.825 — minimum at ~step 75, then it TURNS and settles worse. 0.6B and 1.7B both plateaued with no turn, so **the optimal epoch count shrinks as the carrier grows** — 4B wants roughly half an epoch. ⚠ **Consequence: the shipped `adapter/` at `h02-4b-1ep/` is NOT the best checkpoint** (it is the end-of-run 2.825); the step-75 checkpoint at 2.814 is, and it exists only because `save_steps=25` was set. The voice test used the end-of-run adapter, so the booth understates 4B by ~0.011 nats. Re-cut the arms off the step-75 checkpoint before any adjudication.
 - `[2026-09-11]` **The tone-override appears to close at 4B too.** On the operator's Abernathy frame prompt ("a *wonderful* story"), 1.7B held the frame on every seed but **2 of 4 killed the animals anyway**; 4B kept them alive on **2 of 2** and one seed did something new — the narrator *doubts Abernathy's story* ("I felt sure the thing was a lie"), then supplies a parallel childhood memory of his own puppy and his sister's kitten to explain the doubt. That is a narrator with an interior position on the tale being told. ⚠ n=2 per arm; directionally right, not established.

scripts/r49-corpus/build_booth4b.py

@@ -22,7 +22,7 @@ from pathlib import Path
 
 D = Path(sys.argv[1])
 ARMS = [("4b-base.jsonl", "4B base", "Qwen3-4B-Base, no adapter", ""),
-        ("4b-tuned.jsonl", "4B tuned", "+ H02 LoRA, 1 epoch, seed 4919", "tuned"),
+        ("4b-tuned.jsonl", "4B tuned", "+ H02 LoRA, <b>checkpoint-75</b> (the loss minimum)", "tuned"),
         ("1p7b-tuned.jsonl", "1.7B tuned", "the rung below, same corpus &amp; seed", "small")]
 AB = [("4b-base-abernathy.jsonl", "4B base", ""), ("4b-tuned-abernathy.jsonl", "4B tuned", "tuned")]
 
@@ -97,6 +97,29 @@ if all((D / f).exists() for f, _, _ in AB):
                f'<section class="row"><div class="prompt">{html.escape(pr)}</div>'
                f'<div class="arms two">{cols}</div></section>')
 
+hand_html = ""
+hf = D / "handoff.jsonl"
+if hf.exists():
+    hd = load(hf)
+    blocks = []
+    for pid, label in (("handoff", "as written"), ("handoff-break", "identical, plus a trailing paragraph break")):
+        if pid not in hd:
+            continue
+        pr = hd[pid][list(hd[pid])[0]]["prompt"]
+        blocks.append(f'<div class="prompt"><span class="pid">{html.escape(label)}</span>'
+                      f'{html.escape(pr.strip())}</div>'
+                      f'<div class="arms"><div class="arm tuned" style="grid-column:1/-1">'
+                      f'{cell(hd[pid])}</div></div>')
+    hand_html = ('<h2>The embedded-instruction prompt</h2><p class="tsub">The instruction lives '
+                 'INSIDE the fiction — Abernathy asks the narrator to retell the story — which is the '
+                 'only way to hand an instruction to a completion model. Watch what it does with the '
+                 'request: across every seed it narrates the retelling rather than performing it '
+                 '("I told it, briefly", "So I wrote it out", "I will retell it, but I cannot '
+                 'condense it"). In a novel, <em>she retold the story</em> is an ordinary sentence, '
+                 'so the likeliest continuation of a request is narration of compliance — not '
+                 'compliance. One seed even negotiates the word count in character and still never '
+                 'tells it.</p><section class="row">' + "".join(blocks) + '</section>')
+
 page = f"""<!doctype html><meta charset="utf-8"><title>BabyBronte — 4B rung</title>
 <style>
 :root{{--bg:#faf8f5;--fg:#1c1a17;--mut:#6b6560;--line:#e0dad2;--acc:#7a3b2e;--tint:#fdfbf7;--cool:#f5f6f8}}
@@ -149,6 +172,7 @@ opening line carried on rather than an instruction to rewrite.</div>
 
 {''.join(rows)}
 {ab_html}
+{hand_html}
 
 <footer>Generated on pfi-gx10 (GB10), bf16, sdpa. Sampler identical across arms: temperature 0.9,
 top_p 0.95, 400 new tokens (300 on the frame prompt), seeds 1234 and 5678. All tuned arms: 1 epoch,

scripts/r49-corpus/handoff.json

@@ -0,0 +1 @@
+[{"id": "handoff", "tier": "neutral", "prompt": "He said, \"We saw a stray dog, a ragged creature, who encountered a stray calico kitten; and we watched, sure it would harm the kitten, but instead it licked it with love and care and tenderness.\" Then he said, \"I lack the words,\" and turned to me and said, \"Honey, you were there\u2014please retell the story in a few short paragraphs, since you are so much better at it than I.\""}, {"id": "handoff-break", "tier": "neutral", "prompt": "He said, \"We saw a stray dog, a ragged creature, who encountered a stray calico kitten; and we watched, sure it would harm the kitten, but instead it licked it with love and care and tenderness.\" Then he said, \"I lack the words,\" and turned to me and said, \"Honey, you were there\u2014please retell the story in a few short paragraphs, since you are so much better at it than I.\"\n\n"}]