vh
e47295d2ce
The 4B tuned arms now come from checkpoint-75, the actual loss minimum at 2.813826 confirmed from loss-series.json rather than from reading the training log. Only the tuned arms needed re-cutting; the base arm never touches the adapter. One surprise in the comparison: step-75 and end-of-run differ on typography rather than voice. Curly quotes read 16 of 18 against 17, collapse is 0 either way, but the hard-wrap ratio is 0.33 at step-75 against 0.12 at the end. Further training washes the residual line-break habit out while held-out loss gets worse, so best loss and best typography are different checkpoints. Neither is near the original 0.85 defect and the corpus's own residual from preserved verse is 0.25. The operator's second prompt embeds the instruction inside the fiction: Abernathy tells the tale badly and then asks the narrator to retell it in a few short paragraphs. Across six seeds the model acknowledged the handoff every time and never once performed it. It wrote "I told it, briefly, to his satisfaction", and "So I wrote it out, and kept it in my pocket-book", and one seed negotiated the brief in character, objecting that it could not condense the story into a few short paragraphs because there were too many points to touch. The reason is structural rather than a failure of the adapter. In a novel, "she retold the story" is an ordinary sentence, so the likeliest continuation of a request is narration of compliance rather than compliance. A trailing paragraph break did shift the behaviour -- one seed opened in the narrator's own quoted speech -- so typography is a real lever, just not a sufficient one. That is direct evidence on the instruct question raised earlier: if the product is asking for a scene and getting the scene, no amount of in-fiction framing substitutes for a post-trained instruction-follower, which favours rebuilding the corpus as instruction pairs over more prompt cleverness.
1 line
884 B
JSON
1 line
884 B
JSON
[{"id": "handoff", "tier": "neutral", "prompt": "He said, \"We saw a stray dog, a ragged creature, who encountered a stray calico kitten; and we watched, sure it would harm the kitten, but instead it licked it with love and care and tenderness.\" Then he said, \"I lack the words,\" and turned to me and said, \"Honey, you were there\u2014please retell the story in a few short paragraphs, since you are so much better at it than I.\""}, {"id": "handoff-break", "tier": "neutral", "prompt": "He said, \"We saw a stray dog, a ragged creature, who encountered a stray calico kitten; and we watched, sure it would harm the kitten, but instead it licked it with love and care and tenderness.\" Then he said, \"I lack the words,\" and turned to me and said, \"Honey, you were there\u2014please retell the story in a few short paragraphs, since you are so much better at it than I.\"\n\n"}] |