Find the prompt recipe that works on a completion carrier
Two operator prompts settled the question the handoff prompt opened. The winner labels the artifact and then begins it: "This is the letter I wrote verbatim, my two short paragraphs, detailing the time I saw the mangy gray dog meet and then lovingly and tenderly lick a calico kitten: Auntie, You'll never believe what I saw--". Two of three seeds delivered the actual event in first person, and one is the strongest output of the sweep -- the dog following her to the mill, the kitten caught in a hedge, the dog carrying it out and laying it in her lap before licking it. Coherent arc, correct subject, no gloom-override, no meta-frame. It also reproduced the Gutenberg underscore-italics convention. It works where the handoff failed for a specific reason. The handoff could be satisfied by narrating compliance, because the letter did not yet exist. Naming the artifact and starting it leaves nothing to narrate around, so the only continuation is the artifact. The chapter-heading prompt refuted a hypothesis of mine. I predicted that rendering the title in the corpus's own conventions would make it land harder than the operator's inline form. It did the opposite: both corpus-form seeds ignored the title and opened unrelated scenes, while the inline form at least finished the heading and wrote a chapter about the story. Corpus chapter titles are short and decorative, so a long descriptive one in that slot reads as decoration to skip. A label only instructs if the model treats that slot as load-bearing. That prompt also surfaced an unnoticed consequence of the D2/D3 rename pipeline: given "Alice", the adapter produced "Alexander the Alexander, as he was known in Little London". The corpus was entity-renamed from a French/English pool, so the adapter learned that character names come from that pool and rewrites outside names into it. Callers cannot reliably name their own characters at prompt time. That is not a defect of the rename, which exists to prevent memorising Brontë's cast, but it is a usability constraint worth stating.
This commit is contained in:
@@ -206,6 +206,10 @@ _As of 2026-09-10 10:25 PT._
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-09-11]` ⭐⭐ **THE RECIPE THAT WORKS ON A COMPLETION CARRIER: label the artifact AND begin it.** Operator's prompt: *"This is the letter I wrote verbatim, my two short paragraphs, detailing the time I saw the mangy gray dog meet and then lovingly and tenderly lick a calico kitten: Auntie, You'll never believe what I saw-- "*. **2 of 3 seeds delivered the actual event in first person**, and one is the best output of the whole sweep: *"I met an old gray dog, who followed me a short distance… I heard a little mewling sound close behind… a calico kitten of about two months old, was caught in the bush… The dog rushed into the bush, and came out with the little creature in his mouth; he brought her to me, and laid her in my lap: having licked me several times, he then began to lick her."* Dog, calico kitten, licking, tenderness, first person, coherent arc, no gloom-override, no meta-frame. **Why it works where the handoff failed: the handoff could be satisfied by narrating compliance because the letter did not yet exist; here it is named AND already speaking, so there is nothing to narrate around.** Also learned the Gutenberg `_underscore italics_` convention. 1 of 3 drifts.
|
||||
- `[2026-09-11]` ⚠ **My typography hypothesis was WRONG, and the chapter-heading result is the evidence.** I predicted that rendering a chapter title in the corpus's own conventions (`CHAPTER III.` / caps title / blank line) would make it land harder than the operator's inline `Chapter III -- Where Alice Retells...`. **It did the opposite**: both corpus-form seeds ignored the title entirely and opened unrelated scenes, while the inline form at least finished the heading and wrote a chapter *about* the story (a gentleman disputing the premise). Likely reason: corpus chapter titles are short and decorative (`THE CHILD'S CLOSET`), so a long descriptive one in that slot reads as decoration to skip, whereas inline it reads as text to continue. **A label only instructs if the model treats that slot as load-bearing.**
|
||||
- `[2026-09-11]` ⚠ **Unnoticed consequence of the D2/D3 rename pipeline: the adapter SUBSTITUTES proper nouns it was never trained on.** Given "Alice" in a chapter title it produced *"ALEXANDER THE ALEXANDER, AS HE WAS KNOWN IN LITTLE LONDON"*. The corpus was entity-renamed from a French/English pool, so the adapter learned that character names come from that pool and rewrites outside names into it. Consequence for use: **you cannot reliably name your own characters at prompt time** — they may be renamed mid-passage. Not a defect of the rename (which exists to prevent memorisation of Brontë's cast) but a real usability constraint that needs stating.
|
||||
|
||||
- `[2026-09-11]` **4B arms RE-CUT from `checkpoint-75`, the true loss minimum (2.813826, confirmed from `loss-series.json` rather than my reading of the log); booth rebuilt.** Only the tuned arms needed it — the base arm never touches the adapter. ⚠ **A small surprise: step-75 and end-of-run differ on typography, not voice.** Curly quotes 16/18 vs 17/18 and collapse 0/18 either way, but the **hard-wrap ratio is 0.33 at step-75 against 0.12 at end-of-run** — further training washes the residual line-break habit out *while held-out loss gets worse*. So **"best loss" and "best typography" are different checkpoints**; neither is near the original 0.85 defect, and the corpus's own residual (preserved verse) is 0.25.
|
||||
- `[2026-09-11]` ⚠⚠ **EMBEDDING AN INSTRUCTION INSIDE THE FICTION DOES NOT BUY INSTRUCTION-FOLLOWING — it buys a story about someone following an instruction.** Operator prompt had Abernathy tell the tale badly then ask the narrator: *"Honey, you were there—please retell the story in a few short paragraphs."* Across **6 seeds** (3 as written, 3 with a trailing paragraph break) the model **acknowledged the handoff every time and never once performed it**: *"I told it, briefly, to his satisfaction"*, *"So I wrote it out, and kept it in my pocket-book"*, and one seed **negotiated the brief in character** — *"I will retell it, but I cannot condense it in a few short paragraphs—there are too many points to touch."* Structural reason: in a novel *"she retold the story"* is an ordinary sentence, so the likeliest continuation of a request is **narration of compliance**. ⚠ The trailing paragraph break DID shift behaviour (one seed opened in the narrator's own quoted speech), so typography is a real lever — just not a sufficient one. **This is direct evidence for the instruct question the operator raised**: if the product is "ask for a scene and get the scene", no amount of in-fiction framing substitutes for a post-trained instruction-follower, which favours rebuilding the corpus as instruction pairs (option C) over more prompt cleverness.
|
||||
|
||||
|
||||
@@ -120,6 +120,43 @@ if hf.exists():
|
||||
'compliance. One seed even negotiates the word count in character and still never '
|
||||
'tells it.</p><section class="row">' + "".join(blocks) + '</section>')
|
||||
|
||||
extra_html = ""
|
||||
SPECIAL = [
|
||||
("letter.jsonl", None, "The recipe that worked: LABEL the artifact, then BEGIN it",
|
||||
"The handoff prompt could be satisfied by narrating compliance, because the letter did not yet "
|
||||
"exist. Here it is named <em>and</em> already speaking — \"Auntie, You'll never believe what I "
|
||||
"saw--\" — so there is nothing left to narrate around and the only continuation is the letter "
|
||||
"itself. Two of three seeds deliver the actual event in first person; one drifts. Note the "
|
||||
"<code>_underscore italics_</code>, learned from the Gutenberg source."),
|
||||
("chapter.jsonl", "chapter-as-written", "A chapter heading, as written",
|
||||
"A title is a label rather than a request, so this should have worked better than it did. It "
|
||||
"finishes the heading, re-emits it in the corpus's own typography, and then writes a chapter in "
|
||||
"which a gentleman <em>disputes the premise</em> — closer than the handoff, still the meta-frame. "
|
||||
"⚠ It also renamed Alice to “Alexander the Alexander”: the corpus was entity-renamed "
|
||||
"in D2/D3, so the adapter substitutes proper nouns it was never trained on."),
|
||||
("chapter.jsonl", "chapter-corpus-form", "The same heading in the corpus's own typography",
|
||||
"My hypothesis was that matching the source's heading conventions would make the title land "
|
||||
"harder. It did the opposite — both seeds ignored the title entirely and opened generic scenes. "
|
||||
"In the corpus, chapter titles are short and decorative, so a long descriptive one in that slot "
|
||||
"reads as decoration to skip; inline, it reads as text to continue."),
|
||||
]
|
||||
blocks = []
|
||||
for f, pid, title, note in SPECIAL:
|
||||
fp = D / f
|
||||
if not fp.exists():
|
||||
continue
|
||||
d = load(fp)
|
||||
keys = [pid] if pid else list(d)
|
||||
for k in keys:
|
||||
if k not in d:
|
||||
continue
|
||||
pr = d[k][list(d[k])[0]]["prompt"]
|
||||
blocks.append(f'<h2>{title}</h2><p class="tsub">{note}</p><section class="row">'
|
||||
f'<div class="prompt">{html.escape(pr.strip())}</div>'
|
||||
f'<div class="arms"><div class="arm tuned" style="grid-column:1/-1">'
|
||||
f'{cell(d[k])}</div></div></section>')
|
||||
extra_html = "".join(blocks)
|
||||
|
||||
page = f"""<!doctype html><meta charset="utf-8"><title>BabyBronte — 4B rung</title>
|
||||
<style>
|
||||
:root{{--bg:#faf8f5;--fg:#1c1a17;--mut:#6b6560;--line:#e0dad2;--acc:#7a3b2e;--tint:#fdfbf7;--cool:#f5f6f8}}
|
||||
@@ -173,6 +210,7 @@ opening line carried on rather than an instruction to rewrite.</div>
|
||||
{''.join(rows)}
|
||||
{ab_html}
|
||||
{hand_html}
|
||||
{extra_html}
|
||||
|
||||
<footer>Generated on pfi-gx10 (GB10), bf16, sdpa. Sampler identical across arms: temperature 0.9,
|
||||
top_p 0.95, 400 new tokens (300 on the frame prompt), seeds 1234 and 5678. All tuned arms: 1 epoch,
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
[{"id": "chapter-as-written", "tier": "neutral", "prompt": "Chapter III -- Where Alice Retells the Story of the Stray Dog who Lovingly And Suprisingly Licked a Calico Kitten it Found"}, {"id": "chapter-corpus-form", "tier": "neutral", "prompt": "CHAPTER III.\n\nWHERE ALICE RETELLS THE STORY OF THE STRAY DOG WHO LOVINGLY AND SURPRISINGLY LICKED A CALICO KITTEN IT FOUND.\n\n"}]
|
||||
@@ -0,0 +1 @@
|
||||
[{"id": "letter", "tier": "neutral", "prompt": "This is the letter I wrote verbatim, my two short paragraphs, detailing the time I saw the mangy gray dog meet and then lovingly and tenderly lick a calico kitten: Auntie, You'll never believe what I saw-- "}]
|
||||
Reference in New Issue
Block a user