diff --git a/persistent-memory.md b/persistent-memory.md index 5cfc53e..5568a1e 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -206,6 +206,9 @@ _As of 2026-09-10 10:25 PT._ ## Recent decisions +- `[2026-09-11]` ⭐⭐ **THE INSTRUCT PROBE ANSWERS ITS QUESTION: voice and instruction-following DO coexist. Option C is de-risked.** `Qwen3-4B` **instruct** (not `-Base`), same corpus/seed/steps so the carrier is the only variable; best checkpoint `checkpoint-150` picked by loss (applying the 4B-Base lesson automatically this time). **Voice installed at full strength — curly quotes 16/18, IDENTICAL to the 4B-Base tuned arm's 16/18**, against the unadapted control's 1/18, and **task-leak 0/18 vs the base carrier's 4/18**. So the assistant prior did NOT block Brontë, which was the central risk. **Instruction-following SURVIVED: 10/10 on-beat through the chat template**, same as the untuned control. ⚠ **The cost is length discipline, not comprehension** — in-band 10/10 → **6/10**, median 124w → 140w. Training on Victorian prose made it wordier, a soft degradation rather than a break. ⚠ **Held-out 2.908 vs 4B-Base's 2.814** — the instruct carrier fits the corpus **0.094 nats worse** and **plateaus without turning** where base overfit at step 75: the assistant prior competes for capacity, so it absorbs less rather than overfitting more. +- `[2026-09-11]` ⚠ **What raw-continuation training on an instruct carrier does NOT fix: the plot furniture.** Reading the product artifact, the tuned-instruct arm renders the beat and then drags the referent — *"He licked her clean… my master thus—my husband thus"*, turning the dog into a man, because Brontë's corpus is about masters and husbands. Another beat ran 247w and gave the narrator a list of duties. **This is exactly what instruction-PAIR training is for** — pairs teach "render this and stop", continuation teaches "keep writing Victorian prose". So the probe de-risks option C without substituting for it. ⚠ Also: my `ran_on` metric is uninformative on this job (10/10 on BOTH arms) because a single paragraph contains no blank line — it measures "no paragraph break found", which is correct and useless here. Do not read it as a finding. + - `[2026-09-11]` ⭐⭐⭐ **SKALDSONG'S SHAPE SETTLES THE ARCHITECTURE: the adapted completion carrier CANNOT do beat→paragraph, and an instruct model can. Option C (instruct carrier + corpus rebuilt as instruction→response pairs) is now evidence-backed, not opinion.** Operator's requirement: *"skaldsong will want to write story beats which are a sentence, and have the LLM expound on that sentence to a paragraph and stitch it together."* Booth: `http://10.100.10.50:8090/b/skaldsong-beats/`. **Adapted 4B (checkpoint-75): TEN prompt formats × 3 seeds = 30 samples, ZERO that reliably render the beat** — bare, para-break, labelled, epigraph, fewshot(1), fewshot-bare, fewshot3, elaborate, recount, label-begin. Every one drifts, frames, or truncates. Root cause is structural: *"write a paragraph **about** this sentence"* is an instruction, and a completion model has no mechanism for *about* — it continues the text it is given. ⚠⚠ **Two formats leaked PRETRAINING TASK DATA**: `para-break` emitted an NLI multiple-choice item (*"Does it follow that... OPTIONS: (1). yes (2). it is not possible to tell"*) and `label-begin` a grammar-correction exercise (*"CORRECTION: ... The passage appears to be a sentence fragment"*). A standalone sentence plus a blank line looks exactly like a dataset entry; **style adaptation does not remove base-model task artifacts.** **Instruct arm (`gen` seat + style prompt, no adapter): 10/10 samples inside the requested 90–140 band (124–148w, median 130), every one on-beat, zero drift** — but the voice is generic literary pastiche, abstract-noun-heavy and over-written, not Brontë. **So: voice without direction vs direction without voice; the product needs both.** ⚠ **This applies to Yarros identically** — the carrier question is orthogonal to the author, so the next corpus must NOT re-run this experiment. - `[2026-09-11]` ⚠ **Stitching has its own failure mode, visible in the booth's Panel C: independently-generated paragraphs drift in POINT OF VIEW.** By beat 4 of 5 the narrator is simultaneously watching the girl carry the animals and carrying them herself ("their weight a strange, heavy secret carried between my ribs"). Each paragraph was generated with no knowledge of the others. **A real stitcher must feed prior paragraphs back as context**, which also means the instruction-pair corpus should include multi-paragraph continuity examples, not just isolated beat→paragraph pairs. diff --git a/scripts/r49-corpus/chain-instruct-probe.sh b/scripts/r49-corpus/chain-instruct-probe.sh new file mode 100644 index 0000000..ace4ab8 --- /dev/null +++ b/scripts/r49-corpus/chain-instruct-probe.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# Chained after the instruct probe: run the tests that decide whether it worked. +# +# ⚠ PICKS THE BEST CHECKPOINT, not the end-of-run adapter. 4B-Base turned at step 75 +# and its shipped adapter was the WORSE weights; that cost a re-cut. Applying the +# lesson: read loss-series.json, take the argmin, and say which one was chosen. +# +# Four arms, and the CONTROL is the one that makes the rest readable: +# 1. tuned + chat template -- did instruction-following SURVIVE the training? +# 2. UNTUNED + chat template -- the control. Panel B used the 27B gen seat, so +# without a 4B control any degradation is unattributable between "the adapter +# broke it" and "a 4B is just weaker at this than a 27B". +# 3. tuned + raw continuation, the 9 voice prompts -- comparable to every rung. +# 4. tuned + the plot-furniture prompts -- the operator's point that a completion +# carrier reaches for Brontë's devices instead of Skaldsong's. If the instruct +# prior resists the gloom-override, that shows up here. +set -uo pipefail +PREV=/home/infra-ops/r49-runs/h02-4b-instruct-1ep +OUT=/home/infra-ops/r49-runs/instruct-probe +V=/home/infra-ops/ml/.venv/bin/python +BASE=/home/infra-ops/carriers/Qwen3-4B-Instruct +exec >> /home/infra-ops/r49-runs/chain-instruct-probe.log 2>&1 +echo "=== $(date -Is) armed, waiting on $PREV" +while [ -f "$PREV/run.pid" ] && kill -0 "$(cat "$PREV/run.pid")" 2>/dev/null; do sleep 60; done +echo "=== $(date -Is) training finished" +[ -f "$PREV/adapter/adapter_model.safetensors" ] || { echo "=== REFUSING: no adapter"; exit 1; } + +CK=$($V - <<'PY' +import json +d = json.load(open("/home/infra-ops/r49-runs/h02-4b-instruct-1ep/loss-series.json")) +rows = [(r["step"], r["eval_loss"]) for r in + (d if isinstance(d, list) else d.get("log_history", d.get("series", []))) + if isinstance(r, dict) and r.get("eval_loss") is not None] +best = min(rows, key=lambda r: r[1]) +import os +p = f"/home/infra-ops/r49-runs/h02-4b-instruct-1ep/checkpoints/checkpoint-{best[0]}" +print(p if os.path.isdir(p) else "/home/infra-ops/r49-runs/h02-4b-instruct-1ep/adapter") +PY +) +echo "=== best checkpoint selected: $CK" +cd /home/infra-ops/r49-prep +$V gen_beats_chat.py --base "$BASE" --adapter "$CK" --beats beats.json --arm instruct-tuned-chat --out "$OUT/tuned-chat.jsonl" +$V gen_beats_chat.py --base "$BASE" --beats beats.json --arm instruct-untuned-chat --out "$OUT/untuned-chat.jsonl" +$V gen_voice_test.py --base "$BASE" --adapter "$CK" --arm instruct-tuned --prompts voice_prompts.json --out "$OUT/voice.jsonl" +for pf in abernathy.json letter.json; do + $V gen_voice_test.py --base "$BASE" --adapter "$CK" --arm instruct-tuned \ + --prompts "$pf" --seeds 1234 5678 4242 --max-new-tokens 320 \ + --out "$OUT/${pf%.json}.jsonl" +done +echo "=== $(date -Is) instruct probe arms written to $OUT" diff --git a/scripts/r49-corpus/gen_beats_chat.py b/scripts/r49-corpus/gen_beats_chat.py new file mode 100644 index 0000000..2c9b99e --- /dev/null +++ b/scripts/r49-corpus/gen_beats_chat.py @@ -0,0 +1,97 @@ +"""Beat → paragraph through the CHAT TEMPLATE, which is the product's real shape. + +The point of this script is to ask the one question the whole instruct experiment +rests on: after training raw Brontë continuation text into an instruct model, does +it still take direction? Raw text trained into a model whose weights expect +<|im_start|> framing can degrade the template behaviour, and if it has, the adapter +bought voice at the cost of the only capability Skaldsong needs. + +So this drives the model exactly as Skaldsong would -- system prompt stating the job +and the length, user message carrying the beat -- and reports the three things that +decide whether the answer is usable: + + on-beat did it render THIS beat, or wander into Brontë's own plot furniture + in-band did it honour the length, which is the cheapest proxy for "took direction" + ran-on did it close a paragraph, or keep going into the next beat's territory + +Operator's constraint, and it is the one that rules out fixing this downstream: if a +frontier model has to judge every paragraph, the tiny model has no purpose. So these +have to be checkable without one. +""" +from __future__ import annotations +import argparse, json, re, time +from pathlib import Path +import torch +from transformers import AutoModelForCausalLM, AutoTokenizer + +SYS = ("You expand a single story beat into ONE paragraph of prose in the manner of Charlotte " + "Brontë — her diction, sentence rhythm and first-person retrospective narration, mid-19th " + "century. Render the beat itself; do not move past it, do not add a new scene, do not " + "comment. Output the paragraph only, 90–140 words.") + +ap = argparse.ArgumentParser() +ap.add_argument("--base", required=True) +ap.add_argument("--adapter", default=None) +ap.add_argument("--beats", required=True) +ap.add_argument("--out", required=True) +ap.add_argument("--arm", required=True) +ap.add_argument("--seeds", type=int, nargs="+", default=[1234, 5678]) +ap.add_argument("--max-new-tokens", type=int, default=320) +ap.add_argument("--temperature", type=float, default=0.9) +ap.add_argument("--top-p", type=float, default=0.95) +a = ap.parse_args() + +beats = json.loads(Path(a.beats).read_text()) +tok = AutoTokenizer.from_pretrained(a.base) +if tok.chat_template is None: + raise SystemExit("REFUSING: this carrier has no chat template -- it is not an instruct build") +model = AutoModelForCausalLM.from_pretrained(a.base, dtype=torch.bfloat16, + attn_implementation="sdpa").to("cuda") +if a.adapter: + from peft import PeftModel + model = PeftModel.from_pretrained(model, a.adapter) + nz = sum(1 for m in model.modules() if hasattr(m, "lora_B") + and float(m.lora_B["default"].weight.abs().sum()) > 0) + tot = sum(1 for m in model.modules() if hasattr(m, "lora_B")) + print(f"[gen] adapter bound: {nz}/{tot} lora_B tensors non-zero", flush=True) + if nz == 0: + raise SystemExit("REFUSING: adapter applied but every lora_B is zero") +model.eval() + +STOP = re.compile(r"\n\s*\n") +DROP = {"the", "a", "an", "and", "or", "but", "in", "on", "at", "to", "of", "his", "her", + "he", "she", "it", "was", "were", "had", "would", "not", "be", "by", "as", + "with", "for", "from", "that", "this", "up", "down"} + +out = Path(a.out); out.parent.mkdir(parents=True, exist_ok=True) +t0 = time.time() +with out.open("w", encoding="utf-8") as fh: + for b in beats: + for seed in a.seeds: + torch.manual_seed(seed) + text = tok.apply_chat_template( + [{"role": "system", "content": SYS}, + {"role": "user", "content": "BEAT: " + b["beat"]}], + tokenize=False, add_generation_prompt=True, + **({"enable_thinking": False} if "enable_thinking" in (tok.chat_template or "") else {})) + ids = tok(text, return_tensors="pt", add_special_tokens=False).to("cuda") + with torch.no_grad(): + g = model.generate(**ids, do_sample=True, temperature=a.temperature, + top_p=a.top_p, max_new_tokens=a.max_new_tokens, + pad_token_id=tok.eos_token_id) + raw = tok.decode(g[0][ids["input_ids"].shape[1]:], skip_special_tokens=True).strip() + m = STOP.search(raw) + para = (raw[:m.start()] if m else raw).strip() + kws = [w for w in re.findall(r"[a-z']+", b["beat"].lower()) + if w not in DROP and len(w) > 3] + hit = sum(1 for k in kws if k[:5] in para.lower()) + w = len(para.split()) + fh.write(json.dumps({"format": a.arm, "id": b["id"], "beat": b["beat"], "seed": seed, + "prompt": "BEAT: " + b["beat"], "paragraph": para, + "ran_on": m is None, "words": w, + "in_band": 90 <= w <= 140, + "beat_keywords": kws, "keyword_hits": hit}) + "\n") + print(f" {b['id']:>4} seed={seed} {w:>4}w kw {hit}/{len(kws)} " + f"{'in-band' if 90 <= w <= 140 else 'OUT-OF-BAND'}" + f"{' RAN-ON' if m is None else ''}", flush=True) +print(f"[gen] {a.arm} -> {out} in {time.time()-t0:.0f}s", flush=True) diff --git a/scripts/r49-corpus/launch-h02-4b-instruct.sh b/scripts/r49-corpus/launch-h02-4b-instruct.sh new file mode 100644 index 0000000..33d6130 --- /dev/null +++ b/scripts/r49-corpus/launch-h02-4b-instruct.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# R49 H02 — the instruct probe. Qwen3-4B (INSTRUCT, not -Base), 1 epoch, seed 4919. +# +# WHY, in the operator's words: "I think we need the instruct model trained -- if +# nothing else to see what our options are." Three arguments drove it and all three +# are about the REGIME rather than the output quality: +# +# 1. A completion carrier reaches for BRONTË's plot devices, not Skaldsong's. We +# measured that: the gloom-override killed the animals on a prompt that asked +# for a wonderful story, and the rename pool rewrote "Alice" into "Alexander +# the Alexander". When the author owns the plot, importing hers is a conflict, +# not a blemish. +# 2. Drift scales with generated length. In the ten-format bake-off the short +# samples stayed nearest the beat and the 224-253 word ones wandered furthest. +# 3. ⭐ If a frontier model must judge every paragraph, the tiny model has no +# purpose. Reliability has to be intrinsic, not bought back downstream. +# +# ⚠ THIS IS A PROBE, NOT THE PRODUCT PATH. It trains RAW CONTINUATION text into a +# model whose weights expect <|im_start|> framing, which can degrade the template +# behaviour — that is exactly the risk being measured, and the reason the chained +# test drives it through the chat template afterwards. The product path builds an +# instruction-pair corpus; that is a separate and larger piece of work, and it is +# only worth starting if instruction-following survives this. +# +# Everything else held from the 4B-Base rung so the CARRIER is the only variable: +# unwrapped corpus (sha 77f37057b2782e49), seed 4919, rank 32, lr 1e-4, seq 4096, +# batch 1 x accum 8, 1 epoch, eval+save every 25. +# +# ⚠ Expect the loss minimum EARLY. 4B-Base turned at ~step 75 of 159, and an +# instruct model already fluent in English should saturate at least as fast. +set -euo pipefail +OUT=/home/infra-ops/r49-runs/h02-4b-instruct-1ep +LOG=$OUT/train.log +apps=$(nvidia-smi --query-compute-apps=pid --format=csv,noheader | tr -d '[:space:]') +[ -n "$apps" ] && { echo "REFUSING: GPU not clear" >&2; nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv >&2; exit 1; } +[ -e "$LOG" ] && { echo "REFUSING: $LOG exists" >&2; exit 1; } +mkdir -p "$OUT" +echo "# launched $(date -Is) Qwen3-4B INSTRUCT, 1 epoch, seed 4919, unwrapped corpus" > "$LOG" +setsid nohup /home/infra-ops/ml/.venv/bin/python /home/infra-ops/r49-prep/train_voice_lora.py \ + --corpus /home/infra-ops/r49-corpus-renamed-unwrapped \ + --base /home/infra-ops/carriers/Qwen3-4B-Instruct \ + --seed 4919 --epochs 1 --eval-steps 25 --save-steps 25 \ + --out "$OUT" >> "$LOG" 2>&1 < /dev/null & +echo $! > "$OUT/run.pid" +echo "launched pid $(cat "$OUT/run.pid") -> $LOG"