Unwrap the Brontë corpus and launch the 1.7B rung
Operator: "start the 1.7b training." The 0.6B adapter learned the Gutenberg transcription's ~70-character line breaks along with the prose -- its output wrapped at a mid-length-line ratio of 0.85 against the base model's 0.00. That is typography rather than style, and every further rung would have inherited it, so the corpus is reflowed before rung 2 rather than after the sweep. The reflow joins 57,430 of 85,380 paragraph blocks and keeps 27,950. Verse is the hazard a blind join would destroy, so the decision is per block by median line length: blocks whose lines cluster near the wrap width are flowed prose, blocks of consistently short lines keep their breaks. Every kept multi-line block in the sample was genuinely verse with its lineation intact. No line ended in a lone hyphen, so the space-join could not split a word across lines. The acceptance check is content identity -- " ".join(text.split()) byte-identical before and after -- and it passed on all 852 records, proving only whitespace changed. Concrete cost of the old defect: 5.7% of the training budget was newline tokens. The same words pack to 5,210,112 tokens unwrapped against 5,525,504 wrapped. The 1.7B run is live at 159 steps and roughly 18.7 s/it. Everything but the carrier and the corpus is held from the 0.6B run: seed 4919, rank 32, lr 1e-4, seq 4096, batch 1 by accum 8, one epoch, eval and save every 25 steps so the minimum is located rather than assumed. That corpus change is a second variable and it is named as one. A 0.6B-vs-1.7B comparison is descriptive, not attributable, until the chained 0.6B rerun on the same unwrapped corpus lands behind it -- gated on the 1.7B actually producing an adapter, because a chain that fires on failure turns one lost run into two. "Did sense come back at 1.7B" is a within-arm reading and survives the confound; any between-rung delta does not. The original wrapped corpus is untouched, so the 0.6B run's pinned corpus sha 3959036cf851bf62 stays reproducible.
This commit is contained in:
Executable
+44
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env bash
|
||||
# R49 H02 — rung 2 of the carrier sweep: Qwen3-1.7B-Base, 1 epoch.
|
||||
#
|
||||
# WHY THIS RUN. The 0.6B rung answered the narrow question and the operator's read
|
||||
# was the finding: "it's all nonsense, but it sounds like Brontë's nonsense."
|
||||
# Voice transferred (curly quotes 1/18 -> 18/18, worksheet collapse 3/18 -> 0/18)
|
||||
# while coherence did not. That separation is the premise the whole lightweight
|
||||
# author-voice regime rests on, so the live question is which carrier size brings
|
||||
# sense back while the voice stays. 1.7B is the next rung; 4B is after it.
|
||||
#
|
||||
# ⚠ THE CORPUS CHANGED, DELIBERATELY, AND IT IS A SECOND VARIABLE.
|
||||
# The 0.6B adapter learned the Gutenberg transcription's ~70-char line breaks along
|
||||
# with the prose -- its output wrapped at a mid-length-line ratio of 0.85 against
|
||||
# the base model's 0.00. That is typography, not style, and every further rung
|
||||
# would inherit it. So this trains on r49-corpus-renamed-unwrapped: same words,
|
||||
# reflowed into paragraphs, verified whitespace-only on all 852 records, with verse
|
||||
# blocks detected by median line length and their lineation preserved (0 lines
|
||||
# ended in a lone hyphen, so the space-join could not split a word).
|
||||
# CONSEQUENCE: a 0.6B-vs-1.7B comparison is now DESCRIPTIVE, not attributable --
|
||||
# carrier size and corpus typography both moved. "Did sense come back at 1.7B" is a
|
||||
# within-arm reading and survives that; any between-rung delta does not. The 0.6B
|
||||
# rerun on this same corpus is chained after this run so the clean single-variable
|
||||
# comparison exists too.
|
||||
#
|
||||
# Everything else is held from the 0.6B run: seed 4919, rank 32, lr 1e-4, seq 4096,
|
||||
# batch 1 x accum 8, 1 epoch, eval+save every 25 steps so the minimum is LOCATED
|
||||
# rather than assumed (the 3-epoch run overfit with save_strategy="no" and left
|
||||
# nothing to fall back to).
|
||||
set -euo pipefail
|
||||
OUT=/home/infra-ops/r49-runs/h02-1p7b-1ep
|
||||
LOG=$OUT/train.log
|
||||
apps=$(nvidia-smi --query-compute-apps=pid --format=csv,noheader | tr -d '[:space:]')
|
||||
[ -n "$apps" ] && { echo "REFUSING: GPU not clear" >&2; nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv >&2; exit 1; }
|
||||
[ -f "$OUT/run.pid" ] && kill -0 "$(cat "$OUT/run.pid")" 2>/dev/null && { echo "REFUSING: live pid" >&2; exit 1; }
|
||||
[ -e "$LOG" ] && { echo "REFUSING: $LOG exists" >&2; exit 1; }
|
||||
mkdir -p "$OUT"
|
||||
echo "# launched $(date -Is) Qwen3-1.7B-Base, 1 epoch, seed 4919, UNWRAPPED corpus, eval+save every 25" > "$LOG"
|
||||
setsid nohup /home/infra-ops/ml/.venv/bin/python /home/infra-ops/r49-prep/train_voice_lora.py \
|
||||
--corpus /home/infra-ops/r49-corpus-renamed-unwrapped \
|
||||
--base /home/infra-ops/carriers/Qwen3-1.7B-Base \
|
||||
--seed 4919 --epochs 1 --eval-steps 25 --save-steps 25 \
|
||||
--out "$OUT" >> "$LOG" 2>&1 < /dev/null &
|
||||
echo $! > "$OUT/run.pid"
|
||||
echo "launched pid $(cat "$OUT/run.pid") -> $LOG"
|
||||
Reference in New Issue
Block a user