vh
935b35ac2e
Operator: "start the 1.7b training." The 0.6B adapter learned the Gutenberg transcription's ~70-character line breaks along with the prose -- its output wrapped at a mid-length-line ratio of 0.85 against the base model's 0.00. That is typography rather than style, and every further rung would have inherited it, so the corpus is reflowed before rung 2 rather than after the sweep. The reflow joins 57,430 of 85,380 paragraph blocks and keeps 27,950. Verse is the hazard a blind join would destroy, so the decision is per block by median line length: blocks whose lines cluster near the wrap width are flowed prose, blocks of consistently short lines keep their breaks. Every kept multi-line block in the sample was genuinely verse with its lineation intact. No line ended in a lone hyphen, so the space-join could not split a word across lines. The acceptance check is content identity -- " ".join(text.split()) byte-identical before and after -- and it passed on all 852 records, proving only whitespace changed. Concrete cost of the old defect: 5.7% of the training budget was newline tokens. The same words pack to 5,210,112 tokens unwrapped against 5,525,504 wrapped. The 1.7B run is live at 159 steps and roughly 18.7 s/it. Everything but the carrier and the corpus is held from the 0.6B run: seed 4919, rank 32, lr 1e-4, seq 4096, batch 1 by accum 8, one epoch, eval and save every 25 steps so the minimum is located rather than assumed. That corpus change is a second variable and it is named as one. A 0.6B-vs-1.7B comparison is descriptive, not attributable, until the chained 0.6B rerun on the same unwrapped corpus lands behind it -- gated on the 1.7B actually producing an adapter, because a chain that fires on failure turns one lost run into two. "Did sense come back at 1.7B" is a within-arm reading and survives the confound; any between-rung delta does not. The original wrapped corpus is untouched, so the 0.6B run's pinned corpus sha 3959036cf851bf62 stays reproducible.
121 lines
6.4 KiB
Python
121 lines
6.4 KiB
Python
"""Render the base-vs-tuned voice A/B into a booth page.
|
|
|
|
Layout is the argument. A flat gallery would let you read one arm at a time, which
|
|
is exactly how you talk yourself into seeing a difference. So every prompt is one
|
|
row, the arms are side by side, and BOTH seeds of each arm sit in the same cell --
|
|
so the within-arm variation is visible in the same glance as the between-arm
|
|
variation. If the two base samples differ from each other as much as base differs
|
|
from tuned, there is nothing here, and the layout should make that obvious rather
|
|
than hide it.
|
|
|
|
Prompts are ordered by tier, hardest first: modern/mundane, then period-neutral,
|
|
then Victorian-adjacent. The modern tier is the one that matters -- Brontë showing
|
|
up there is the adapter's doing, whereas Brontë showing up in the period tier could
|
|
just be the prompt.
|
|
"""
|
|
import html
|
|
import json
|
|
import sys
|
|
from collections import defaultdict
|
|
from pathlib import Path
|
|
|
|
base_f, tuned_f, out_dir = sys.argv[1], sys.argv[2], Path(sys.argv[3])
|
|
out_dir.mkdir(parents=True, exist_ok=True)
|
|
|
|
def load(p):
|
|
d = defaultdict(dict)
|
|
for line in Path(p).read_text(encoding="utf-8").splitlines():
|
|
if not line.strip():
|
|
continue
|
|
r = json.loads(line)
|
|
d[r["id"]][r["seed"]] = r
|
|
return d
|
|
|
|
base, tuned = load(base_f), load(tuned_f)
|
|
ids = [i for i in base if i in tuned]
|
|
TIER = {"modern": ("Tier A — modern / mundane",
|
|
"Nothing here invites Victorian prose. Brontë in this tier is the adapter's doing."),
|
|
"neutral": ("Tier B — period-neutral",
|
|
"Could be any century. A voice shift shows cleanly without the prompt supplying it."),
|
|
"period": ("Tier C — Victorian-adjacent, plainly worded",
|
|
"The setting leans period but the diction does not. Easiest tier; weakest evidence.")}
|
|
order = {"modern": 0, "neutral": 1, "period": 2}
|
|
ids.sort(key=lambda i: (order.get(base[i][list(base[i])[0]]["tier"], 9), i))
|
|
|
|
def cell(rec_by_seed):
|
|
parts = []
|
|
for seed in sorted(rec_by_seed):
|
|
t = (rec_by_seed[seed]["continuation"] or "").strip()
|
|
parts.append(f'<div class="s"><span class="seed">seed {seed}</span>'
|
|
f'<p>{html.escape(t) or "<em>(empty)</em>"}</p></div>')
|
|
return "".join(parts)
|
|
|
|
rows, seen = [], set()
|
|
for i in ids:
|
|
any_rec = base[i][list(base[i])[0]]
|
|
tier = any_rec["tier"]
|
|
if tier not in seen:
|
|
seen.add(tier)
|
|
title, sub = TIER.get(tier, (tier, ""))
|
|
rows.append(f'<h2>{html.escape(title)}</h2><p class="tsub">{html.escape(sub)}</p>')
|
|
rows.append(f"""
|
|
<section class="row">
|
|
<div class="prompt"><span class="pid">{html.escape(i)}</span>{html.escape(any_rec["prompt"])}</div>
|
|
<div class="arms">
|
|
<div class="arm"><h3>Base <small>Qwen3-0.6B-Base, no adapter</small></h3>{cell(base[i])}</div>
|
|
<div class="arm tuned"><h3>Tuned <small>+ H02 LoRA, 1 epoch, seed 4919</small></h3>{cell(tuned[i])}</div>
|
|
</div>
|
|
</section>""")
|
|
|
|
page = f"""<!doctype html><meta charset="utf-8"><title>BabyBronte — voice A/B</title>
|
|
<style>
|
|
:root{{--bg:#faf8f5;--fg:#1c1a17;--mut:#6b6560;--line:#e0dad2;--acc:#7a3b2e;--tint:#fdfbf7}}
|
|
*{{box-sizing:border-box}}
|
|
body{{margin:0;background:var(--bg);color:var(--fg);font:16px/1.6 Georgia,"Iowan Old Style",serif;padding:2.5rem 1.5rem 5rem}}
|
|
.wrap{{max-width:1180px;margin:0 auto}}
|
|
h1{{font-size:1.9rem;margin:0 0 .3rem}}
|
|
.lede{{color:var(--mut);max-width:70ch;margin:0 0 .9rem}}
|
|
.warn{{border-left:3px solid var(--acc);background:#fff;padding:.8rem 1rem;margin:1.2rem 0 2rem;max-width:80ch;font-size:.93rem}}
|
|
h2{{font-size:1.15rem;margin:2.8rem 0 .2rem;padding-top:1rem;border-top:1px solid var(--line)}}
|
|
.tsub{{color:var(--mut);font-size:.9rem;margin:0 0 1.2rem;font-style:italic}}
|
|
.row{{margin:0 0 2.2rem}}
|
|
.prompt{{background:#fff;border:1px solid var(--line);border-left:3px solid var(--acc);padding:.7rem .9rem;font-size:1.02rem;margin-bottom:.7rem}}
|
|
.pid{{display:inline-block;font:600 .72rem/1 ui-monospace,monospace;color:var(--mut);background:var(--bg);border:1px solid var(--line);padding:.22rem .4rem;margin-right:.6rem;vertical-align:1px}}
|
|
.arms{{display:grid;grid-template-columns:1fr 1fr;gap:1rem}}
|
|
@media(max-width:820px){{.arms{{grid-template-columns:1fr}}}}
|
|
.arm{{background:#fff;border:1px solid var(--line);padding:.9rem 1rem}}
|
|
.arm.tuned{{background:var(--tint);border-color:#d8ccbe}}
|
|
.arm h3{{margin:0 0 .6rem;font-size:.9rem;letter-spacing:.04em;text-transform:uppercase;color:var(--acc)}}
|
|
.arm h3 small{{display:block;text-transform:none;letter-spacing:0;color:var(--mut);font-weight:400;font-size:.82rem;margin-top:.15rem}}
|
|
.s{{border-top:1px dotted var(--line);padding-top:.6rem;margin-top:.6rem}}
|
|
.arm .s:first-of-type{{border-top:0;padding-top:0;margin-top:0}}
|
|
.seed{{display:block;font:600 .7rem/1 ui-monospace,monospace;color:var(--mut);margin-bottom:.25rem}}
|
|
.s p{{margin:0;white-space:pre-wrap;font-size:.95rem}}
|
|
footer{{margin-top:3rem;padding-top:1rem;border-top:1px solid var(--line);color:var(--mut);font-size:.85rem;max-width:80ch}}
|
|
</style>
|
|
<div class="wrap">
|
|
<h1>BabyBronte — did the voice move?</h1>
|
|
<p class="lede">Same prompts, same sampler, same box, same seeds. The only difference between the
|
|
columns is the H02 LoRA adapter (Charlotte Brontë, 680k words, 1 epoch, seed 4919).</p>
|
|
|
|
<div class="warn"><strong>Read this as an eyeball test, not a result.</strong> Two samples per arm
|
|
per prompt is enough to see whether the gap between the columns is bigger than the gap between the
|
|
two seeds <em>inside</em> a column — and not enough for anything else. No scoring, no statistics.
|
|
The frozen adjudication rule and the Burrows's-Delta instrument are untouched by this page and
|
|
nothing here feeds them.<br><br>
|
|
Both arms are <strong>Qwen3-0.6B-Base doing continuation</strong>, not instruction-following. The
|
|
adapter was trained as pure continuation, so each prompt is an opening line the model carries on
|
|
from — asking a base model to "rewrite this in Brontë's voice" would test instruction-following
|
|
instead of voice.</div>
|
|
|
|
{''.join(rows)}
|
|
|
|
<footer>Generated on pfi-gx10 (GB10), bf16, sdpa. Sampler pinned identical across arms:
|
|
temperature 0.9, top_p 0.95, 400 new tokens, seeds 1234 and 5678. Adapter binding proven at
|
|
generation time (196/196 lora_B tensors non-zero) — a silent no-op looks exactly like a tune that
|
|
changed nothing.</footer>
|
|
</div>"""
|
|
|
|
(out_dir / "index.html").write_text(page, encoding="utf-8")
|
|
print(f"wrote {out_dir/'index.html'} ({len(ids)} prompts x 2 arms x 2 seeds)")
|