Vuong Hoang
9c8a4e959c
Freezes the lv-mccarthy gate design while no McCarthy generation exists, per the
lv-hemingway precedent: a threshold chosen after seeing the numbers is not a
threshold. Three arms (base-unadapted, ckpt900, ckpt450), 60 beats, 4 seeds.
Settles the voice-axis question D1 deferred. McCarthy's corpus measures 0.0 quote
marks per 10k against Hemingway's 838, so "emit no quotation marks" is the cheapest
way to move a char-bigram Delta without learning a sentence. Three parts:
PRIMARY unchanged -- the mccarthy register names the punctuation and
--system-from drives the base control with the same prompt, so the
cheap win is handed to both sides.
SECONDARY voice_distance.py --secondary-normalised re-runs the whole analysis
with punctuation stripped from the reference and every arm. A
conservative lower bound; reported, never the verdict.
TRIGGER voice_distance.py --punct-report evaluates a pre-registered line --
base-unadapted quote density above 100 per 10k means the control did
not take the win it was handed, and the normalised read is promoted
to load-bearing. 100 is the order-of-magnitude line between this
corpus's 0.0 and Hemingway's 838, chosen now rather than after the
table prints.
ckpt450 is NOT tied with the minimum (+4.9x the 0.00393 median neighbour jitter)
and the pre-registration says so. It is generated to test a stated prior and to
price the memorisation headroom an earlier checkpoint buys on an in-copyright
author, with a decision rule that says exactly what result would let it displace
ckpt900.
Instrument controls, run before this landed:
- the voice_distance.py refactor reproduces the shipped lv-hemingway
voice_distance.txt BYTE FOR BYTE on the default path
- strip_punct drops a quote-bearing sample from 2500.0 to 0.0 marks per 10k
- the secondary read still resolves a gap on the Hemingway arms (+0.410 at
7.3x floor), so a null on McCarthy would be a finding, not a blind detector
Pre-flight re-run at gate time rather than quoted from 2026-09-17:
- leak gate: 0 of 75 renameable, 0 of 37 sub-threshold, 0 separator-split,
four controls green
- beat-contamination audit: 0 of 3942 beats AND 0 of 3942 responses, against
Hemingway's 70 of 7094 -- build_sft_pairs.py --source-entities earned its
mandatory flag
Also records three provenance defects found on first read of the run and their
disposition: the hardcoded "r49-babyyarros-pairs-pilot" run label (cosmetic, same
literal on all three runs), the empty harness_commit (all three runs), and a
pairs_sha256_16 that is not a sha256sum of the file (consistent across runs, so a
cache key rather than a fault). The run is bound to McCarthy's pairs by record
count, not by the label.
50 lines
2.0 KiB
Python
50 lines
2.0 KiB
Python
"""Adapt the lv-mccarthy arms for voice_distance.py, and refuse if its reference is empty.
|
|
|
|
Same contract as the lv-hemingway sibling — the arm NAMES are the only real difference,
|
|
and the control must carry the substring `unadapted` or voice_distance.py cannot identify
|
|
it and prints an empty vs-control table instead of an error.
|
|
|
|
voice_distance.py expects:
|
|
- files matching voice.<arm>.jsonl in the eval dir
|
|
- a `continuation` field per record (gen_beats_chat writes `raw`)
|
|
- a `seed` field (present)
|
|
- the control arm's NAME to contain the substring "unadapted"
|
|
- a reference built from corpus records whose split == "val"
|
|
"""
|
|
import json, sys
|
|
from collections import Counter
|
|
from pathlib import Path
|
|
|
|
CORP = Path("/home/infra-ops/lv-mccarthy/corpus-renamed/copies")
|
|
EVAL = Path("/home/infra-ops/r49-runs/mccarthy-eval")
|
|
|
|
c = Counter()
|
|
n = 0
|
|
for p in sorted(CORP.glob("*.jsonl")):
|
|
for line in p.read_text(encoding="utf-8").splitlines():
|
|
if not line.strip():
|
|
continue
|
|
r = json.loads(line)
|
|
n += 1
|
|
c[r.get("split")] += 1
|
|
print(f"corpus records: {n} split values: {dict(c)}")
|
|
if c.get("val", 0) == 0:
|
|
print("== REFUSING: no split=val records; voice_distance would build an EMPTY reference")
|
|
print(" and every delta_cb would be meaningless rather than absent.")
|
|
sys.exit(1)
|
|
|
|
NAMES = {"base": "base-unadapted", "ckpt900": "ckpt900", "ckpt450": "ckpt450"}
|
|
for src_arm, out_arm in NAMES.items():
|
|
src = EVAL / f"beats5.{src_arm}.jsonl"
|
|
if not src.exists():
|
|
print(f"== missing {src}")
|
|
sys.exit(1)
|
|
rows = [json.loads(l) for l in src.read_text(encoding="utf-8").splitlines() if l.strip()]
|
|
out = EVAL / f"voice.{out_arm}.jsonl"
|
|
with out.open("w", encoding="utf-8") as fh:
|
|
for r in rows:
|
|
fh.write(json.dumps({"id": r["id"], "seed": r["seed"],
|
|
"continuation": r["raw"]}, ensure_ascii=False) + "\n")
|
|
print(f" {src.name} -> {out.name} ({len(rows)} records)")
|
|
print("ready")
|