memory: snapshot — lv-bronte shipped with a failed voice axis, next goal is landing lv-hemingway
In-flight rewritten for the next goal. lv-hemingway is TRAINED and nothing else has been done to it: ship candidate is checkpoint-1750 (ep 1.97, eval 2.2783), the end-of-run adapter is 0.0763 worse, and the v2 gate has not been run. Every instrument it needs was parameterised during the lv-bronte run tonight and the in-flight section names all four with their traps. New detail files: 2026-09-17-lv-bronte-gate.md shipped, voice axis failed, why anyway 2026-09-17-beat-contamination-leak.md the leak the corpus gate cannot see 2026-09-17-esh-fiber-outages.md two Cityside failures, rotation fragility Also commits the memorization_check.py parameterisation, which was left uncommitted: its hardcoded Yarros defaults would have compared a Hemingway arm against the Yarros corpus and reported a meaningless clean zero. Auto-archival: index was 415 lines pre-run, over the 300 cap. Only five entries cleared the 14-day age guard, and three of those carry open deferred pointers (fused MoE park 47, nconnect=8, AI-tab belayed) and are referenced by in-flight. A fourth — every CI job on pfi-fleet runs as root on ana-docker — is a live security property rather than settled history, so it is held back deliberately. One entry archived. The file stays over cap, which is the guard working: an over-cap file that keeps live decisions beats a scannable one that lost them.
This commit is contained in:
@@ -10,12 +10,25 @@ Instrument: longest and mean maximal verbatim n-gram shared with the TRAINING co
|
||||
generation. Controls run every time -- the base-unadapted arm never saw the corpus so it is
|
||||
the negative control, and a slice of the corpus scored against itself is the positive.
|
||||
"""
|
||||
import json, pathlib, re, sys
|
||||
import argparse, json, pathlib, re, sys
|
||||
from collections import Counter
|
||||
|
||||
EVAL = pathlib.Path("/home/infra-ops/r49-runs/yarros-eval")
|
||||
CORP = pathlib.Path("/home/infra-ops/yarros-corpus-renamed/copies")
|
||||
N = 8
|
||||
# ⚠ These were hardcoded to Yarros. Pointed at a Brontë arm they would have compared
|
||||
# it against the YARROS corpus and reported a clean zero — a negative that means
|
||||
# "different book", not "did not memorise". Defaults are unchanged so every Yarros
|
||||
# number already recorded stays reproducible byte for byte.
|
||||
_ap = argparse.ArgumentParser()
|
||||
_ap.add_argument("--eval-dir", default="/home/infra-ops/r49-runs/yarros-eval")
|
||||
_ap.add_argument("--corpus", default="/home/infra-ops/yarros-corpus-renamed/copies",
|
||||
help="the RENAMED copies the adapter actually trained on")
|
||||
_ap.add_argument("--glob", default="beats5.*.jsonl", help="arm files inside --eval-dir")
|
||||
_ap.add_argument("--strip", default="beats5.", help="prefix trimmed to name the arm")
|
||||
_ap.add_argument("-n", type=int, default=8, help="n-gram length")
|
||||
_a = _ap.parse_args()
|
||||
|
||||
EVAL = pathlib.Path(_a.eval_dir)
|
||||
CORP = pathlib.Path(_a.corpus)
|
||||
N = _a.n
|
||||
|
||||
def norm(t): return re.findall(r"[a-z']+", t.lower())
|
||||
|
||||
@@ -44,8 +57,8 @@ def longest_match(words):
|
||||
|
||||
print(f"{'arm':<22} {'gens':>5} {'hit-rate':>9} {'mean-longest':>13} {'max':>5}")
|
||||
print("-" * 60)
|
||||
for f in sorted(EVAL.glob("beats5.*.jsonl")):
|
||||
arm = f.stem.replace("beats5.", "")
|
||||
for f in sorted(EVAL.glob(_a.glob)):
|
||||
arm = f.stem.replace(_a.strip, "")
|
||||
rows = [json.loads(l) for l in f.read_text(encoding="utf-8").splitlines()]
|
||||
longs = [longest_match(norm(r["raw"])) for r in rows]
|
||||
hits = sum(1 for x in longs if x >= N)
|
||||
|
||||
Reference in New Issue
Block a user