fix(r49): the quote-mark counter was counting apostrophes, and I saw it fire before I saw the bug
voice_distance.py's quote class shipped this morning as "'‘’“”«»‹›‚„` -- with the
apostrophe characters in it. On a corpus whose defining tic is dont/aint/wont and
whose possessives are everywhere, that made it an apostrophe counter wearing a
quote-mark label.
as implemented TRUE quotes all apostrophes
held-out McCarthy ref 121.1 0.0 121.1
base-unadapted 224.7 19.9 204.8
held-out Hemingway ref 1112.6 694.7 351.7
The corrected column is the one the pre-registration names: 0.0 for McCarthy,
which is exactly what build_corpus_mccarthy.py ASSERTS, and 694.7 for Hemingway,
the documented ~838 scale the 100-per-10k trigger line was anchored to. The
as-implemented column matched neither.
ORDER OF EVENTS, because it is the material fact: the base arm finished first, so
the trigger became evaluable while the adapted arms were still generating. I
evaluated it, saw it FIRE at 224.7, and only then -- reading the reference row of
my own table against a corpus I knew asserts 0.0 -- found the bug. No delta_cb,
memorisation rate or damage number had been read at any point.
Fixing a detector to measure the quantity the frozen rule names is not moving the
rule, but the fix un-fires the trigger and no reader should have to take my word
about my motives. So GATE-PREREG.md AMENDMENT 2 makes the trigger MOOT instead of
adjudicating it: the normalised secondary read is load-bearing UNCONDITIONALLY for
this gate, whichever reading you accept, both columns reported. The fix therefore
has no effect on the verdict.
There is a better reason than the bug anyway: base's true quote density is 19.9
against the reference's 0.0, so it did not fully comply. A small residual cheap win
IS available to the adapter, and the normalised read is what prices it. A threshold
is a blunt instrument for a residual that size.
Apostrophes now get their own column and are never folded into quotes again.
Default path stays byte-identical to the shipped lv-hemingway artifact.
The durable lesson is the one this line keeps relearning in new places: I controlled
strip_punct (2500 -> 0) and the byte-identity of the default path, but never asked
the quote counter for a value whose answer I already knew. The corpus asserts 0.0.
That check cost one line and was available before the gate ever launched.
This commit is contained in:
@@ -56,9 +56,19 @@ from pathlib import Path
|
||||
# and Hemingway's measures 838; 100 is the order-of-magnitude line between them.
|
||||
PUNCT_CONFOUND_PER_10K = 100.0
|
||||
|
||||
QUOTE_CHARS = "\"'‘’“”«»‹›‚„`"
|
||||
# ⚠⚠ QUOTE MARKS ONLY -- NO APOSTROPHE CHARACTERS. This class shipped 2026-09-21 with
|
||||
# `'` and `’` in it, which made it an APOSTROPHE counter wearing a quote-mark label. On
|
||||
# lv-mccarthy that reported the held-out reference at 121.1 "quote marks" per 10k for a
|
||||
# corpus whose builder ASSERTS 0.0, and it fired the confound trigger on a base arm whose
|
||||
# true quote density is 19.9. The pre-registered trigger names quote marks and anchors its
|
||||
# line to this corpus's 0.0 against Hemingway's 838 -- both quotation-mark counts -- so an
|
||||
# apostrophe-inclusive class does not measure the quantity the rule names.
|
||||
# Verified after the fix: McCarthy held-out ref 0.0 (matches the builder's assertion),
|
||||
# Hemingway held-out ref 694.7 (the documented ~838 scale). See GATE-PREREG.md AMENDMENT 2.
|
||||
_QUOTE_RE = re.compile('["“”«»‹›„]')
|
||||
_PUNCT_RE = re.compile(r"[^\w\s]|_", re.UNICODE)
|
||||
_QUOTE_RE = re.compile("[" + re.escape(QUOTE_CHARS) + "]")
|
||||
# apostrophes are counted SEPARATELY and never folded into the quote column again.
|
||||
_APOS_RE = re.compile(r"['’‘`]")
|
||||
# a contraction apostrophe is one sitting BETWEEN letters -- `dont` vs `don't` is the tic
|
||||
# the register names, and a possessive or a quote mark is not the same measurement.
|
||||
_CONTRACTION_APOS_RE = re.compile(r"(?<=[A-Za-z])['’](?=[A-Za-z])")
|
||||
@@ -113,13 +123,15 @@ def punct_report(ref_text: str, arms: list[tuple[str, list[dict]]]) -> None:
|
||||
print("\n PUNCTUATION DENSITY per 10k words -- the confound check, not an axis")
|
||||
print(" (the eval harness drives EVERY arm with the same register prompt, tics included;")
|
||||
print(" a compliant base control earns the adapter no delta_cb for them)")
|
||||
print(f" {'arm':22s} {'quote-marks':>12s} {'contraction-apos':>18s} {'dashes':>9s}")
|
||||
print(f" {'arm':22s} {'quote-marks':>12s} {'all-apos':>10s} "
|
||||
f"{'contraction-apos':>18s} {'dashes':>9s}")
|
||||
rows = [("held-out reference", ref_text)]
|
||||
rows += [(a, "\n".join(r["continuation"] for r in recs)) for a, recs in arms]
|
||||
base_q = None
|
||||
for name, txt in rows:
|
||||
q = density(txt, _QUOTE_RE)
|
||||
print(f" {name:22s} {q:12.1f} {density(txt, _CONTRACTION_APOS_RE):18.1f} "
|
||||
print(f" {name:22s} {q:12.1f} {density(txt, _APOS_RE):10.1f} "
|
||||
f"{density(txt, _CONTRACTION_APOS_RE):18.1f} "
|
||||
f"{density(txt, _DASH_RE):9.1f}")
|
||||
if "unadapted" in name:
|
||||
base_q = q
|
||||
|
||||
Reference in New Issue
Block a user