vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(r49): the quote-mark counter was counting apostrophes, and I saw it fire before I saw the bug

Browse Source 
voice_distance.py's quote class shipped this morning as "'‘’“”«»‹›‚„` -- with the
apostrophe characters in it. On a corpus whose defining tic is dont/aint/wont and
whose possessives are everywhere, that made it an apostrophe counter wearing a
quote-mark label.

                          as implemented    TRUE quotes    all apostrophes
  held-out McCarthy ref            121.1            0.0              121.1
  base-unadapted                   224.7           19.9              204.8
  held-out Hemingway ref          1112.6          694.7              351.7

The corrected column is the one the pre-registration names: 0.0 for McCarthy,
which is exactly what build_corpus_mccarthy.py ASSERTS, and 694.7 for Hemingway,
the documented ~838 scale the 100-per-10k trigger line was anchored to. The
as-implemented column matched neither.

ORDER OF EVENTS, because it is the material fact: the base arm finished first, so
the trigger became evaluable while the adapted arms were still generating. I
evaluated it, saw it FIRE at 224.7, and only then -- reading the reference row of
my own table against a corpus I knew asserts 0.0 -- found the bug. No delta_cb,
memorisation rate or damage number had been read at any point.

Fixing a detector to measure the quantity the frozen rule names is not moving the
rule, but the fix un-fires the trigger and no reader should have to take my word
about my motives. So GATE-PREREG.md AMENDMENT 2 makes the trigger MOOT instead of
adjudicating it: the normalised secondary read is load-bearing UNCONDITIONALLY for
this gate, whichever reading you accept, both columns reported. The fix therefore
has no effect on the verdict.

There is a better reason than the bug anyway: base's true quote density is 19.9
against the reference's 0.0, so it did not fully comply. A small residual cheap win
IS available to the adapter, and the normalised read is what prices it. A threshold
is a blunt instrument for a residual that size.

Apostrophes now get their own column and are never folded into quotes again.
Default path stays byte-identical to the shipped lv-hemingway artifact.

The durable lesson is the one this line keeps relearning in new places: I controlled
strip_punct (2500 -> 0) and the byte-identity of the default path, but never asked
the quote counter for a value whose answer I already knew. The corpus asserts 0.0.
That check cost one line and was available before the gate ever launched.
This commit is contained in:
Vuong Hoang
2026-09-21 15:19:33 -07:00
parent a601267fa5
commit 0d80e493a8
2 changed files with 82 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/mccarthy-corpus/GATE-PREREG.md
+66
View File
@@ -286,3 +286,69 @@ correct one**, in the direction that makes passing harder, on an argument publis
four days before this gate existed and independent of any McCarthy number — none of
which had been read when this was written. The honest alternative was to run an axis
whose control I already knew to be unearned.
---
## AMENDMENT 2 — 2026-09-21, the confound trigger's own instrument was broken
**Read the order of events before the numbers, because the order is the point.** The base
arm finished first, so §5c's trigger became evaluable while the two adapted arms were still
generating. I evaluated it, **saw it FIRE**, and only then — reading the reference row of my
own table — found that the detector was measuring the wrong thing. No delta_cb, memorisation
rate or damage number had been read at any point.
### The bug
`voice_distance.py`'s quote-mark class shipped as ``"'‘’“”«»‹›‚„` `` — it included the
apostrophe characters. So it was an **apostrophe counter wearing a quote-mark label**, on a
corpus whose defining tic is `dont`/`aint`/`wont` and whose possessives are everywhere.
```
as implemented TRUE quote marks all apostrophes
held-out McCarthy ref 121.1 0.0 121.1
base-unadapted 224.7 19.9 204.8
held-out Hemingway ref 1112.6 694.7 351.7
```
The corrected column is the one the pre-registration names: **0.0 for this corpus**, which is
exactly what `build_corpus_mccarthy.py` asserts, and **694.7 for Hemingway's val split**,
which is the documented ~838 scale the 100-per-10k line was anchored to. The as-implemented
column matched neither, so it was never measuring the quantity §5c specified.
### The two readings, and why I am not choosing between them
```
as implemented (apostrophe-inclusive) base 224.7 > 100 -> TRIGGERED
as specified (quote marks only) base 19.9 < 100 -> does NOT trigger
```
Fixing a detector so it measures the quantity the frozen rule names is not moving the rule.
**But I saw the firing value first and the fix un-fires it, which is precisely the shape of
threshold-shopping**, and no reader should have to take my word about my own motives.
So the trigger is made moot instead of adjudicated:
> **For this gate the NORMALISED SECONDARY READ (§5b) is load-bearing UNCONDITIONALLY,
> whichever reading of §5c you accept.** Both quote-mark columns are reported. The bug fix
> therefore has no effect on the verdict, and the conservative branch is taken by default.
There is an argument for that independent of the bug, and it is the better reason: **base's
true quote density is 19.9 against the reference's 0.0, so it did not fully comply.** A small
residual cheap win is genuinely available to the adapter, and the normalised read is what
prices it. A trigger threshold is a blunt instrument for a residual this size; running the
conservative read always is simply better than deciding when to.
### What changed in the code
`_QUOTE_RE` is quote marks only. Apostrophes get their own column and are never folded back
in. `--punct-report` now prints quote-marks, all-apostrophes, contraction-apostrophes and
dashes, so a future reader can see the thing that fooled this one.
### The durable lesson
This is the same failure the r49 line keeps finding, in a new place: **a detector validated
only against the present case cannot tell a wrong measurement from a right one.** I controlled
`strip_punct` (2500 → 0, it works) and the byte-identity of the default path (it matches), but
I never asked the quote counter for a value I already knew the answer to. The corpus asserts
0.0. Reading 121.1 against a known 0.0 is what exposed it — and that check cost one line and
was available before the gate ever launched.
scripts/r49-corpus/voice_distance.py
+16 -4
View File
@@ -56,9 +56,19 @@ from pathlib import Path
# and Hemingway's measures 838; 100 is the order-of-magnitude line between them.
PUNCT_CONFOUND_PER_10K = 100.0
QUOTE_CHARS = "\"'‘’“”«»‹›‚„`"
# ⚠⚠ QUOTE MARKS ONLY -- NO APOSTROPHE CHARACTERS. This class shipped 2026-09-21 with
# `'` and `’` in it, which made it an APOSTROPHE counter wearing a quote-mark label. On
# lv-mccarthy that reported the held-out reference at 121.1 "quote marks" per 10k for a
# corpus whose builder ASSERTS 0.0, and it fired the confound trigger on a base arm whose
# true quote density is 19.9. The pre-registered trigger names quote marks and anchors its
# line to this corpus's 0.0 against Hemingway's 838 -- both quotation-mark counts -- so an
# apostrophe-inclusive class does not measure the quantity the rule names.
# Verified after the fix: McCarthy held-out ref 0.0 (matches the builder's assertion),
# Hemingway held-out ref 694.7 (the documented ~838 scale). See GATE-PREREG.md AMENDMENT 2.
_QUOTE_RE = re.compile('["“”«»‹›„]')
_PUNCT_RE = re.compile(r"[^\w\s]|_", re.UNICODE)
_QUOTE_RE = re.compile("[" + re.escape(QUOTE_CHARS) + "]")
# apostrophes are counted SEPARATELY and never folded into the quote column again.
_APOS_RE = re.compile(r"['’‘`]")
# a contraction apostrophe is one sitting BETWEEN letters -- `dont` vs `don't` is the tic
# the register names, and a possessive or a quote mark is not the same measurement.
_CONTRACTION_APOS_RE = re.compile(r"(?<=[A-Za-z])['’](?=[A-Za-z])")
@@ -113,13 +123,15 @@ def punct_report(ref_text: str, arms: list[tuple[str, list[dict]]]) -> None:
print("\n PUNCTUATION DENSITY per 10k words -- the confound check, not an axis")
print(" (the eval harness drives EVERY arm with the same register prompt, tics included;")
print(" a compliant base control earns the adapter no delta_cb for them)")
print(f" {'arm':22s} {'quote-marks':>12s} {'contraction-apos':>18s} {'dashes':>9s}")
print(f" {'arm':22s} {'quote-marks':>12s} {'all-apos':>10s} "
f"{'contraction-apos':>18s} {'dashes':>9s}")
rows = [("held-out reference", ref_text)]
rows += [(a, "\n".join(r["continuation"] for r in recs)) for a, recs in arms]
base_q = None
for name, txt in rows:
q = density(txt, _QUOTE_RE)
print(f" {name:22s} {q:12.1f} {density(txt, _CONTRACTION_APOS_RE):18.1f} "
print(f" {name:22s} {q:12.1f} {density(txt, _APOS_RE):10.1f} "
f"{density(txt, _CONTRACTION_APOS_RE):18.1f} "
f"{density(txt, _DASH_RE):9.1f}")
if "unadapted" in name:
base_q = q