Files
esh-pfi-infrastructure/docs/pfi
vh a0f59d2778 docs(training-playbook): 4.6.2 — a null result needs a positive control
From the run-2 gate. A memorisation probe reporting 0.00% across all 72 items is
the correct output for a model that has not seen the corpus, and is also the
exact output of a probe that is not firing. Nothing in the number distinguishes
them. brokkr-smithy-dev drove the overlap function with known-answer inputs
(identical 100%, half-verbatim 65.38%, unrelated 0%, empty 0%) before trusting
the null, which is what converts a suspicious zero into evidence.

This is 4.5's inert gate wearing a different face: there a check that could not
return 'fail', here a measurement that cannot return non-zero. A clean null is
the most reassuring output an instrument produces and the least
self-evidencing.

Same section records the identical-on-both-arms variant: the diversity battery's
rp family froze zero markers, so its attractor hit rate read 0.0 on base AND
tuned. That reads as a clean result and means the instrument cannot discriminate
on that family. Report as a bounded limitation, never as a delta of zero -- a
check returning the same value for every input is not measuring.

Checklist gains the line.
2026-08-26 02:08:59 -07:00
..