§3.13, plus the probe that produced it. Two lessons, both about measuring the
wrong thing confidently.
First: a tune applied AFTER an abliteration can walk it back, and a
reasoning/craft/memorisation gate cannot see that. brokkr-smithy-dev's
preregistered gate measured none of it — a tune that gains 41 items of
contradiction detection and quietly restores refusals passes every check. The
compliance axis has to be added explicitly.
Second, and this is the trap: measure the axis the abliteration was actually
FOR. Ours was run so the model engages explicit fiction. The probe reached for
mlabonne/harmful_behaviors — weapons, malware, fraud — because it was cached and
carried a recorded baseline. Different refusal surface entirely, and a model
moves on them independently. 29/100 general-harm refusals on a tune whose prose
the operator was praising at the time is not obviously a defect and may be
desirable: general-harm refusals returning while domain compliance holds is
close to the ideal shape for an internal creative seat. The measurement was
real; its relevance was assumed.
Also recorded, because both were nearly missed:
- Read the interesting cell. In 29 hard / 0 deflect / 71 comply, the
load-bearing number is 71. Stock refused 100/100; near that would mean the
abliteration was undone. 71 complying means partially walked back on one
axis — a different finding, and only one of the two threatens the seat.
- A baseline from a different harness is not a baseline. The recorded 3/100
came from the abliteration tool's scorer, which reads first-token probability
distributions; a probe that generates and regexes is a different instrument.
Run your own against both arms on the same seat or report the number alone.
- A refusal regex undercounts, so classify hard/deflect/comply — and the free
discriminator: if both arms return zero deflections the model is binary; if
only one does, the regex is fine. An artifact does not care which arm it runs
against.
vh
64bf9d313f