Files
esh-pfi-infrastructure/services/refusal-probe/.gitignore
T
vh b9e68c3fd2 feat(refusal-probe): three-arm A/B result -- Fable-Fusion 711 cuts refusals 92.5% -> 15.8%
brokkr-smithy-dev supplied the framing that reproduces the production
failure: a bare assistant-mode instruction, no character card, no
permission preamble. Added as battery-instruct.yaml, verbatim to his shape.
Refusal is monotonic in framing, and that third arm is the discriminator.

                                    DS v1.0      Fable-Fusion 711
  permission framing (temp 1.0)     0.0%  0/75   0.0%  0/75
  plain character cards (temp 1.0)  1.4%  1/74   0.0%  0/74
  bare instruction (temp 0.7)      92.5% 37/40  15.8%  6/38

Per-axis on arm 3, DS -> FF: incest 100->20, non-con 100->20, bestiality
100->25, necrophilia 100->40, gore 100->0, consensual 80->20, dubcon 80->0,
self-harm 80->0. DS refused 25/25 on the five axes brokkr flagged.

My own arm-3 measurement exceeded brokkr's 62.5% at n=16; his headline
under-read his own finding rather than over-reading it, though he had
already corrected the claim in the other direction.

Known battery bug, left unfixed for comparability: DS's arm-3 control gate
failed at 11% because ictrl-reunion pairs 'explicit / do not fade to black'
with brothers, which DS reasonably read as an incest request. FF did not
take that read. Both models faced an identical battery; ictrl-storm is the
clean control.

MTP loads on this model and survives RP temperatures: vLLM resolved
Qwen3_5MTP, loaded the drafter, shared embedding + lm_head. 47.0%
acceptance (229,169/487,725) over ~163k draft windows at temp 0.7-1.0,
1.41 extra tokens per window, ~80.6 tok/s decode at temp 1.0. This
CORRECTS the caution in the probe stack README that acceptance would fall
at RP temps -- it did not, and 47.0% matches the gen seat's 47.7%.

No decision taken; seat state reported separately.
2026-08-16 09:20:59 -07:00

4 lines
41 B
Plaintext

results/
results-bare/
results-instruct/