vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(coldfusion-abliteration): Robinson's real 416-prompt corpus, batched capture, two new gates

Browse Source 
The 8/8 calibration set gave |cos| agreement 0.594 against the recipe's 0.9925.
This wires in the corpus the recipe actually used and makes a capture at that
scale affordable.

Corpus (calibration.py, new). The recipe's "held-out train/test split of 416/104
with overlap 0" names mlabonne/harmful_behaviors exactly — 416 train / 104 test,
AdvBench-derived — and it plus harmless_alpaca were already staged in ana-ml2's
HF dataset cache. Read via pyarrow, no datasets dependency, no hub access.
Harmful is order-deterministic (no seed), so a re-capture is reproducible from
the flags alone. The 104-prompt test split is reserved as the held-out
generalization probe and asserted disjoint, so the post-write re-profile cannot
silently become in-distribution. --calib builtin reproduces the legacy run.

Batched capture. 832 prompts x 2 templates = 1664 forwards. Padding is on the
RIGHT: in a causal stack nothing after position t reaches position t, so
trailing pads cannot touch the token read, whereas left padding feeds pads into
the DeltaNet recurrence ahead of the prompt — the path whose torch fallback
already NaN'd once here. Means accumulate in float64; the direction is a
difference of means, which is where cancellation lives on this model.

Gates added, both protecting numbers rather than tensors:
- batch-equivalence: proves padded-batch == single-prompt (rel 1e-3) before
  spending the capture window.
- surgery pre-check: aborts if any of the 131 targets is absent or on the meta
  device. orthogonalize_ edits in place, and an in-place write to an
  accelerate-offloaded tensor is a silent no-op — that ships a half-abliterated
  model past a smoke test.

Fixed a reporting bug: the agreement line printed the global agree.max() beside
the window's argmax layer, so the first capture read as 0.8538 when the real
in-window number was 0.5944. The global peak sits in the early layers where the
dim-3994 massive activation inflates agreement for reasons unrelated to refusal.
Now prints window max, a top-5, and labels the global figure informational.

--max-layer truncates the decoder for capture. Exact, not approximate: a causal
stack's layer-N state cannot depend on layers above N, so any value above the
window top leaves the direction bit-identical while cutting fp32 residency and
forward cost. 46 drops 18 of 64 layers and is what keeps fp32 off CPU offload.
Refused on the write path, where it would emit a truncated checkpoint.

Verified on ana-ml2 without the GPU: dry-run still 1:1 (131 tensors, all
coverage gates), calibration loads 416/416 deterministically with its guards
firing, both --max-layer guards exit as designed. Also confirmed against
chat_template.jinja that enable_thinking=True does resolve reasoning_effort to
xhigh, so the two renderings are the recipe's — template selection was not the
cause of the low agreement.

The re-capture itself is unrun: it needs the fp32 VRAM window and therefore
production seat downtime.
This commit is contained in:
vh
2026-08-20 07:52:37 -07:00
parent 530f1452e8
commit f714f28195
3 changed files with 476 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
services/coldfusion-abliteration/README.md
+118 -14
View File
@@ -43,26 +43,117 @@ Two hard gates from the recipe, both of which halt before any write:
The refusal direction is captured from **two chat-template renderings**
(`enable_thinking=false` and thinking at `xhigh`); the layer is auto-picked by
peak two-template `|cos|` agreement in the recipe's [18,45] window (anchor: 26).
Verified 2026-08-20 against `chat_template.jinja`: `enable_thinking=True` resolves
`reasoning_effort` to `'xhigh'` by default, so these really are the recipe's two
renderings — the low agreement is not a template-selection bug.
Two more gates were added 2026-08-20, both protecting numbers rather than
tensors:
3. **Batch-equivalence gate** — capture batches prompts, so before the real run
it proves a padded batch reproduces one-at-a-time forwards (rel. tolerance
1e-3) and aborts otherwise. Padding is on the **right**, and that is load-
bearing: in a causal stack nothing after position *t* reaches position *t*, so
trailing pads cannot touch the token we read, whereas left padding would feed
pad tokens *into* the DeltaNet recurrence ahead of the prompt — the exact path
whose torch fallback is already known-untrustworthy here.
4. **Surgery pre-check** — on the write path, aborts if any of the 131 target
tensors is absent or on the meta device. `orthogonalize_` edits in place, and
an in-place write to an accelerate-offloaded tensor is a **silent no-op**;
without this gate an under-provisioned run ships a half-abliterated model that
passes a smoke test. Free the VRAM instead of defeating it.
## Calibration corpus
`calibration.py`. The first capture used 8 harmful / 8 harmless and produced
`|cos|` agreement of **0.594** — valid but far off the recipe's 0.9925, and a
difference-in-means is only as clean as the number of prompts in each mean.
The recipe's line about a "held-out train/test split of 416/104 with overlap 0"
turns out to name the corpus exactly: **`mlabonne/harmful_behaviors` is 416 train
/ 104 test** (the AdvBench-derived pair used by the standard abliteration
notebooks), and both it and `mlabonne/harmless_alpaca` were **already staged** in
ana-ml2's HF dataset cache. So `--calib mlabonne` reproduces Robinson's
calibration set rather than approximating it. Read via pyarrow — no `datasets`
dependency, no hub access.
- **harmful** = `harmful_behaviors[train]`, file order, truncated to `n`. No
seed dependence, so a re-capture is bit-reproducible from the flags alone.
- **harmless** = `harmless_alpaca[train]`, seeded sample (pool is 25058).
- **`harmful_behaviors[test]` (104) is reserved, not calibration.** It is the
held-out generalization probe — the set Robinson reported 8% post-abliteration
refusal on, and therefore our one directly comparable number. `load_calibration`
will not draw from it and asserts overlap 0 against it, so a later edit cannot
quietly turn the evaluation in-distribution.
- `--calib builtin` reproduces the legacy 8/8 run exactly.
Note the axis mismatch, and that it is deliberate: this corpus is **operational**
harm (hacking, fraud, weapons) while Cold-Fusion's measured refusal surface is
**creative** (explicit-sexual, graphic-torture). Robinson calibrated on exactly
this set and still drove creative refusal to 8% with self-harm guardrails intact,
which is the single-direction result holding across refusal types. Reproduce
first; a creative-axis supplement is the *second* experiment, not a variable to
change in the same run — and if one is added it must stay disjoint from
`services/refusal-probe/battery*.yaml`, or the post-write re-profile stops being
a held-out measurement.
## Sequence
Run from `/tank/aimodels/coldfusion-abliteration` on ana-ml2 (the deployed copy
of this directory), as `llmuser`, with `pylibs` on `PYTHONPATH`:
```bash
P=/tank/aimodels/coldfusion-abliteration
V=/tank/aimodels/quant-work/.venv/bin/python
M=/tank/aimodels/qwen38-27b-coldfusion-bf16
A=/tank/aimodels/qwen38-27b-coldfusion-abliterated-bf16
RUN="sudo -u llmuser env HF_HUB_OFFLINE=1 PYTHONPATH=$P/pylibs \
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True $V $P/abliterate.py --model $M"
# 1. DRY RUN FIRST — verify the tensor map + both gates on the static surface,
# no forward, no write. Do not skip: this is what confirms the recipe maps
# onto THIS checkpoint's names before anything irreversible.
sudo -u llmuser $V services/coldfusion-abliteration/abliterate.py --model $M --dry-run
# 1. DRY RUN FIRST — verify the tensor map + coverage gate on the static
# surface, no forward, no write. Safe with the seats up. Do not skip: this
# confirms the recipe maps onto THIS checkpoint's names.
$RUN --dry-run
# 2. Capture the direction + screen the sink (loads the model; no write yet).
sudo -u llmuser $V services/coldfusion-abliteration/abliterate.py --model $M --capture
# --- everything below needs the fp32 VRAM window; stop the seats first ---
sudo docker stop vllm-gen vllm-meromero-rp vllm-fablefusion-probe
cp $M/refusal-direction.pt $M/refusal-direction.pt.bak # capture overwrites it
# 3. Abliterate (writes the new bf16). Only after 1 and 2 pass.
sudo -u llmuser $V services/coldfusion-abliteration/abliterate.py --model $M --out $A
# 2. CONTROL RUN — the legacy 8/8 set through the new batched path. It must
# reproduce the 2026-08-20 result (layer 22, |cos| 0.594, sink 0.001%). This
# is the regression test: batching, layer truncation and the refactor all
# validate against a known number for ~1 minute of forwards, before the
# expensive run. A mismatch here bisects cleanly — the batch-equivalence gate
# has already cleared batching, so truncation is the remaining suspect.
$RUN --capture --calib builtin --max-layer 46 --batch-size 8
# 3. THE REAL CAPTURE — Robinson's 416-prompt corpus.
$RUN --capture --calib mlabonne --max-layer 46 --batch-size 8
# Expect |cos| agreement in the window to rise well above 0.594. If it does
# not, set size was NOT the cause and the write stays gated.
# 4. Restore the seats — meromero FIRST, gen LAST (gen grabs a fraction of FREE
# VRAM at startup and will starve meromero if it goes first).
sudo docker start vllm-meromero-rp && sleep 60 && sudo docker start vllm-gen vllm-fablefusion-probe
# 5. Abliterate (writes the new bf16). Only after 1-3 pass, and only on the
# operator's go — this is the destructive step. Needs the VRAM window again
# (bf16, 55.6 GB, must be fully resident — the surgery pre-check enforces it).
# NOTE: no --max-layer here; the guard refuses it.
$RUN --out $A --direction $M/refusal-direction.pt
```
### Flags added 2026-08-20
| flag | default | why |
|---|---|---|
| `--calib {mlabonne,builtin}` | `mlabonne` | corpus selection; `builtin` = legacy 8/8 |
| `--calib-n-harmful` | 416 | the full train split, as the recipe used |
| `--calib-n-harmless` | 416 | matched n from alpaca |
| `--calib-seed` | 0 | harmless sample only; harmful is order-deterministic |
| `--batch-size` | 8 | 832 prompts x 2 templates = 1664 forwards; batching is what makes that affordable |
| `--max-layer` | off | capture-only. Truncates the decoder. **Exact, not an approximation** — a causal stack's layer-N state cannot depend on layers above N, so any value above the window top (45) leaves the chosen direction bit-identical while cutting fp32 residency and forward cost by the dropped fraction. 46 drops 18 of 64 layers (~28%) and is what keeps fp32 off CPU offload. Refused on the write path, where it would emit a truncated checkpoint. |
## Verify after (do not trust the write blind)
1. **Vision byte-identical** — diff `visual.*` tensors source vs output (recipe
@@ -117,11 +208,24 @@ refusal direction is **finite, unit-normed, layer 22**, sink energy **0.0008%**
in dim 3994 (recipe L26 ref 0.06%, threshold 1%) — clean, not sink-dominated.
Saved to `qwen38-27b-coldfusion-bf16/refusal-direction.pt`.
⚠️ **Quality caveat:** two-template `|cos|` agreement at layer 22 is **0.59**,
notably below Robinson's 0.99 — almost certainly the small calibration set (8
harmful / 8 harmless here vs Robinson's 416 / 104). The direction is valid and
sink-clean, but **before the `--out` write, expand the harmful/harmless sets**
(HARMFUL/HARMLESS in `abliterate.py`) for a cleaner, higher-agreement direction.
⚠️ **Quality caveat:** two-template `|cos|` agreement at layer 22 is **0.594**,
notably below Robinson's 0.9925 — the 8/8 calibration set is the suspect.
> ⚠️ The first capture's log reported this as `|cos|=0.8538`. That was a
> reporting bug, fixed 2026-08-20: the line printed the **global** `agree.max()`
> next to the **window's** argmax layer. The global peak sits in the early layers
> where the dim-3994 massive activation dominates both templates and inflates
> agreement for reasons unrelated to refusal. `0.5944` was always the real
> in-window number. The report now prints the window max, a top-5, and labels the
> global figure as informational.
**Where it stands 2026-08-20 (second session):** harness upgraded for the
re-capture — Robinson's actual 416-prompt corpus wired in (already on the box),
batched capture with an equivalence gate, optional exact layer truncation, the
agreement report fixed, and a surgery pre-check added for the write. Dry-run
re-verified 1:1 (131 tensors) and the calibration path tested end-to-end on the
box. **What has not run is anything needing the GPU** — the re-capture needs the
fp32 VRAM window, which costs production seat downtime.
**The destructive `--out` write has NOT been executed** — it gates on the
operator's go (and, recommended, a larger calibration set first).
operator's go, and on the re-capture showing a healthy agreement first.