feat(coldfusion-abliteration): Robinson-formula harness, gated, staged
Harness to abliterate DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 using the MTP-aware, vision-preserving recipe in docs/pfi/abliteration-recipe-qwen38.md. Motivation is measured, not assumed: the stock model's refusal profile (probed 2026-08-19, hand-verified) is ~33% on creative content, concentrated on explicit-sexual and graphic-torture, with 4/5 hard-harm refused, self-harm guardrails intact, and zero benign over-refusal. So there is a real creative-content refusal surface. The Robinson formula is chosen specifically because it abliterates the MTP head IN-BAND -- which the current gen seat's Heretic pass does not (its MTP head is a byte-identical base graft the Qwen3_5 wrapper never loads). That in-band MTP edit is the additive delta. The script refuses to brick the model. Two hard gates from the recipe halt before any write: the coverage identity o_proj(16)+linear_out(48)==64 (catches a tensor-name mismatch that would ship a half-abliterated model), and the attention-sink screen on dim 3994 (orthogonalizing a direction living there produces a model that loads, runs, and emits garbage). The direction is captured from two chat templates and the layer auto-picked by peak |cos| agreement in [18,45]. Classification is suffix-based and name-agnostic so it survives minor drift; the coverage gate is the backstop. Modes: --dry-run (enumerate + gate, no forward, no write), --capture (direction + sink screen, no write), default (write to --out). The README sequences dry-run -> capture -> write -> verify, and names the post-checks (vision byte-identical, refusal re-profile via services/refusal-probe/, MTP acceptance on the quant, PPL/coherence). bf16 staged to ana-ml2:/tank/aimodels/qwen38-27b-coldfusion-bf16 (pinned 9c44193, provenance recorded). The destructive run is NOT executed here -- dry-run verification and operator go gate it.
This commit is contained in:
@@ -0,0 +1,88 @@
|
||||
# Cold-Fusion abliteration — Robinson formula
|
||||
|
||||
Abliterate `DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1` using the MTP-aware,
|
||||
vision-preserving single-direction recipe documented in
|
||||
[`docs/pfi/abliteration-recipe-qwen38.md`](../../docs/pfi/abliteration-recipe-qwen38.md).
|
||||
|
||||
**Why this model, why this recipe.** Its stock refusal profile (probed
|
||||
2026-08-19, hand-verified) is ~33% on creative content — it still hard-refuses
|
||||
explicit sexual content and graphic torture, and refuses 4/5 hard-harm technical
|
||||
prompts, while keeping self-harm guardrails and over-refusing zero benign
|
||||
prompts. So there is a real creative-content refusal surface to remove. The
|
||||
Robinson formula is chosen specifically because **it abliterates the MTP head
|
||||
in-band** — which the current gen seat's Heretic pass does *not* (per
|
||||
`qwen38-27b-heresy-bf16.PROVENANCE.txt`, the MTP head there is a byte-identical
|
||||
base graft the wrapper never loaded). That is the additive delta this
|
||||
experiment tests.
|
||||
|
||||
## Where it runs
|
||||
|
||||
**ana-ml2** (dual RTX PRO 6000 Blackwell, 96 GB each). A 55.6 GB bf16 loads
|
||||
comfortably; the output feeds the same box's NVFP4 quant pipeline
|
||||
(`services/gen-seat-mixed-quant/`).
|
||||
|
||||
- bf16 source: `/tank/aimodels/qwen38-27b-coldfusion-bf16`
|
||||
(pinned `9c44193f07782c85c0f437a5d8466ba5c95c95fe`)
|
||||
- env: `/tank/aimodels/quant-work/.venv` (torch 2.12.1+cu130, CUDA live)
|
||||
- run **as `llmuser`** (owns `/tank/aimodels`): `sudo -u llmuser <venv>/bin/python …`
|
||||
|
||||
## The gates — this script refuses to brick the model
|
||||
|
||||
Two hard gates from the recipe, both of which halt before any write:
|
||||
|
||||
1. **Coverage gate** — `o_proj(16) + linear_out(48) == 64 == num_hidden_layers`,
|
||||
plus `down_proj==64`, MTP writers `==2`, exactly one `embed_tokens`. Catches a
|
||||
tensor-name mismatch that would otherwise ship a half-abliterated model. **131
|
||||
tensors** edited when it passes; vision (333) never touched.
|
||||
2. **Attention-sink screen** — Qwen3.8-27B's massive-activation dimension is
|
||||
**3994**. Orthogonalizing a direction that lives in dim 3994 produces a model
|
||||
that loads, runs, and emits garbage. The script aborts if the chosen layer's
|
||||
direction carries >1% of its energy in dim 3994 (recipe's layer-26 reference:
|
||||
0.06%).
|
||||
|
||||
The refusal direction is captured from **two chat-template renderings**
|
||||
(`enable_thinking=false` and thinking at `xhigh`); the layer is auto-picked by
|
||||
peak two-template `|cos|` agreement in the recipe's [18,45] window (anchor: 26).
|
||||
|
||||
## Sequence
|
||||
|
||||
```bash
|
||||
V=/tank/aimodels/quant-work/.venv/bin/python
|
||||
M=/tank/aimodels/qwen38-27b-coldfusion-bf16
|
||||
A=/tank/aimodels/qwen38-27b-coldfusion-abliterated-bf16
|
||||
|
||||
# 1. DRY RUN FIRST — verify the tensor map + both gates on the static surface,
|
||||
# no forward, no write. Do not skip: this is what confirms the recipe maps
|
||||
# onto THIS checkpoint's names before anything irreversible.
|
||||
sudo -u llmuser $V services/coldfusion-abliteration/abliterate.py --model $M --dry-run
|
||||
|
||||
# 2. Capture the direction + screen the sink (loads the model; no write yet).
|
||||
sudo -u llmuser $V services/coldfusion-abliteration/abliterate.py --model $M --capture
|
||||
|
||||
# 3. Abliterate (writes the new bf16). Only after 1 and 2 pass.
|
||||
sudo -u llmuser $V services/coldfusion-abliteration/abliterate.py --model $M --out $A
|
||||
```
|
||||
|
||||
## Verify after (do not trust the write blind)
|
||||
|
||||
1. **Vision byte-identical** — diff `visual.*` tensors source vs output (recipe
|
||||
requires max delta 0).
|
||||
2. **Refusal re-profile** — re-run the same battery from the 2026-08-19 probe
|
||||
(reuse `services/refusal-probe/`, the gen-seat harness — NOT the ad-hoc GGUF
|
||||
one) and confirm creative refusals dropped toward the RobinsonLabs 8% floor
|
||||
while self-harm guardrails survive.
|
||||
3. **MTP acceptance** — the whole point of the in-band MTP edit; measure on the
|
||||
quantized build per `services/gen-seat-mixed-quant/RUNBOOK-heresy-swap.md`.
|
||||
Gate ≳40% (`reference_abliteration_mtp_lessons` — gate on acceptance, not KL).
|
||||
4. **PPL / coherence / no catatonia** — DavidAU fine-tunes are idiosyncratic;
|
||||
eyeball the outputs, don't trust the metric alone.
|
||||
|
||||
Then, if it holds, NVFP4-quantize via `services/gen-seat-mixed-quant/` and it
|
||||
becomes a gen-seat candidate — **do not delete the incumbent weights** until it
|
||||
survives real multi-turn use (the 2026-08-14 delete-too-early lesson).
|
||||
|
||||
## Status
|
||||
|
||||
Harness written 2026-08-19; bf16 staging in progress. **The destructive run has
|
||||
NOT been executed** — the dry-run tensor-map verification and the capture/screen
|
||||
are the gates before it, and the go is the operator's.
|
||||
Reference in New Issue
Block a user