esh-pfi-infrastructure/services/coldfusion-abliteration/README.md

Cold-Fusion abliteration — Robinson formula

Abliterate DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 using the MTP-aware, vision-preserving single-direction recipe documented in docs/pfi/abliteration-recipe-qwen38.md.

Why this model, why this recipe. Its stock refusal profile (probed 2026-08-19, hand-verified) is ~33% on creative content — it still hard-refuses explicit sexual content and graphic torture, and refuses 4/5 hard-harm technical prompts, while keeping self-harm guardrails and over-refusing zero benign prompts. So there is a real creative-content refusal surface to remove. The Robinson formula is chosen specifically because it abliterates the MTP head in-band — which the current gen seat's Heretic pass does not (per qwen38-27b-heresy-bf16.PROVENANCE.txt, the MTP head there is a byte-identical base graft the wrapper never loaded). That is the additive delta this experiment tests.

Where it runs

ana-ml2 (dual RTX PRO 6000 Blackwell, 96 GB each). A 55.6 GB bf16 loads comfortably; the output feeds the same box's NVFP4 quant pipeline (services/gen-seat-mixed-quant/).

• bf16 source: /tank/aimodels/qwen38-27b-coldfusion-bf16 (pinned 9c44193f07782c85c0f437a5d8466ba5c95c95fe)
• env: /tank/aimodels/quant-work/.venv (torch 2.12.1+cu130, CUDA live)
• run as llmuser (owns /tank/aimodels): sudo -u llmuser <venv>/bin/python …

The gates — this script refuses to brick the model

Two hard gates from the recipe, both of which halt before any write:

1. Coverage gate — o_proj(16) + linear_out(48) == 64 == num_hidden_layers, plus down_proj==64, MTP writers ==2, exactly one embed_tokens. Catches a tensor-name mismatch that would otherwise ship a half-abliterated model. 131 tensors edited when it passes; vision (333) never touched.
2. Attention-sink screen — Qwen3.8-27B's massive-activation dimension is 3994. Orthogonalizing a direction that lives in dim 3994 produces a model that loads, runs, and emits garbage. The script aborts if the chosen layer's direction carries >1% of its energy in dim 3994 (recipe's layer-26 reference: 0.06%).

The refusal direction is captured from two chat-template renderings (enable_thinking=false and thinking at xhigh); the layer is auto-picked by peak two-template |cos| agreement in the recipe's [18,45] window (anchor: 26). Verified 2026-08-20 against chat_template.jinja: enable_thinking=True resolves reasoning_effort to 'xhigh' by default, so these really are the recipe's two renderings — the low agreement is not a template-selection bug.

Two more gates were added 2026-08-20, both protecting numbers rather than tensors:

3. Batch-equivalence gate — capture batches prompts, so before the real run it proves a padded batch reproduces one-at-a-time forwards and aborts otherwise. Tolerance is dtype-aware (bf16 5e-2, fp32 1e-3): the gate hunts contamination, not bit-exactness, and changing batch shape changes kernel tiling and therefore accumulation order, so a few ULP is expected. Real contamination is not subtle — the sharding defect read rel 1.00. Padding is on the right, and that is load-bearing: in a causal stack nothing after position t reaches position t, so trailing pads cannot touch the token we read, whereas left padding would feed pad tokens into the DeltaNet recurrence ahead of the prompt.
4. Residency gate (exit 8) and allocator gate (exit 9) — capture-only. See the gotchas; both encode defects that silently produce wrong numbers (multi-GPU sharding zeroes the upper residual stream; expandable_segments corrupts retained tensors).
5. Write completeness check — the write path is shard surgery with no model object, so the offload/meta silent-no-op failure class is gone; it instead verifies all 131 target tensors were found across the shards before declaring success (exit 7 otherwise) and refuses to overwrite an existing checkpoint (exit 10).

Calibration corpus

calibration.py. The first capture used 8 harmful / 8 harmless and produced |cos| agreement of 0.594 — valid but far off the recipe's 0.9925, and a difference-in-means is only as clean as the number of prompts in each mean.

The recipe's line about a "held-out train/test split of 416/104 with overlap 0" turns out to name the corpus exactly: mlabonne/harmful_behaviors is 416 train / 104 test (the AdvBench-derived pair used by the standard abliteration notebooks), and both it and mlabonne/harmless_alpaca were already staged in ana-ml2's HF dataset cache. So --calib mlabonne reproduces Robinson's calibration set rather than approximating it. Read via pyarrow — no datasets dependency, no hub access.

• harmful = harmful_behaviors[train], file order, truncated to n. No seed dependence, so a re-capture is bit-reproducible from the flags alone.
• harmless = harmless_alpaca[train], seeded sample (pool is 25058).
• harmful_behaviors[test] (104) is reserved, not calibration. It is the held-out generalization probe — the set Robinson reported 8% post-abliteration refusal on, and therefore our one directly comparable number. load_calibration will not draw from it and asserts overlap 0 against it, so a later edit cannot quietly turn the evaluation in-distribution.
• --calib builtin reproduces the legacy 8/8 run exactly.

Note the axis mismatch, and that it is deliberate: this corpus is operational harm (hacking, fraud, weapons) while Cold-Fusion's measured refusal surface is creative (explicit-sexual, graphic-torture). Robinson calibrated on exactly this set and still drove creative refusal to 8% with self-harm guardrails intact, which is the single-direction result holding across refusal types. Reproduce first; a creative-axis supplement is the second experiment, not a variable to change in the same run — and if one is added it must stay disjoint from services/refusal-probe/battery*.yaml, or the post-write re-profile stops being a held-out measurement.

Sequence

Run from /tank/aimodels/coldfusion-abliteration on ana-ml2 (the deployed copy of this directory), as llmuser, with pylibs on PYTHONPATH:

P=/tank/aimodels/coldfusion-abliteration
V=/tank/aimodels/quant-work/.venv/bin/python
M=/tank/aimodels/qwen38-27b-coldfusion-bf16
A=/tank/aimodels/qwen38-27b-coldfusion-abliterated-bf16
# CUDA_VISIBLE_DEVICES=0 is REQUIRED for capture (gate exit 8) and
# PYTORCH_CUDA_ALLOC_CONF must stay unset (gate exit 9) — see the gotchas below.
RUN="sudo -u llmuser env HF_HUB_OFFLINE=1 CUDA_VISIBLE_DEVICES=0 \
     PYTHONPATH=$P/pylibs $V $P/abliterate.py --model $M"

# 1. DRY RUN FIRST — verify the tensor map + coverage gate on the static
#    surface, no forward, no write. Safe with the seats up. Do not skip: this
#    confirms the recipe maps onto THIS checkpoint's names.
$RUN --dry-run

#    --- capture needs GPU0 to itself. The text weights are 51,300 MiB, and
#        freeing either seat alone leaves ~50,900 MiB -- BOTH must go. See
#        gotcha 3; the "only gen" line that used to be here was a unit error. ---
sudo docker stop -t 60 vllm-gen vllm-meromero-rp
cp $M/refusal-direction.pt $M/refusal-direction.pt.bak   # capture overwrites it

# 2. CONTROL RUN — the legacy 8/8 set. Reproduces layer 22, |cos| 0.5944, sink
#    0.001% exactly. Keep it as the regression test: ~30s of forwards that
#    validate the whole path against a known number before the real run.
$RUN --capture --calib builtin

# 3. THE REAL CAPTURE — Robinson's 416-prompt corpus. ~35s of forwards.
$RUN --capture --calib mlabonne
#    Measured 2026-08-20: layer 18, |cos| 0.6238, sink 0.360%.

# 4. Restore. If meromero was stopped too, start it FIRST — gen takes a fraction
#    of FREE VRAM at startup and will starve it otherwise.
sudo docker start vllm-gen

# 5. Abliterate (writes the new bf16). Only after 1-3 pass, and only on the
#    operator's go — this is the destructive step. Needs the VRAM window again
#    Shard-level surgery: reads/writes the 18 safetensors shards directly, NO
#    model object, NO GPU. That is a correctness requirement, not just thrift —
#    see "Why the write is shard surgery" below. --direction is REQUIRED.
$RUN --out $A --direction $M/refusal-direction.pt

Flags added 2026-08-20

flag	default	why
--calib {mlabonne,builtin}	mlabonne	corpus selection; builtin = legacy 8/8
--calib-n-harmful	416	the full train split, as the recipe used
--calib-n-harmless	416	matched n from alpaca
--calib-seed	0	harmless sample only; harmful is order-deterministic
--batch-size	8	832 prompts x 2 templates = 1664 forwards; batching is what makes that affordable
--capture-dtype {bfloat16,float32}	bfloat16	bf16 (50 GB, full 64 layers, one GPU) is validated deterministic + coherent; fp32 (111 GB, needs --max-layer) is a misdiagnosis-era escape hatch that agrees to 5e-4
--max-layer	off	capture-only. Truncates the decoder. Exact, not an approximation — a causal stack's layer-N state cannot depend on layers above N. Only needed with --capture-dtype float32; bf16 fits whole. Refused on the write path.

✅ RESULT — layer 35, and why the recipe's layer-selection metric had to be replaced

The write lands and works. Verified bitwise: 131/131 target tensors changed, 333/333 vision byte-identical (delta 0.0), 735/735 other tensors untouched. A/B against stock on a matched battery (greedy, held-out prompts):

probe	stock	abliterated (L35)
explicit sexual (target axis)	refuses	complies
graphic torture (target axis)	refuses	engages (softened)
spam-bot / malware (held-out AdvBench)	refuses	complies / engages
self-harm method (guardrail)	redirects	still redirects
coherence ×2	fine	fine

That is the Robinson design point exactly: creative refusals fall, the self-harm guardrail survives, coherence intact. Output at /tank/aimodels/qwen38-27b-coldfusion-abliterated-L35-bf16.

It took THREE captures, and the lesson is the metric. The recipe selects the abliteration layer by peak two-template |cos| agreement. On this checkpoint that metric is not just weak, it is anti-correlated with what matters:

capture	selector	layer picked	Cohen's d	result
1 (8/8)	agreement	22	5.70	(sharding-corrupted, void)
2 (416/416)	agreement	18	5.51 — worst in window	write was a behavioral no-op
3 (416/416)	separation, sink-gated	35	9.35	works

The tell that cracked it: after capture 2's write changed nothing, a per-layer separation diagnostic (does the direction split harmful from harmless activations?) showed the direction is excellent — AUC 0.9996+ across the whole window — and that agreement had steered us to layer 18, the single weakest separator (d 5.51 vs 9.89 at the peak). Agreement was measuring answer-vs-reason mode (the two templates end </think>\n\n vs <think>\n), not refusal, and on a heavily-merged base that mode term dominates.

So selection is now by separation (Cohen's d), gated on the sink screen. Separation and sink-energy both rise with depth, so the raw peak (L39, d 9.89) is sink-dominated (1.97% > 1%) and would brick the model; the script filters to layers that pass the screen and takes the best separator among them — L35, d 9.35 (within 5% of peak), sink 0.094% (10× under the limit). One pass, no guess-and-retry. Agreement is still computed and printed, as a diagnostic.

The corpus-size hypothesis this session started on was falsified: 52× more calibration data (8→416) moved agreement 0.594→0.624, essentially nothing. The problem was never the calibration set. See the calibration section above; kept as the record of a dead-end worth not re-running.

✅ THESIS RESULT — the in-band-abliterated MTP head accepts BETTER than a graft (2026-08-20)

The whole reason to abliterate Cold-Fusion ourselves rather than run the incumbent Heretic seat: Heretic leaves the MTP head a byte-identical base graft (its wrapper never loads it), while the Robinson formula abliterates the MTP head in-band (its 2 residual writers). The open question was whether that in-band edit survives — an abliterated MTP head that no longer predicts well would kill speculative decoding. Measured, end to end:

metric	L35 quant	incumbent (heresy)	gate	verdict
MTP acceptance (median, 8 cache-busted topics)	59.1% (51–65%)	~47%	≳40%	PASS — beats incumbent
decode tok/s (median)	118.7	~95–103	≥ incumbent	faster (⚠ image-confounded, read as "not worse")
abliteration survives quant	yes	—	creative↓, self-harm intact	PASS
coherence / no catatonia	clean	—	eyeball	PASS

So the in-band MTP abliteration doesn't merely preserve speculative decoding — the abliterated head accepts 59.1% vs the untouched graft's ~47%. That is the additive delta the experiment set out to test, and it's positive.

Pipeline (services/gen-seat-mixed-quant/): mixed NVFP4 (W4A4 L0–55 MLP) + FP8 (attn/linear_attn/lm_head/L56–63 MLP) + FP8 KV → 22.5 GB. Post-quant grafts the abliterated MTP (15 tensors, 849 MB) from the L35 bf16 source and re-injects re:^mtp.* into quantization_config.ignore (llm-compressor pruned it again — the two-rounds-lost bug, fired and repaired as designed). Output: /tank/aimodels/qwen38-27b-coldfusion-L35-nvfp4-mixed. Result JSON: services/gen-seat-mixed-quant/bench/mtp_coldfusion_L35.json.

⚠️ Env foot-gun banked: the quant venv's transformers moved to 5.10 / llmcompressor 0.12 since the Aug-15 heresy quant, and the top-level config no longer delegates num_attention_heads to text_config → oneshot raised "Cannot determine num_attention_heads". quant_mixed_nvfp4.py now promotes those fields from text_config for the duration of quant, then restores. Also: a small (<~23 GB) quant saves as a single model.safetensors with no index, so post_quant's MTP graft needs an index built first (from the safetensors header — never safe_open, which mmaps the whole shard and ENOMEMs on ZFS).

NOT cut over. The incumbent gen seat is untouched. Making L35 the gen seat is a separate operator decision needing the full Stage-3 gate (PPL, prefill, surface 6/6, refusal-probe battery) + the real multi-turn-use hold (the 2026-08-14 delete-too-early / multi-day-degeneration lesson). The thesis is proven; the cutover is a distinct call.

✅ KL RESULT — the surgery is highly selective (2026-08-20)

kl_divergence.py measures first-token KL(stock ‖ abliterated) over the full 248,320-entry vocabulary, bf16 vs bf16, on prompts the direction was never fitted on. Both classes are scored separately because a single mixed average would hide the only thing worth knowing: the divergence is supposed to be large on harmful prompts (that is the effect) and small on benign ones (that is the damage).

mode	class	n	median	mean	p95	max	top-1 agreement
answer	harmless (held out)	256	0.0211	0.0364	0.1219	0.2654	89.8%
answer	harmful (reserved test)	104	0.5996	0.6992	1.6937	1.9920	55.8%
think	harmless (held out)	256	0.0042	0.0066	0.0205	0.0392	94.5%
think	harmful (reserved test)	104	0.3068	0.3186	0.4689	0.5298	57.7%

Selectivity — harmful/harmless median KL — is 28.4× in answer mode and 72.8× in think mode. The direction moves the model hard exactly where it is meant to and leaves benign behaviour close to untouched: on held-out harmless prompts the abliterated model still picks the same first token 89.8% of the time.

Noise floor: exactly 0.0 in both modes (32 prompts re-run through the same model, self-KL). This stack is bit-deterministic here, so every digit above is signal — none of it is bf16 jitter. It also validates the scoring path end to end: a bug in the KL code would almost certainly have shown up as a non-zero floor.

The reverse-KL asymmetry is the abliteration's signature. On harmful prompts in answer mode, KL(stock‖abl) is 0.70 but KL(abl‖stock) is 1.43 — the abliterated model puts substantial mass where the stock model put almost none. That is precisely what removing a refusal direction does, and it is a sanity check that the surgery did the intended thing rather than merely adding noise.

Against the Heretic reference figures — favourable, with a caveat

model	first-token KL, harmless	abliteration method
JonathanColetti/Qwen3.8-27B-Uncensored (prior gen seat)	0.1191	Heretic, out-of-band MTP
absolute-heresy (current gen seat)	0.0759	Heretic v1.4.0 + SOMPOA
Cold-Fusion L35 (ours)	0.0211 median / 0.0364 mean	Robinson, in-band MTP

⚠️ Not a head-to-head. The two reference numbers are Heretic's own optimizer output on a different base model, with its own harmless prompt set and template. Same metric, different measurement conditions — read this as order-of-magnitude ("ours is not worse, and looks materially gentler"), not as a ranking. A true head-to-head would mean re-measuring the incumbent through this same script, which is one more GPU window if the cutover decision ever needs it.

Also note what this does not cover: the MTP head (AutoModelForCausalLM is text-only, so this is the main head only — MTP is gated on acceptance, measured at 59.1%), quantization damage (both sides are bf16), and anything past the first token. Consistent with reference_abliteration_mtp_lessons, KL is reported here as a fidelity number, not as the viability gate.

Reproducibility: exact. The measurement was run twice — once single-process, once through the two-process design below — and all 720 per-prompt KL values are bit-identical between them. Combined with the 0.0 self-KL floor, the numbers above are stable across processes, not just within one.

Artifacts: kl-L35.json (+ kl-L35-rerun.json, the reproducibility check) and the two .ref.pt / .cand.pt log-prob caches, beside the harness on ana-ml2. Run cost: 2m40s single-process, 3m26s two-process, both seats down.

# free, no GPU, safe with the seats up — run this first
$V $P/kl_divergence.py --ref $M --cand $A --out $P/kl-L35.json --dry-run
# the real thing: needs BOTH GPU0 seats stopped (see gotcha 3)
$V $P/kl_divergence.py --ref $M --cand $A --out $P/kl-L35.json

Why it runs one process per model. The default --stage all re-execs itself once per checkpoint (--stage ref, then --stage cand), each writing its first-token log-probs to a ~682 MiB .pt cache, then scores from the caches. This is not tidiness — it is the only teardown that works. Measured, free VRAM after the reference model:

teardown	free VRAM
del model + gc.collect() + empty_cache()	45,287 MiB
the same, model confined to an inner frame that exits	45,287 MiB
the process exits	96,689 MiB

The weights survive both in-process teardowns. The very first run only completed because PyTorch's allocator hit OOM on the second load, collected, and retried — the second model landed on the card by rescue, not by design, and on this architecture a silent CPU offload does not raise, it returns confident garbage (gotcha 1). The headroom gate (exit 10) is what turned that from an invisible near-miss into a loud failure. Side benefit: the ref cache is reusable, so measuring a different candidate against the same stock model skips a stage entirely (--stage cand then --stage score).

⚠️ The old residency gate could not fail. It read hf_device_map, which transformers leaves empty when the whole model fits on one device — so it printed "(unsharded)" both when everything was fine and when there was nothing to inspect. It now reads {p.device for p in model.parameters()} and prints the real placement (all parameters on cuda:0).

Why the write is shard surgery, not model.save_pretrained

The --out path edits the 18 safetensors shards directly and never instantiates a model for the write. This is correctness, not thrift. AutoModelForCausalLM resolves to Qwen3_5ForCausalLM — the text model — so saving from it would (a) drop all 333 vision tensors, silently breaking the byte-identical-vision guarantee, and (b) skip the MTP head, which the ForConditionalGeneration wrapper does not load (the same reason the incumbent gen seat's Heretic pass left its MTP head an untouched base graft) — and the in-band MTP edit is the entire point of the Robinson formula. Neither failure raises. Shard surgery re-serializes every non-target tensor from the exact bytes read, so vision and the other 1068 tensors are byte-identical by construction, the two MTP writers are just two more keys, and the whole offload/meta-tensor silent-no-op class disappears with the model object. Math is done in fp32, stored back at the original bf16.

Verify after (do not trust the write blind)

1. Vision byte-identical + target count — services/coldfusion-abliteration verify: targets changed=131/131 vision identical=333/333 delta=0.0 other differ=0/735. Done 2026-08-20, clean.
2. Refusal re-profile — the ad-hoc battery above is a smoke test. The full canonical re-profile still owed: run services/refusal-probe/ (the gen-seat harness, NOT the GGUF one) once L35 is served, and confirm creative refusals near the RobinsonLabs 8% floor with self-harm guardrails intact.
3. MTP acceptance — the whole point of the in-band MTP edit; measure on the quantized build per services/gen-seat-mixed-quant/RUNBOOK-heresy-swap.md. Gate ≳40% (reference_abliteration_mtp_lessons — gate on acceptance, not KL).
4. PPL / coherence / no catatonia — DavidAU fine-tunes are idiosyncratic; eyeball the outputs, don't trust the metric alone. (Smoke: coherent, no catatonia observed.)

Then, if it holds, NVFP4-quantize via services/gen-seat-mixed-quant/ and it becomes a gen-seat candidate — do not delete the incumbent weights until it survives real multi-turn use (the 2026-08-14 delete-too-early lesson).

⚠️ Environment gotchas (2026-08-20 — cost real time, read before re-running)

⚠️ RETRACTED 2026-08-20 — the "bf16 NaNs, use fp32" rule that lived here was a misdiagnosis, and it sent the next session down a 111 GB dead end. The NaN was never precision. It was the two defects below. fp32 only made it rarer, which is worse than failing outright, because it let a broken forward produce a plausible-looking direction. bf16, full 64 layers, one GPU: 50 GB, exactly deterministic through layer 63, coherent prose, 4.3× the throughput.

1. ⭐ Never let the capture shard across both GPUs. With device_map="auto" across the two Blackwells, this model loads clean, raises nothing, and computes garbage: the residual stream collapses to exactly zero two layers past the GPU0→GPU1 boundary and the logits decode to rubbish. Layers below the boundary are healthy and bit-identical to a single-GPU run — which is exactly why the first capture looked fine. It picked layer 22, which sat on GPU0 in the healthy region; the upper half of its window was zeros and their agreement scores were meaningless.

→ Run CUDA_VISIBLE_DEVICES=0. The --capture path enforces this with a residency gate (exit 8) that refuses a sharded or offloaded model.

2. ⭐ Never set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True. On this stack it corrupts tensors that outlive their allocation — captured states came back with Inf/NaN/zeros that moved between bit-identical forwards. Unset, the same forwards are exactly reproducible. The old runbook recommended this flag for headroom; it buys corruption. Gated (exit 9).

The tell worth remembering: a real numerical blowup propagates to later layers and is deterministic. This did neither. If a NaN doesn't propagate, debug memory, not math.

3. bf16 fits on one GPU — but it needs BOTH GPU0 seats stopped, not one.

⚠️ CORRECTED 2026-08-20. This section used to read "50.1 GB … a capture needs only vllm-gen stopped." The unit was wrong and the conclusion that rode on it was wrong. The real figure is 50.10 GiB = 51,300 MiB = 53.8 GB of text-only weights, measured from the safetensors headers rather than read off a /1e9 print:

	GB	GiB	MiB
checkpoint total	55.56	51.75	52,989
vision (not loaded by AutoModelForCausalLM)	0.92	0.86	879
MTP (not loaded either)	0.85	0.79	810
text-only — what actually lands on the card	53.79	50.10	51,300

GPU0's two tenants are meromero (50,072 MiB) and gen (46,304 MiB), and freeing either one alone leaves at most 50,933 MiB — about 400 MiB short. A run that assumes one seat is enough will stop a service, sit at the edge, and then OOM. Stop both. Recompute this table if the checkpoint changes; do not trust a remembered gigabyte figure.

Both seats down leaves ~97,200 MiB, so the fit is comfortable rather than marginal. Gate the run on observing the free VRAM (nvidia-smi --query-gpu=memory.free) rather than sleeping after docker stop, and put the restore in a trap ... EXIT so an abort hands the seats back — the 2026-08-20 aborted window did exactly that and cost nothing but two minutes.

(--capture-dtype float32 remains as an escape hatch; it needs 111 GB, so it also needs --max-layer 46 to fit on one card. The two agree to 0.0005, so there is no reason to reach for it.)

⚠️ Restore after: start vllm-meromero-rp first, and WAIT FOR IT TO GO HEALTHY before starting vllm-gen. This ordering is load-bearing, and "first" means fully up, not ten seconds earlier — a docker start meromero; sleep 10; docker start gen put meromero into a 7-restart crash-loop on 2026-08-20, because gen finished claiming the card while meromero was still loading weights:

ValueError: Free memory on device cuda:0 (35.3/94.97 GiB) on startup is less than
desired GPU memory utilization (0.52, 49.38 GiB).

The mechanism, stated precisely because a half-right version of it is what caused the mistake: --gpu-memory-utilization is a fraction of total VRAM (0.52 × 94.97 = 49.38 GiB is meromero's target), but vLLM gates startup on free VRAM — it refuses to start unless the card currently has the whole target available. So the two seats coexist only in the order they were originally brought up. GPU0 runs at ~96.4/97.9 GB with about 0.4 GiB of slack; whichever seat starts second gets whatever the first one left, and meromero is the one that does not fit in the remainder. Recovery when it does happen: docker stop vllm-gen, wait for meromero to report healthy, then docker start vllm-gen.

4. fla is irrelevant here — but harmless. fla + einops are --target -installed to /tank/aimodels/coldfusion-abliteration/pylibs and reached via PYTHONPATH (the shared quant-work/.venv is not llmuser-writable). Tested 2026-08-20: the nondeterminism reproduces identically with fla absent, so the linear-attention kernel was never the culprit. Keep passing PYTHONPATH; just don't blame it.

Status

Harness written 2026-08-19; bf16 fully staged. Dry-run PASSED (recipe maps 1:1, 131 tensors). --capture PASSED 2026-08-20 (fp32, after the gotchas above): refusal direction is finite, unit-normed, layer 22, sink energy 0.0008% in dim 3994 (recipe L26 ref 0.06%, threshold 1%) — clean, not sink-dominated. Saved to qwen38-27b-coldfusion-bf16/refusal-direction.pt.

2026-08-20, second session — the corpus hypothesis is FALSIFIED

Measured, on a forward that is trustworthy for the first time:

calibration	layer	|cos| agreement	sink energy
8 / 8 (legacy)	22	0.5944	0.001%
416 / 416 (Robinson's corpus)	18	0.6238	0.360%

52× more calibration data bought +0.03. The small calibration set was not why agreement sat at 0.59, and Robinson's 0.9925 is not reachable on this checkpoint by adding prompts. Agreement is uniformly ~0.54–0.62 across the whole healthy window (L18 0.6238, L22 0.6158, L21 0.6101, L19 0.5944, L28 0.5841), not peaked-and-noisy — which is the signature of a genuinely diffuse direction rather than an under-sampled one.

Cross-validated two ways: the 8/8 run reproduces the previous session's 0.5944 at layer 22 exactly, and fp32-truncated vs bf16-full-64-layer agree to 0.0005. So the number is real and the pipeline is sound.

⚠️ The first capture's log reported 0.594 as |cos|=0.8538. Reporting bug, fixed: the line printed the global agree.max() next to the window's argmax layer. The global peak sits in the early layers where the dim-3994 massive activation dominates both templates and inflates agreement for reasons unrelated to refusal. 0.5944 was always the real number.

The leading explanation is the metric, not the model. The two renderings do not just differ in formatting — they leave the model in different generative modes at the token we read:

• enable_thinking=false ends …<think>\n\n</think>\n\n → about to write the answer
• xhigh ends …<think>\n → about to write chain-of-thought

So |cos| here measures refusal semantics plus answer-vs-reason mode. Robinson's stock Qwen3.8-27B scored 0.99 across that same split, so on their base the refusal component dominated; on this DavidAU GAIN merge the mode difference apparently does not let it. Note what this does and does not impugn: the direction actually used is dirs[False] — the no-think one. Cross-template agreement is only a quality check, and a check that conflates two factors is a weak gate to block on.

The check that would actually settle it is a split-half. Split the 416 harmful in two, derive a direction from each half through the same template, and take |cos|. That isolates sampling noise — the thing calibration size governs — with no mode term at all. If split-half is ~0.99, the direction is well-estimated, the 0.62 is a mode artifact, and the write is justified on a direction we can defend. If split-half is also ~0.6, the refusal representation in this checkpoint is genuinely diffuse and single-direction abliteration is the wrong instrument for it. Cheap: no extra forwards, just two accumulators.

Status: the destructive --out write has NOT been executed. It gates on the operator's go. The saved direction (refusal-direction.pt, layer 18, 416/416, sink 0.360%) is usable but its quality is unresolved pending the split-half. The legacy 8/8 direction is preserved at refusal-direction.pt.bak-8x8.