feat(coldfusion-abliteration): abliteration LANDS at layer 35 — separation selector, shard-surgery write, three false diagnoses corrected

The abliterated model works. A/B vs stock on a matched greedy battery: explicit
sexual + graphic torture (the measured stock refusal surface) go from refused to
complied/engaged, held-out AdvBench prompts loosen, the self-harm guardrail
survives, coherence intact — the Robinson design point exactly. Output at
/tank/aimodels/qwen38-27b-coldfusion-abliterated-L35-bf16, verified bitwise:
131/131 targets changed, 333/333 vision byte-identical (delta 0.0), 735/735
others untouched.

Getting there corrected three diagnoses the prior session had backwards.

1. The layer-selection metric was wrong, and that was the whole ballgame. The
   recipe picks the abliteration layer by peak two-template |cos| agreement. On
   this heavily-merged base that metric is anti-correlated with efficacy: its
   argmax (layer 18) is the WORST-separating layer in the window (Cohen's d 5.51
   vs 9.89 at the peak), and abliterating there was a measured behavioral no-op —
   stock and "abliterated" refused all six probes identically. Cause: the two
   renderings end in different generative modes (</think> vs <think>), so |cos|
   scores answer-vs-reason mode, not refusal, and on a merge the mode term
   dominates. Replaced selection with harmful/harmless SEPARATION (Cohen's d /
   AUC of the direction's projection), gated on the sink screen since separation
   and sink-energy both climb with depth. Picks layer 35 (d 9.35, AUC 0.9997,
   sink 0.094%). Agreement is kept as a printed diagnostic.

2. The "bf16 NaNs, use fp32" rule was a misdiagnosis. The NaN was never
   precision — it was multi-GPU sharding (the residual stream zeroes two layers
   past the GPU0->GPU1 boundary; the first capture's layer 22 happened to sit in
   the healthy region, which is why it looked fine) plus
   PYTORCH_CUDA_ALLOC_CONF=expandable_segments (corrupts retained tensors; the
   corruption MOVED between bit-identical forwards, the tell that it was memory
   not math). On one GPU with a plain allocator, bf16 full-64-layer is exactly
   deterministic and coherent, at 50 GB and 4.3x the throughput of the 111 GB
   fp32 it replaced. Both defects are now hard gates (residency exit 8, allocator
   exit 9); capture pins CUDA_VISIBLE_DEVICES=0.

3. The corpus-size hypothesis was falsified. 52x more calibration data (8->416,
   mlabonne/harmful_behaviors = the recipe's actual AdvBench split, already on the
   box) moved agreement 0.594->0.624 — nothing. Kept the 416/416 corpus anyway
   (calibration.py); it gives the clean separation signal. The held-out 104-prompt
   test split is reserved and asserted disjoint.

Also: the --out write is now shard-level surgery (reads/writes the 18 safetensors
directly, no model object, no GPU). This is correctness, not thrift —
AutoModelForCausalLM resolves to the TEXT model, so save_pretrained would drop all
333 vision tensors AND skip the MTP head (the in-band MTP edit is the entire point
of the Robinson formula). Neither failure raises. Shard surgery makes vision and
the other 1068 tensors byte-identical by construction.

Batched capture with a dtype-aware equivalence gate; hidden states captured via
forward pre-hook (reading output_hidden_states off the returned object is unsafe
here — buffers get recycled). Sharding/allocator lessons promoted to the
quantization playbook (model-agnostic, sections 3.9-3.11 + superseded table); the
selection-metric lesson added to the recipe doc.

The dead layer-18 no-op checkpoint was removed (52 GB, confirmed identical to
stock). Incumbent gen seat untouched. Full canonical refusal-probe re-profile and
MTP-acceptance-on-quant still owed before this becomes a gen-seat candidate.

docs/pfi/abliteration-recipe-qwen38.md

@@ -93,6 +93,23 @@ The two agree at **|cos| 0.96–0.99 across layers 18–45, peaking 0.9925 at la
 vector is the evidence that the direction encodes *refusal semantics* rather than
 *template formatting*. A single-template capture cannot distinguish the two.
 
+> ⚠️ **Two-template agreement is a bad LAYER SELECTOR on a heavily-merged base —
+> use harmful/harmless SEPARATION instead (added 2026-08-20).** On RobinsonLabs'
+> stock Qwen3.8 the agreement was 0.99 and picking its peak was fine. On DavidAU's
+> Cold-Fusion GAIN merge the same metric tops out at **0.62**, and its argmax
+> (layer 18) is the layer with the **worst** refusal separation in the window
+> (Cohen's d 5.51 vs 9.89 at the peak) — abliterating there was a measured
+> behavioral **no-op**. The reason: the two renderings end in different generative
+> modes (`</think>\n\n` = about to answer vs `<think>\n` = about to reason), so
+> `|cos|` scores refusal *plus* mode, and on a merge the mode term dominates. The
+> selector that actually predicts efficacy is **how cleanly the direction splits
+> harmful from harmless prompt activations** (Cohen's d / AUC), gated on the sink
+> screen (separation and sink-energy both rise with depth, so the raw peak is
+> usually sink-dominated). On Cold-Fusion this picked **layer 35** (d 9.35, AUC
+> 0.9997, sink 0.094%) and the abliteration worked. Keep agreement as a
+> diagnostic; do not select on it. See
+> `services/coldfusion-abliteration/README.md`.
+
 ### The attention-sink dimension — the one that bricks the model
 
 **Qwen3.8-27B's massive-activation dimension is `3994`.** It carries 19–21% of

docs/pfi/model-quantization-playbook.md

@@ -254,6 +254,62 @@ not nvidia-modelopt.**
   `read()` + `load(bytes)`, one shard cached at a time.
 - **`vm.overcommit_memory=1`** on ana-ml2 (durable via `playbooks/ana-ml2-overcommit-memory.yaml`).
 
+### 3.9 ⭐⭐ A sharded forward can be silently WRONG — never trust `device_map="auto"` for activations
+
+Splitting **Qwen3.8-27B (Qwen3_5 hybrid)** across the two Blackwells with `device_map="auto"`
+produces a model that loads clean, reports no error, and computes **garbage**: the residual stream
+collapses to **exactly zero** a couple of layers past the GPU0→GPU1 boundary, and the logits decode
+to rubbish (`'8'`, `'�'`, `'b'`). Every layer *below* the boundary stays healthy, deterministic, and
+bit-identical to a single-GPU run — which is what makes it so dangerous. A capture that reads a
+low layer looks perfectly plausible and is fine; one that reads a high layer is reading zeros, and
+nothing in the pipeline says so. Measured 2026-08-20 (§9 Cold-Fusion).
+
+**Rule: any workload that reads activations — refusal-direction capture, calibration, activation
+statistics, PPL — must run on ONE device.** Sharding is for *storage*, and it is only safe when you
+consume the model's final output through an engine that was built for it (vLLM does TP correctly;
+`device_map="auto"` in transformers is not the same thing). If it does not fit on one card, shrink
+the model, not the guarantee: **truncating the decoder to N layers is exact** for any activation
+read at a layer < N (a causal stack's layer-N state cannot depend on layers above N), and it is
+cheap — verified by reproducing the full model's layers 18/20/22/26 bit-for-bit.
+
+**Gate it, don't remember it.** Assert single-device residency and zero offload before the forward:
+
+```python
+dmap = getattr(model, "hf_device_map", {}) or {}
+gpus = {str(v) for v in dmap.values()} - {"cpu", "disk"}
+offloaded = [k for k, v in dmap.items() if str(v) in ("cpu", "disk")]
+if len(gpus) > 1 or offloaded:
+    sys.exit("residency gate FAILED — sharded/offloaded forward reads garbage")
+```
+
+### 3.10 ⭐⭐ `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` corrupts retained tensors
+
+On torch 2.12+cu130 / Blackwell, tensors that **outlive their allocation** come back corrupted with
+this flag set: captured hidden states carried Inf / NaN / zeros that **moved between bit-identical
+forwards** (same input, same weights → a different layer corrupted each time). Unset, the identical
+forwards are exactly reproducible. Several runbooks recommend this flag for headroom on large
+loads; for anything that *keeps* activations it buys corruption.
+
+Two tells that distinguish this from a real numerical blowup, both worth knowing because they
+generalise: a genuine blowup **propagates** to later layers and is **deterministic**. Corruption
+does neither — downstream layers were finite and consistent, and the affected layer moved run to
+run. **If a "NaN" fails to propagate, stop debugging the math and start debugging memory.**
+
+Corollary: **do not read `output_hidden_states=True` off a returned object** on a large multi-device
+load. Take what you need *during* the forward with a `register_forward_pre_hook` that clones to CPU
+immediately — it closes the reuse window and never retains a `[B, seq, hidden]` tensor per layer, so
+it is cheaper than the thing it replaces.
+
+### 3.11 Determinism is a necessary check, not a sufficient one
+
+Both defects above were found by the cheapest possible test — **run the same input twice and diff**
+— which no amount of eyeballing plausible-looking numbers would have caught. Add it to any
+activation-reading pipeline. But note the trap that followed: after fixing the allocator, the run
+went perfectly "deterministic" *because the corrupted layers were now stably zero*. Pair the
+determinism check with a **magnitude** check (residual norms should grow smoothly with depth; an
+exact 0.0 mid-stack is impossible) and, where you can, a **coherence** check (generate 40 tokens and
+read them).
+
 ---
 
 ## 4. Pipeline shape
@@ -346,6 +402,7 @@ day if followed:
 | "Use modelopt, NOT compressed-tensors — compressed-tensors can't load the BF16 MTP head, 0% acceptance" | `docs/runbooks/heretic2-nvfp4-mtp-seat.md` §landmine 2 | **SUPERSEDED 2026-08-14.** The 0% was the missing `re:^mtp.*` ignore (§3.3), not the format. compressed-tensors + the ignore gives 47.7–83.2% acceptance, live. Use compressed-tensors. |
 | "Abliteration desyncs the MTP head → uncensored models can't do MTP" | earlier auto-memory | **SUPERSEDED 2026-08-14.** A modest abliteration preserves MTP (83.7% at bf16). Test MTP on **bf16 first** to isolate abliteration from quant/graft confounds — and isolate before deleting a 50 GB source. |
 | "NVFP4 W4A4 is infeasible, no 4-bit wins both axes, FP8 is the Blackwell answer" | `reference_nvfp4_w4a4_granite_infeasible` | **NARROWED.** True for *uniform* W4A4 (measured on Granite-8B at 30k ctx). W4A4 on bulk MLPs **with FP8 on attention and late layers** is fine and is the current default (§2). |
+| "transformers' Qwen3.5 DeltaNet linear-attention NaNs in bf16 without causal-conv1d; it is precision-driven cancellation and fp32 resolves it" | `services/coldfusion-abliteration/README.md`, `persistent-memory.d/2026-08-20-coldfusion-abliteration-capture.md` | **SUPERSEDED 2026-08-20.** Precision was never the variable. The NaN came from **multi-GPU sharding** and **`expandable_segments`** (§3.9, §3.10); fp32 only made it rarer, which is worse than failing. On one GPU with a plain allocator, **bf16 is exactly deterministic through all 64 layers and generates coherent prose** — at 50 GB and 4.3× the throughput of the 111 GB fp32 it replaced. |
 
 ---