vh
530f1452e8
Captures the session's real work as the in-flight focus: abliterating DavidAU Cold-Fusion with the Robinson formula. fp32 capture succeeded (finite direction, layer 22, sink-clean) but two-template agreement is 0.59 vs Robinson's 0.99 — calibration-set expansion is the next step. New detail file records the full saga including the transformers/DeltaNet bf16-NaN fight (fp32 fix, the causal-conv1d kernel gap, the seat-restart VRAM-greed gotcha). Supersedes the earlier "watch for DavidAU's heretic build" posture — we abliterate it ourselves. Auto-archived 4 closed entries (Recent decisions: Booth-3-features 08-05, worldtree-sdk 07-31; Tried and abandoned: containerd-race 08-03, mv-rename 08-02) to archival-memory.md; the rest of the over-cap entries are held back by the <14-day and open-deferred guards. Index 331 -> 327.
5.5 KiB
[2026-08-20] Cold-Fusion abliteration — Robinson recipe captured, and the transformers/DeltaNet bf16-NaN fight
The real work of the session: abliterate DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1
using the MTP-aware, vision-preserving Robinson formula (documented in
docs/pfi/abliteration-recipe-qwen38.md from RobinsonLabs/Qwen3.8-27B-abliterated).
Harness: services/coldfusion-abliteration/. Runs on ana-ml2.
Why this model, why abliterate it ourselves
Stock Cold-Fusion's refusal profile was probed 2026-08-19 (Q6_K GGUF on llama.cpp, 24-prompt battery, hand-verified after a keyword-classifier bug): ~33% creative refusal, concentrated on explicit-sexual + graphic-torture; 4/5 hard-harm technical refused; self-harm guardrails intact 3/3; benign over-refusal 0. So there is a real creative-content refusal surface to remove. This supersedes the earlier "watch for DavidAU's own heretic build" posture — we abliterate it ourselves.
It is additive over the current gen seat. The live Heretic seat
(qwen38-27b-heresy-bf16) left its MTP head a byte-identical base graft —
the Qwen3_5ForConditionalGeneration wrapper never loads it, so Heretic could
not touch it. The Robinson formula abliterates the MTP head in-band (its 2
residual-write matrices), and the MTP head is what gates speculative acceptance.
That in-band MTP edit is the delta this experiment tests.
Recipe maps 1:1 — dry-run PASSED
Against the staged bf16: 1199 tensors, 333 vision preserved,
down_proj=64 o_proj=16 linear_out=48 mtp=2 embed=1, coverage gate 6/6, exactly
131 tensors to orthogonalize. Same architecture as RobinsonLabs' base, no
name drift. Two hard gates in the harness halt before any write: the coverage
identity o_proj(16)+linear_out(48)==64, and the attention-sink screen on
dim 3994 (orthogonalizing a direction living there bricks the model).
Capture SUCCEEDED — but only after a real environment fight (the durable lessons)
The transformers Qwen3.5 DeltaNet linear-attention NaNs in bf16 on ana-ml2.
The fast-path needs BOTH flash-linear-attention (fla, triton, installs fine)
AND causal-conv1d (needs nvcc to build — absent, no prebuilt wheel).
Without causal-conv1d the DeltaNet short-conv runs the torch fallback, which
produces nondeterministic all-NaN hidden states in bf16 (same 11-token input:
finite on one forward, NaN at layer 4 on the next). bf16 and fp32 share exponent
range, so this is precision-driven catastrophic cancellation, not overflow —
fp32 resolves it. Diagnosed via diag_nan.py / diag2.py: sdpa + plain
prompt = 65 layers all finite; chat-template input = NaN; the trigger is the
input path through the unstable recurrence.
Fixes, all in the committed harness (7abd301):
--captureloads fp32; the write/surgery path stays bf16 (no forward, no NaN).- A finite-gate aborts on a non-finite direction — the sink screen alone
can't catch it (
nan > thresholdis False, so a NaN direction "passed" it and saved silently on the first run). attn_implementation="sdpa"pinned.
fp32 (110 GB) needs the whole GPU. device_map=auto packed it tight and the
forward OOM'd against the resident seats. Had to stop three seats for VRAM:
vllm-meromero-rp, vllm-fablefusion-probe, and production vllm-gen.
⚠ Restart order matters: gen restarted into an empty GPU0 and greedily
grabbed 64 GB (vLLM takes a fraction of free memory at startup), starving
meromero into a crash-loop. Fixed by bringing meromero up first, then gen
into the remainder. All three restored to healthy.
⚠ fla lives in a side dir, not the venv. The shared
/tank/aimodels/quant-work/.venv is not llmuser-writable; fla + einops are
--target-installed to /tank/aimodels/coldfusion-abliteration/pylibs and
reached via PYTHONPATH. Prune deps that shadow the venv's torch/transformers.
Result
Refusal direction: finite, unit-normed, layer 22, sink energy 0.0008%
in dim 3994 (recipe L26 ref 0.06%, threshold 1%) — clean, not sink-dominated.
Saved to /tank/aimodels/qwen38-27b-coldfusion-bf16/refusal-direction.pt.
⚠ QUALITY CAVEAT — the reason the next step is calibration-set expansion.
Two-template |cos| agreement at layer 22 is 0.59, well below Robinson's
0.99. Almost certainly the small calibration set: 8 harmful / 8 harmless
(HARMFUL/HARMLESS in abliterate.py) vs Robinson's 416 / 104. The direction
is valid and sink-clean but noisier than ideal; abliterating on it risks
under-removing refusals or nicking capability. Expand the sets to a few
hundred each and re-capture before the --out write.
Sequence from here
- Expand HARMFUL/HARMLESS calibration sets → re-capture (fp32, seats down).
--outwrite (bf16 surgery, no forward) →qwen38-27b-coldfusion-abliterated-bf16.- Verify: vision byte-identical, refusal re-profile via
services/refusal-probe/(the canonical harness, NOT the ad-hoc GGUF one), MTP acceptance on the quant (gate ≳40%, not KL —reference_abliteration_mtp_lessons), PPL/coherence. - NVFP4-quantize via
services/gen-seat-mixed-quant/→ gen-seat candidate. Do NOT delete the incumbent (qwen38-27b-heresy-nvfp4-mixed) until it holds through real multi-turn use.
bf16 staged at /tank/aimodels/qwen38-27b-coldfusion-bf16 (pinned 9c44193,
provenance recorded). All write paths re-stop the seats for fp32 VRAM — batch
re-capture + write in one window. Commits ccb56a0, 1857a8e, b56cb0d,
7abd301.