From 48410a6a907f61ba253810e41ef30e0933c89467 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 21 Aug 2026 00:45:54 -0700 Subject: [PATCH] =?UTF-8?q?chore(coldfusion-abliteration):=20delete=20the?= =?UTF-8?q?=20Cold-Fusion=20bf16=20weights=20=E2=80=94=20~154=20GB=20recla?= =?UTF-8?q?imed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator directive following the decision to abandon the Cold-Fusion base. Removed with explicit literal paths, one at a time: qwen38-27b-coldfusion-bf16 stock DavidAU base qwen38-27b-coldfusion-abliterated-L35-bf16 Robinson L35 qwen38-27b-coldfusion-h300-mtp-bf16 Heretic-300 + MTP graft qwen38-27b-coldfusion-heretic300-bf16 raw Heretic export Verified against ZFS used, not df: 4.48T -> 4.33T, ~154 GB. No snapshots were holding the blocks, all four paths confirmed gone, gen seat unaffected. The last two were hardlink twins -- same inode, links=2, because the MTP graft hardlinked every unchanged shard -- so deleting only one would have freed nothing. `du` across several paths in a single invocation dedupes hardlinks and reported heretic300-bf16 as 2.5K, which would have made a size estimate wrong in both directions. Check `stat -c %h` before sizing a delete. Kept deliberately, so the research record outlives the weights: qwen38-27b-coldfusion-bf16.PROVENANCE.txt pinned HF revision 9c44193f coldfusion-abliteration/ harness, 300-trial Optuna journal, catatonia-T260.json With those two, every deleted build is reproducible: re-pull stock at the pin and replay the winning config. Held back pending an explicit call: the two NVFP4 quants, h300-nvfp4-mixed (the only remaining servable copy of the Heretic-300 result) and L35-nvfp4-mixed. The directive named bf16 weights; these are quants, and there is no storage pressure arguing for haste at 4.26T free. --- persistent-memory.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/persistent-memory.md b/persistent-memory.md index eeb0580..8a5b72e 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -129,7 +129,9 @@ _As of 2026-08-21 00:35 — **the Heretic-300 session, and its reversal** (see t - **COST, stated plainly:** we give up **8/100 refusals** (h300) and go back to **29/100** (heresy's own bar) — a 3.6× regression on the refusal axis, which was the entire point of the Heretic-300 run. Also lost: the in-band-vs-pristine MTP comparison stays academic. **Accepted deliberately** — a seat that breaks the operator's daily client is worth less than one that occasionally refuses. - **★ WHAT CARRIES FORWARD (the operator's point, and it is right).** None of the Heretic-300 learning was in the Cold-Fusion weights. Still valid and model-agnostic: `direction_scope=0` beats per-layer on a merged base (8/100 vs 52/100); aggression is not the lever (r=−0.561); **PR #317 silently drops the MTP head on save** — always diff tensor keys; the MPOA/sink-screen reasoning; `graft_mtp.py`, `kl_divergence.py`, `catatonia_gate.py`, `heretic_export.py`; **a pristine MTP graft accepts as well as an in-band edit (59.7% vs 59.1%)**; and the new `think_prior.py` probe. **The methodology is the deliverable; the base was the wrong substrate.** - **★★ NEW ACCEPTANCE GATE, earned here — add a FORMAT-COMPLIANCE check to every abliteration/base evaluation, and run it on the STOCK BASE BEFORE spending a GPU-week.** `think_prior.py` on the stock candidate is a ~10s CPU measurement that would have disqualified Cold-Fusion **before** the 300-trial study ever ran. Screen candidate bases for it. (Related: Heretic's objective has no format term at all — same blindness that removed the self-harm guardrail.) - - **⚠ NOTHING DELETED.** `h300-nvfp4-mixed`, `h300-mtp-bf16`, `coldfusion-L35-*`, `coldfusion-bf16`, the 300-trial Optuna journal and `catatonia-T260.json` are all intact on disk. "Abandon" = stop serving, not `rm`. + - **🗑️ bf16 WEIGHTS DELETED 2026-08-21 00:50 (operator directive) — ~154 GB reclaimed, verified against ZFS `used` (4.48T → 4.33T), no snapshots holding blocks.** Removed with explicit literal paths, one at a time: `qwen38-27b-coldfusion-bf16` (stock), `-abliterated-L35-bf16`, `-h300-mtp-bf16`, `-heretic300-bf16`. ⚠ The last two were **hardlink twins** (`links=2`, same inode — the MTP graft hardlinked every unchanged shard), so deleting only one would have freed **nothing**; `du` across several paths in one invocation dedupes hardlinks and under-reported `heretic300-bf16` as 2.5K. Check `stat -c %h` before sizing a delete. + - **KEPT deliberately:** `qwen38-27b-coldfusion-bf16.PROVENANCE.txt` (4 KB — carries the pinned HF revision **`9c44193f07782c85c0f437a5d8466ba5c95c95fe`**, so stock is re-pullable) and `/tank/aimodels/coldfusion-abliteration/` (1.1 GB — harness, the 300-trial Optuna journal, `catatonia-T260.json`). **The research record survives the weights.** With those two, every deleted build is reproducible: re-pull stock at the pin, replay the winning config. + - **⏳ STILL ON DISK, not named in the delete directive:** `qwen38-27b-coldfusion-h300-nvfp4-mixed` (22 GB — the only remaining *servable* copy of the Heretic-300 result) and `qwen38-27b-coldfusion-L35-nvfp4-mixed` (22 GB). The directive said "bf16 weights"; these are quants, so they were **held pending an explicit call.** No storage pressure argues for haste (4.26T free). - **⚪ GEN SEAT DEFECT 2026-08-21 — RESOLVED BY THE ROLLBACK ABOVE. The h300 build emitted an UNTERMINATED `` into `content`, ~27% of the time, on any temp>0 alias. Operator-reported via Lobe ("sends CoT, never completes the turn").** - **Mechanism.** With `enable_thinking:false` the chat template appends a **pre-closed** `\n\n\n\n` to the prompt (jinja L165-166). The h300 model **opens a fresh `` anyway and never closes it** — verified raw: `has : False`, `finish_reason: stop`, reasoning *and* answer in one `content` blob starting `Ok, let's figure this out:`. vLLM's `qwen3` reasoning parser can't catch it: the prompt already closed the block, so the parser isn't in reasoning state and the tag is just text (`reasoning_content` empty, `reasoning_tokens: 0`). **The client is blameless** — Lobe correctly treats an unterminated `` as still-thinking, so it renders an endless thought bubble and never shows the answer.