From d676a1375b9772bcfebd7b57181d72ea66eddc50 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 17 Aug 2026 19:26:15 -0700 Subject: [PATCH] memory: watch DavidAU's heretic Qwen3.8, not Cold-Fusion-GAIN V1.1 Cold-Fusion-GAIN V1.1 examined and not adopted -- it is a capability finetune of stock Qwen3.8 and every bench row is labelled [non heretic], so adopting it would reintroduce base refusals the current seat does not have. Records why it reads as uncensored at a glance: DavidAU's back catalog is almost entirely Uncensored-Heretic builds, so the naming pattern implies it. The heretic stage for this one is still in progress from base, and that is the release worth watching. Also banks what makes it interesting when the heretic build lands -- real third-party benchmark gains over stock, claimed MTP acceptance well above ours, thinking tokens cut to a fraction -- and the two caveats: the MTP numbers are GGUF/llama.cpp not vLLM, and a trained MTP head means the free CPU-hash gate would not apply. Also drops the now-stale 'primary until the DavidAU Qwen3.8 lands' clause from the superseded seat entry. --- persistent-memory.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/persistent-memory.md b/persistent-memory.md index ce6cc08..d5c2d02 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -113,7 +113,7 @@ _As of 2026-08-17 โ€” **quiet; gen-seat degeneration saga CLOSED.** Gen seat res - **๐ŸŸข GEN SEAT โ€” SWAPPED to `absolute-heresy` 2026-08-17 (validated, promoted).** Live gen = `/tank/aimodels/qwen38-27b-heresy-nvfp4-mixed` โ€” **MuXodious/Qwen3.8-27B-absolute-heresy** (Heretic v1.4.0 + **SOMPOA**, trial T377, pin `c2374593`) put through our own mixed NVFP4+FP8 recipe. Chosen because it beats the incumbent on **both** axes at once: author refusals 2/101 vs 12/100, first-token KL 0.0759 vs 0.1191. **Gate (probe :8017, pinned nightly, seat-matched flags): MTP 47.2% (inc. 48.2%), decode 103.5 tok/s (96.4), prefill 6618/5403 @6.7k/27k (6334/5085), PPL 6.910 (7.059 โ€” 2.1% BETTER), surface 6/6, abliteration 4/4, and 0/55 refusals on our battery-instruct arm with ZERO EMPTY (no catatonia).** โš  speed deltas are **image-confounded** (probe on the pinned nightly, incumbent numbers from an earlier image) โ€” read as "not worse", not a clean win. All 7 LiteLLM aliases verified end-to-end; GPU0 at 91.3/97.9 GB with meromero healthy (more headroom than the old build's 96.8). โš  **RC1, 2 days old, ~348 downloads** โ€” watch real multi-turn use. **ROLLBACK:** `sudo cp /opt/docker/compose/gen-seat/.env.bak-heresy-20260817 /opt/docker/compose/gen-seat/.env && cd /opt/docker/compose/gen-seat && sudo docker compose up -d vllm-gen`; incumbent weights UNTOUCHED at `qwen38-27b-uncensored-nvfp4-mixed` โ€” **do NOT delete** until this holds. Runbook `services/gen-seat-mixed-quant/RUNBOOK-heresy-swap.md`. -- **๐ŸŸข PRIOR GEN SEAT โ€” RESOLVED 2026-08-17 (the multi-day degeneration saga); now the ROLLBACK target.** Was the in-house **JonathanColetti/Heretic mixed NVFP4+FP8 build** (`/tank/aimodels/qwen38-27b-uncensored-nvfp4-mixed`, FP8 attention) on **vLLM nightly PINNED** `vllm/vllm-openai:nightly-311b3513โ€ฆ` (`v0.27.2rc1.dev150`, carries #51113 mamba fix), **MTP ON, prefix-caching ON**. Operator-confirmed **coherent through 60k tokens** real multi-turn. Root cause = TWO compounding real causes: (1) genuine vLLM `qwen3_5_mtp`ร—GDN partial-accept bug (#51113, architectural across vLLM/SGLang/llama.cpp, fixed by nightly), and (2) **AEON's full W4A4** being lowest-fidelity on the known activation gradient (W4A4 < W4+FP8 < W4+bf16) โ†’ ~15-20% stochastic degeneration on top of (1). **AEON PURGED** (re-pullable `sakamakismile/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4`). Full lesson `docs/pfi/model-quantization-playbook.md` ยง3.8. Primary **until the DavidAU Qwen3.8 lands.** โš  **pinned nightly is bleeding-edge โ€” move to a stable release once #51113 ships in one (the standing follow-up).** 7 aliases (gen/gen-reasoning/summarizer/-large/classifier/image-judge/qwen-image-bench) all route here. Seat carries `--default-chat-template-kwargs '{"reasoning_effort":"medium"}'` (per-request overridable, affects gen-reasoning only). Commits `d28a371`,`2f2bbce`,`2185964`. +- **๐ŸŸข PRIOR GEN SEAT โ€” RESOLVED 2026-08-17 (the multi-day degeneration saga); now the ROLLBACK target.** Was the in-house **JonathanColetti/Heretic mixed NVFP4+FP8 build** (`/tank/aimodels/qwen38-27b-uncensored-nvfp4-mixed`, FP8 attention) on **vLLM nightly PINNED** `vllm/vllm-openai:nightly-311b3513โ€ฆ` (`v0.27.2rc1.dev150`, carries #51113 mamba fix), **MTP ON, prefix-caching ON**. Operator-confirmed **coherent through 60k tokens** real multi-turn. Root cause = TWO compounding real causes: (1) genuine vLLM `qwen3_5_mtp`ร—GDN partial-accept bug (#51113, architectural across vLLM/SGLang/llama.cpp, fixed by nightly), and (2) **AEON's full W4A4** being lowest-fidelity on the known activation gradient (W4A4 < W4+FP8 < W4+bf16) โ†’ ~15-20% stochastic degeneration on top of (1). **AEON PURGED** (re-pullable `sakamakismile/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4`). Full lesson `docs/pfi/model-quantization-playbook.md` ยง3.8. (Superseded as primary by `absolute-heresy` 2026-08-17.) โš  **pinned nightly is bleeding-edge โ€” move to a stable release once #51113 ships in one (the standing follow-up).** 7 aliases (gen/gen-reasoning/summarizer/-large/classifier/image-judge/qwen-image-bench) all route here. Seat carries `--default-chat-template-kwargs '{"reasoning_effort":"medium"}'` (per-request overridable, affects gen-reasoning only). Commits `d28a371`,`2f2bbce`,`2185964`. - **๐Ÿ”ต RP SEAT โ€” FABLE-FUSION serving `char-rp-reasoning` (evaluation window, unchanged this session).** `fablefusion-charrp-probe` ana-ml2 GPU1 `:8019` serving `char-rp-probe` (`kkuspa/Qwen3.6-27B-Fable-Fusion-711-โ€ฆ-MTP-NVFP4A16`). LiteLLM `char-rp-reasoning` + `char-rp-fable` both route to it (deliberate repoint, documented in `stacks/litellm/conf/config.yaml`). `darkscarlett-charrp-reasoning` is `compose down`, weights intact at `/tank/aimodels/darkscarlett-nvfp4-work/`. **โณ STILL AWAITING operator's hands-on read of FF prose** (refusal question settled: FF 15.8% vs DS 92.5% cold-framing; DS v1.0 never abliterated). โš  FF reasons 2.1โ€“4.6k chars โ†’ use `max_tokens` โ‰ฅ3072. `ReadyArt/Dark-Scarlett-27B-v2.0` (Qwen3.8) is GATED (`403 awaiting review`) โ€” operator ruled not-interesting, do NOT re-propose. **DS regeneration for brokkr RETIRED 2026-08-17 โ€” unqueued, do NOT run** (`9c1405b`): brokkr withdrew on the operator's call because (a) our `ictrl-pair-unwrapped`/`-wrapped` control isolates the classifier over-fire cleanly where DS's cross-class delta only bounded it, and (b) **DS v2 releases soon**, so a k=5 v1 baseline baselines a superseded version. Spec at `services/refusal-probe/darkscarlett-regen-spec.md` stays banked as the record of the run that will not happen (axes + per-class grading asymmetry still correct, checklist struck through). No GPU1 window was ever spent. A DS-**v2** characterization would be a fresh purpose-scoped ask. @@ -131,6 +131,8 @@ _As of 2026-08-17 โ€” **quiet; gen-seat degeneration saga CLOSED.** Gen seat res - **๐ŸŸข WT #401 (fd-leak deadlock) CLOSED 2026-08-17 โ€” one ping still owed.** worldtree-dev closed it on our demo verify. Layers: **(a) their `e41b139`** pins `ulimits: nofile 65536/65536` in the worldtree compose anchor โ€” **demo VERIFIED** (api + matrix recreated 22:55:34Z, `ulimit -Sn`=65536); **personal/pinned are covered-not-verified**, they inherit at their next promotion/recreate. **(b) our host floor is STAGED, NOT ACTIVE** โ€” `/etc/docker/daemon.json` on corviduo-dev carries `default-ulimits nofile 65536/65536` but **`default-ulimits` is NOT SIGHUP-reloadable** (measured on 29.4.3: post-reload the daemon's own "Reloaded configuration" log omits it and a fresh container still reports 1024). Activation needs a full dockerd restart = bounces all 13 containers; **worldtree-dev explicitly does NOT want one**, and `live-restore:true`-then-restart is PARKED as a separate host-side improvement for the operator to rule on, never folded into #401. Playbook `playbooks/corviduo-dev-docker-default-ulimits.yaml` (verify step 3 fails BY DESIGN until a restart). Hourly fd tripwire on corviduo-dev stays armed. **โณ OWED: ping worldtree-dev in thread `01M08QQ655XD6VKEV7MA9GX0NS` once worldtree-personal recreates and 65536 is confirmed there.** Commit `7f3f265`. +- **โณ WATCHING: DavidAU's HERETIC build of Qwen3.8-27B โ€” the one worth waiting for.** `DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1` examined 2026-08-17 and **NOT adopted**: it is a capability/efficiency finetune of **stock** Qwen3.8 and every bench row on its card is labelled **`[non heretic]`** โ€” adopting it would reintroduce base refusals the current seat does not have. โš  **Easy to misread as uncensored** (operator did, and it is a fair mistake): DavidAU's back catalog is almost all `Uncensored-Heretic` builds โ€” Fable-Fusion 711, Qwen3.5-9B Cold-Fusion โ€” so the naming pattern implies it. This one simply has not had that stage run yet; the card's roadmap says the HERETIC version is **IN PROGRESS from base**. **That** is the release to watch, not this one. What makes it worth watching: third-party benches (Nightmedia, mxfp8) beat stock Qwen3.8 by **+0.064 arc/c, +0.056 arc/e, +0.056 obkqa**; claimed **MTP acceptance 55.7% (record 59.9%)** vs our measured 47.2%; **thinking tokens cut to 1/10โ€“1/2**; PPL *dropped* vs base. Same GAIN/Cold-Fusion pipeline that produced Fable-Fusion 711, which we already serve on `char-rp-reasoning` โ€” proven in-fleet, not just claimed. Structurally clean (1199 tensors, 15 mtp in shard 18, 333 visual). โš  MTP/speed figures are **GGUF/llama.cpp on a 5090, not vLLM** โ€” may not transfer; and because its MTP head was likely *trained*, the free CPU-hash shortcut would NOT apply (it won't match base) so a real acceptance gate would be needed. + - **OPEN FOLLOW-UPS (parked):** move gen seat off pinned-nightly to stable once #51113 ships; Lobe one-time TTS UI pass; delete the 1.8GB litellm dump; `harden-esh-docker-vm` (park id 28, PROMOTED โ€” Tier-1 done, `/mnt/books` stays hard w/ watchdog); chatterbox-fast build-context divergence; #363 research-wing ingest (no deadline); optionally attach our MTP reproducer to vllm#47087 (needs a GitHub identity โ€” operator's call). - **althing monitor** ARMED (handle `infra-ops`). โš ๏ธ Re-arm ONLY after a real FIRE (rc0), never after a plain operator turn (bounces rc3); spawn `althing-wake-listener` as its OWN `run_in_background` task, never chained with `&` (orphans it โ€” hit this twice 2026-08-17, `stop-monitor` reclaims).