diff --git a/persistent-memory.md b/persistent-memory.md index ce6cc08..d5c2d02 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -113,7 +113,7 @@ _As of 2026-08-17 โ€” **quiet; gen-seat degeneration saga CLOSED.** Gen seat res - **๐ŸŸข GEN SEAT โ€” SWAPPED to `absolute-heresy` 2026-08-17 (validated, promoted).** Live gen = `/tank/aimodels/qwen38-27b-heresy-nvfp4-mixed` โ€” **MuXodious/Qwen3.8-27B-absolute-heresy** (Heretic v1.4.0 + **SOMPOA**, trial T377, pin `c2374593`) put through our own mixed NVFP4+FP8 recipe. Chosen because it beats the incumbent on **both** axes at once: author refusals 2/101 vs 12/100, first-token KL 0.0759 vs 0.1191. **Gate (probe :8017, pinned nightly, seat-matched flags): MTP 47.2% (inc. 48.2%), decode 103.5 tok/s (96.4), prefill 6618/5403 @6.7k/27k (6334/5085), PPL 6.910 (7.059 โ€” 2.1% BETTER), surface 6/6, abliteration 4/4, and 0/55 refusals on our battery-instruct arm with ZERO EMPTY (no catatonia).** โš  speed deltas are **image-confounded** (probe on the pinned nightly, incumbent numbers from an earlier image) โ€” read as "not worse", not a clean win. All 7 LiteLLM aliases verified end-to-end; GPU0 at 91.3/97.9 GB with meromero healthy (more headroom than the old build's 96.8). โš  **RC1, 2 days old, ~348 downloads** โ€” watch real multi-turn use. **ROLLBACK:** `sudo cp /opt/docker/compose/gen-seat/.env.bak-heresy-20260817 /opt/docker/compose/gen-seat/.env && cd /opt/docker/compose/gen-seat && sudo docker compose up -d vllm-gen`; incumbent weights UNTOUCHED at `qwen38-27b-uncensored-nvfp4-mixed` โ€” **do NOT delete** until this holds. Runbook `services/gen-seat-mixed-quant/RUNBOOK-heresy-swap.md`. -- **๐ŸŸข PRIOR GEN SEAT โ€” RESOLVED 2026-08-17 (the multi-day degeneration saga); now the ROLLBACK target.** Was the in-house **JonathanColetti/Heretic mixed NVFP4+FP8 build** (`/tank/aimodels/qwen38-27b-uncensored-nvfp4-mixed`, FP8 attention) on **vLLM nightly PINNED** `vllm/vllm-openai:nightly-311b3513โ€ฆ` (`v0.27.2rc1.dev150`, carries #51113 mamba fix), **MTP ON, prefix-caching ON**. Operator-confirmed **coherent through 60k tokens** real multi-turn. Root cause = TWO compounding real causes: (1) genuine vLLM `qwen3_5_mtp`ร—GDN partial-accept bug (#51113, architectural across vLLM/SGLang/llama.cpp, fixed by nightly), and (2) **AEON's full W4A4** being lowest-fidelity on the known activation gradient (W4A4 < W4+FP8 < W4+bf16) โ†’ ~15-20% stochastic degeneration on top of (1). **AEON PURGED** (re-pullable `sakamakismile/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4`). Full lesson `docs/pfi/model-quantization-playbook.md` ยง3.8. Primary **until the DavidAU Qwen3.8 lands.** โš  **pinned nightly is bleeding-edge โ€” move to a stable release once #51113 ships in one (the standing follow-up).** 7 aliases (gen/gen-reasoning/summarizer/-large/classifier/image-judge/qwen-image-bench) all route here. Seat carries `--default-chat-template-kwargs '{"reasoning_effort":"medium"}'` (per-request overridable, affects gen-reasoning only). Commits `d28a371`,`2f2bbce`,`2185964`. +- **๐ŸŸข PRIOR GEN SEAT โ€” RESOLVED 2026-08-17 (the multi-day degeneration saga); now the ROLLBACK target.** Was the in-house **JonathanColetti/Heretic mixed NVFP4+FP8 build** (`/tank/aimodels/qwen38-27b-uncensored-nvfp4-mixed`, FP8 attention) on **vLLM nightly PINNED** `vllm/vllm-openai:nightly-311b3513โ€ฆ` (`v0.27.2rc1.dev150`, carries #51113 mamba fix), **MTP ON, prefix-caching ON**. Operator-confirmed **coherent through 60k tokens** real multi-turn. Root cause = TWO compounding real causes: (1) genuine vLLM `qwen3_5_mtp`ร—GDN partial-accept bug (#51113, architectural across vLLM/SGLang/llama.cpp, fixed by nightly), and (2) **AEON's full W4A4** being lowest-fidelity on the known activation gradient (W4A4 < W4+FP8 < W4+bf16) โ†’ ~15-20% stochastic degeneration on top of (1). **AEON PURGED** (re-pullable `sakamakismile/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4`). Full lesson `docs/pfi/model-quantization-playbook.md` ยง3.8. (Superseded as primary by `absolute-heresy` 2026-08-17.) โš  **pinned nightly is bleeding-edge โ€” move to a stable release once #51113 ships in one (the standing follow-up).** 7 aliases (gen/gen-reasoning/summarizer/-large/classifier/image-judge/qwen-image-bench) all route here. Seat carries `--default-chat-template-kwargs '{"reasoning_effort":"medium"}'` (per-request overridable, affects gen-reasoning only). Commits `d28a371`,`2f2bbce`,`2185964`. - **๐Ÿ”ต RP SEAT โ€” FABLE-FUSION serving `char-rp-reasoning` (evaluation window, unchanged this session).** `fablefusion-charrp-probe` ana-ml2 GPU1 `:8019` serving `char-rp-probe` (`kkuspa/Qwen3.6-27B-Fable-Fusion-711-โ€ฆ-MTP-NVFP4A16`). LiteLLM `char-rp-reasoning` + `char-rp-fable` both route to it (deliberate repoint, documented in `stacks/litellm/conf/config.yaml`). `darkscarlett-charrp-reasoning` is `compose down`, weights intact at `/tank/aimodels/darkscarlett-nvfp4-work/`. **โณ STILL AWAITING operator's hands-on read of FF prose** (refusal question settled: FF 15.8% vs DS 92.5% cold-framing; DS v1.0 never abliterated). โš  FF reasons 2.1โ€“4.6k chars โ†’ use `max_tokens` โ‰ฅ3072. `ReadyArt/Dark-Scarlett-27B-v2.0` (Qwen3.8) is GATED (`403 awaiting review`) โ€” operator ruled not-interesting, do NOT re-propose. **DS regeneration for brokkr RETIRED 2026-08-17 โ€” unqueued, do NOT run** (`9c1405b`): brokkr withdrew on the operator's call because (a) our `ictrl-pair-unwrapped`/`-wrapped` control isolates the classifier over-fire cleanly where DS's cross-class delta only bounded it, and (b) **DS v2 releases soon**, so a k=5 v1 baseline baselines a superseded version. Spec at `services/refusal-probe/darkscarlett-regen-spec.md` stays banked as the record of the run that will not happen (axes + per-class grading asymmetry still correct, checklist struck through). No GPU1 window was ever spent. A DS-**v2** characterization would be a fresh purpose-scoped ask. @@ -131,6 +131,8 @@ _As of 2026-08-17 โ€” **quiet; gen-seat degeneration saga CLOSED.** Gen seat res - **๐ŸŸข WT #401 (fd-leak deadlock) CLOSED 2026-08-17 โ€” one ping still owed.** worldtree-dev closed it on our demo verify. Layers: **(a) their `e41b139`** pins `ulimits: nofile 65536/65536` in the worldtree compose anchor โ€” **demo VERIFIED** (api + matrix recreated 22:55:34Z, `ulimit -Sn`=65536); **personal/pinned are covered-not-verified**, they inherit at their next promotion/recreate. **(b) our host floor is STAGED, NOT ACTIVE** โ€” `/etc/docker/daemon.json` on corviduo-dev carries `default-ulimits nofile 65536/65536` but **`default-ulimits` is NOT SIGHUP-reloadable** (measured on 29.4.3: post-reload the daemon's own "Reloaded configuration" log omits it and a fresh container still reports 1024). Activation needs a full dockerd restart = bounces all 13 containers; **worldtree-dev explicitly does NOT want one**, and `live-restore:true`-then-restart is PARKED as a separate host-side improvement for the operator to rule on, never folded into #401. Playbook `playbooks/corviduo-dev-docker-default-ulimits.yaml` (verify step 3 fails BY DESIGN until a restart). Hourly fd tripwire on corviduo-dev stays armed. **โณ OWED: ping worldtree-dev in thread `01M08QQ655XD6VKEV7MA9GX0NS` once worldtree-personal recreates and 65536 is confirmed there.** Commit `7f3f265`. +- **โณ WATCHING: DavidAU's HERETIC build of Qwen3.8-27B โ€” the one worth waiting for.** `DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1` examined 2026-08-17 and **NOT adopted**: it is a capability/efficiency finetune of **stock** Qwen3.8 and every bench row on its card is labelled **`[non heretic]`** โ€” adopting it would reintroduce base refusals the current seat does not have. โš  **Easy to misread as uncensored** (operator did, and it is a fair mistake): DavidAU's back catalog is almost all `Uncensored-Heretic` builds โ€” Fable-Fusion 711, Qwen3.5-9B Cold-Fusion โ€” so the naming pattern implies it. This one simply has not had that stage run yet; the card's roadmap says the HERETIC version is **IN PROGRESS from base**. **That** is the release to watch, not this one. What makes it worth watching: third-party benches (Nightmedia, mxfp8) beat stock Qwen3.8 by **+0.064 arc/c, +0.056 arc/e, +0.056 obkqa**; claimed **MTP acceptance 55.7% (record 59.9%)** vs our measured 47.2%; **thinking tokens cut to 1/10โ€“1/2**; PPL *dropped* vs base. Same GAIN/Cold-Fusion pipeline that produced Fable-Fusion 711, which we already serve on `char-rp-reasoning` โ€” proven in-fleet, not just claimed. Structurally clean (1199 tensors, 15 mtp in shard 18, 333 visual). โš  MTP/speed figures are **GGUF/llama.cpp on a 5090, not vLLM** โ€” may not transfer; and because its MTP head was likely *trained*, the free CPU-hash shortcut would NOT apply (it won't match base) so a real acceptance gate would be needed. + - **OPEN FOLLOW-UPS (parked):** move gen seat off pinned-nightly to stable once #51113 ships; Lobe one-time TTS UI pass; delete the 1.8GB litellm dump; `harden-esh-docker-vm` (park id 28, PROMOTED โ€” Tier-1 done, `/mnt/books` stays hard w/ watchdog); chatterbox-fast build-context divergence; #363 research-wing ingest (no deadline); optionally attach our MTP reproducer to vllm#47087 (needs a GitHub identity โ€” operator's call). - **althing monitor** ARMED (handle `infra-ops`). โš ๏ธ Re-arm ONLY after a real FIRE (rc0), never after a plain operator turn (bounces rc3); spawn `althing-wake-listener` as its OWN `run_in_background` task, never chained with `&` (orphans it โ€” hit this twice 2026-08-17, `stop-monitor` reclaims).