diff --git a/persistent-memory.md b/persistent-memory.md index 66c01a5..fea11c1 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -122,6 +122,8 @@ _As of 2026-08-21 00:35 β€” **the Heretic-300 session, and its reversal** (see t - **🟒 HOMEPAGE β€” cleaned + themed (Australis Skyfall).** Three real defects fixed (UltraSeedbox on all tabs, Uptime Kuma double-rendered, fiction column counts), AI tab reordered by clickability, then themed from the operator's Skyfall handoff bundle with a background generated by **Arbo** (`t2i-ui-background`, job `13f0891f4e42`). ⚠ **After any recreate the tab bar/wallpaper/i18n vanish for up to ~an hour and then return on their own β€” do not chase it.** ⚠ CSS is served per-request: a theme change needs a **reload**, not a recreate, and candidate CSS can be injected live via Playwright for seconds-long iteration. β†’ `persistent-memory.d/2026-08-19-homepage-skyfall-theme.md` +- **β›” RESULT 2026-08-21 β€” the gemma4 CoT test on a STABLE (v0.27.1) is BLOCKED by a config incompatibility, NOT the parser.** Tried serving the MeroMero NVFP4A16 quant on `vllm/vllm-openai:v0.27.1`. Two-stage failure: (1) v0.27.1's stricter transformers raised `AmbiguousGlobalPerLayerAttributeError: 'head_dim' is per-layer` on the Gemma-4 config; setting `allow_global_per_layer_attribute_access:true` on `text_config` downgraded it to a warning BUT (2) then `gemma4.py load_weights` crashed with **`AssertionError: load weight (512) into parameter (256)`** β€” **Gemma-4-31B is genuinely HETEROGENEOUS (some layers head_dim 512, not a uniform 256)**, so forcing the global value built wrong-shaped params. The transformers guard was CORRECT; there is no safe override. **The MeroMero quant's config was authored for v0.24.0's Gemma4 loader and cannot load on v0.27.x without a config migration (proper per_layer_config) or a re-quant against the newer transformers.** ⚠ **This also means the eventual gen-seat move to v0.27.2 stable must re-verify any Gemma-4 seat's config-compat** β€” the transformers heterogeneity change affects all Gemma-4 quants of this vintage. **FULLY REVERTED:** config.json restored (flags removed), compose + image back to `latest` (v0.24.0), gateway char-rp-reasoning removed, char-rp prose verified on v0.24.0. Net: char-rp stays pinned to v0.24.0; MeroMero CoT remains undelivered. **The per-request-kwargs hypothesis was never even reachable** β€” couldn't load the model to test it. **For RP-with-CoT: gen-reasoning (works now) or a re-quant of MeroMero against v0.27.x transformers (real work, unproven payoff).** + - **⚠️ CORRECTED 2026-08-21 β€” MeroMero-v2 CoT: NOT a hard wall, and NOT MeroMero-specific. My first conclusion ("gemma4 parser is process-wide") was WRONG.** Read the actual code, not the stale compose comment. **The real mechanism (gemma4-GENERAL, applies to any gemma4 finetune on this template family):** thinking is a **per-request** template toggle β€” `chat_template.jinja:347-352` emits the generation prompt `<|turn>model\n`, and **only when `enable_thinking` is false** does it prefill an empty `<|channel>thought\n` to SUPPRESS thinking; `enable_thinking:true` omits the prefill so the model is free to open a real `<|channel>thought…` block. The vLLM parser (`vllm/reasoning/gemma4_utils.py:parse_thinking_output`) **splits on `<|channel>`/`` tag PRESENCE β€” "works with or without enable_thinking," NOT a process-wide flag.** The stale compose comment I trusted cited an OLD parser API (`vllm/parser/gemma4.py:439`) that this container does not run. **So there is no architectural blocker; the two-served-name gen pattern SHOULD work.** **What actually failed my test:** meromero runs `vllm/vllm-openai:latest` (v0.24.0); per-request `chat_template_kwargs.enable_thinking:true` produced no thinking on it, whereas the **gen seat's pinned nightly demonstrably applies per-request `chat_template_kwargs`** (gen-reasoning works). So the practical block is a **vLLM-version / per-request-plumbing issue on v0.24.0**, not the model and not the architecture β€” and it would hit ANY gemma4 finetune served on that image the same way. **UNVERIFIED FIX (needs a GPU window): re-serve meromero on the nightly image + no process default + per-request enable_thinking; likely yields clean split CoT.** Currently REVERTED to known-good (char-rp prose, process default false, single served-name). ⚠ Kept `MEROMERO_GPU_MEM_UTIL` 0.52β†’0.51 (0.52 no longer boots next to the bigger orcarouter gen; free 49.02 < 49.38 GiB; 0.51 = KV 2.00Γ— @ 262K). - **🟒 PEN-TEST SEAT β€” M.O.G.-SEC LIVE on ana-ml2 GPU1 :8019 (2026-08-21 ~02:35, autonomous overnight run under operator's "full autonomy" grant).** `stacks/mog-sec` serves `/tank/aimodels/mog-sec-27b-nvfp4-mixed` (23.4 GB, our mixed NVFP4+FP8 quant of `Blackfrost-Research/M.O.G.-SEC-27B-1M-CTX-BF16`, rev `deede677`) in the **retired fablefusion-charrp-probe slot**. Aliases: **`mog-sec`** (non-thinking, presence_penalty 0.0 for code) + **`mog-sec-reasoning`** (thinking); served-names `mog-sec-27b`/`-thinking`. **Gates all pass:** KV **418,218 tok / 1.60Γ— @ 262K**; surface **6/6**; MTP **55.3% @ 111.7 tok/s**; format screen **1.11e-05** (rank 196, no `` leak); **vision 7/3/1**; **capability 4/4** (SQL-inj exfil, pentest phasing, port scanner, Kerberoasting β€” it delivers, doesn't refuse). **β˜… Chose the BF16 + self-quant over the pre-made ModelOpt NVFP4**, which was disqualified on W4A4 (4-bit activations, AEON mode) + zero MTP + the self-defeating W4A4-on-1M-context. **⚠ CONTEXT IS 262K NATIVE, NOT the card's 1M** β€” 1M needs YaRN (absent from config) + the SGLang/DFlash2 path the repo ships, neither our vLLM surface; a real 1M seat is a separate SGLang project. **MTP head is a base graft** (identical to stock), not in-band abliterated like orcarouter's. Provenance `mog-sec-27b-nvfp4-mixed.PROVENANCE.txt`; eval `docs/pfi/gen-seat-candidate-eval-2026-08-21.md`. **RETIRED with it:** `char-rp-reasoning` + `char-rp-fable` LiteLLM aliases (zero traffic, pointed at the dead fable :8019; now 404). RP-reasoning's real home is `darkscarlett-charrp-reasoning` (:8018, compose-down, weights intact) if ever wanted. ⚠ brand-new model (0 downloads at pull, org-name mismatch Research-vs-AI) β€” card claims are claims.