vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

config(char-rp-gguf): Deckard reasoning seat to full 256K (GDN-hybrid KV cheap)

Browse Source 
Deckard (Qwen3.5-27B) native ctx = 262144; GDN-hybrid arch (16 KV-caching layers) makes KV
cheap (8.7G q8_0 @ 256K vs Magidonia 10.9G @ 128K/40 dense layers). Bumped 40960 -> 262144;
GPU0 ~4G free (static -> stable). Canonical RP-reasoning samplers pending dvalin (card has
only generic base-Qwen3.5 profiles); empirical temp1.0/top_p0.95/top_k40/min_p0.05+DRY0.8 live.
This commit is contained in:
vh
2026-07-08 15:41:02 -07:00
parent 7a59de3afa
commit 41305bf62c
3 changed files with 10 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.md
+5 -2
View File
@@ -155,8 +155,11 @@ _As of 2026-07-08 — OFF-THE-SHELF INFERENCE STACK is the active work (home-tra
field — beats even Magidonia char-rp 1.774 — BUT 7/30 explicit refusals → rejected: clean-but-refuses loses to
clean-and-uncensored). **MEMORY-LAYOUT note:** Deckard (Qwen3.5-27B) is GDN-hybrid → only **16 KV-caching layers** →
KV is TINY (1.4G @ 40K vs Magidonia's 10.9G @ 128K, 40 dense layers); Deckard native ctx = 256K, so the reasoning
seat's ctx scales nearly free (40K→128K ≈ +3G, →256K ≈ +7G). GPU0 layout: gen ~35G (256K/16-seq/util0.37), char-rp
Magidonia ~30G (128K), char-rp-reasoning Deckard ~21G (40K), ~11.5G free.
seat's ctx scales nearly free. **Deckard BUMPED to full 256K (262144) 2026-07-08** (KV 8.7G q8_0). GPU0 layout now:
gen ~35G (256K/16-seq/util0.37), char-rp Magidonia ~30G (128K/4-slot), char-rp-reasoning Deckard ~28G (256K/4-slot),
**~4G free** (all static allocations → stable, no OOM). **Canonical Deckard RP-reasoning samplers PENDING dvalin**
(card has only generic base-Qwen3.5 profiles, no RP/DRY canon); running empirical temp1.0/top_p0.95/top_k40/min_p0.05
+ DRY0.8 (A/B-proven: 0 loops/0 refusals) until dvalin confirms — thread 01KX1Y7P.
**MAX CONTEXT (2026-07-08):** char-rp **128K** (Magidonia FULL native 131072), char-rp-reasoning **40K** (QwQ
native 40960, YaRN-free max), **q8_0 KV cache both** (near-lossless, ~2× ctx/GB; verified coherent, no Qwen
gibberish). **Funded by gen util 0.40→0.37** (freed ~2.9G of gen's IDLE KV headroom — gen KV usage runs 0-2%,