config(char-rp-gguf): Deckard reasoning seat to full 256K (GDN-hybrid KV cheap)

Deckard (Qwen3.5-27B) native ctx = 262144; GDN-hybrid arch (16 KV-caching layers) makes KV
cheap (8.7G q8_0 @ 256K vs Magidonia 10.9G @ 128K/40 dense layers). Bumped 40960 -> 262144;
GPU0 ~4G free (static -> stable). Canonical RP-reasoning samplers pending dvalin (card has
only generic base-Qwen3.5 profiles); empirical temp1.0/top_p0.95/top_k40/min_p0.05+DRY0.8 live.
This commit is contained in:
vh
2026-07-08 15:41:02 -07:00
parent 7a59de3afa
commit 41305bf62c
3 changed files with 10 additions and 6 deletions
+3 -2
View File
@@ -40,8 +40,9 @@ CHARRP_REASONING_PORT=8018
# 0/30 refusals, clean slop; beat RpR-v4 (1/30 loop, no-DRY), Pantheon (7/30 refusals),
# Snowdrop + Gembrain (template-incompatible with llama.cpp managed reasoning).
CHARRP_REASONING_MODEL=rp/Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M.gguf
# Qwen3.5-27B native ctx is large; 40960 = a sane reasoning-seat cap. ~5G VRAM @ q8_0 KV.
CHARRP_REASONING_CTX=40960
# Deckard = Qwen3.5-27B GDN-hybrid (only 16 KV-caching layers → KV is CHEAP). Native ctx 262144
# (256K, no YaRN). Full 256K @ q8_0 KV ≈ 8.5G VRAM (~0.033G/1K-tok, vs Magidonia ~0.085). Fits GPU0.
CHARRP_REASONING_CTX=262144
# KV cache dtype: q8_0 verified coherent; f16 if a future model gibbers.
CHARRP_REASONING_KV_TYPE=q8_0
# Thinking-token cap (concise scene-plan before the response). 300-500 is a good band.
+2 -2
View File
@@ -124,8 +124,8 @@ services:
- "${CHARRP_REASONING_CTX:-40960}"
- --flash-attn
- on
# Deckard = Qwen3.5-27B (native ctx large); 40960 is a sane reasoning-seat cap. q8_0 KV
# verified coherent; flip to f16 in .env if a future model shows gibberish.
# Deckard = Qwen3.5-27B GDN-hybrid (16 KV-caching layers → KV cheap); native ctx 262144
# (256K, no YaRN). Full 256K @ q8_0 KV ≈ 8.5G. q8_0 verified coherent; f16 in .env if gibberish.
- --cache-type-k
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
- --cache-type-v