docs(char-rp-reasoning): Deckard samplers dvalin-confirmed canonical + tuning ladder
dvalin confirmed the live A/B-proven set IS canonical for Deckard as a dark-RP reasoning seat: temp 1.0/top_p 0.95/top_k 40/min_p 0.05, no presence/rep penalty, DRY 0.8 server-side. Endorsed over the card's base-thinking (top_k 20/min_p 0/presence 1.5). No value change; comment + memory record the confirmation + tuning ladder (flat->min_p 0.08, loops->DRY 0.9, over-damped->DRY 0.6/off).
This commit is contained in:
@@ -157,9 +157,11 @@ _As of 2026-07-08 — OFF-THE-SHELF INFERENCE STACK is the active work (home-tra
|
|||||||
KV is TINY (1.4G @ 40K vs Magidonia's 10.9G @ 128K, 40 dense layers); Deckard native ctx = 256K, so the reasoning
|
KV is TINY (1.4G @ 40K vs Magidonia's 10.9G @ 128K, 40 dense layers); Deckard native ctx = 256K, so the reasoning
|
||||||
seat's ctx scales nearly free. **Deckard BUMPED to full 256K (262144) 2026-07-08** (KV 8.7G q8_0). GPU0 layout now:
|
seat's ctx scales nearly free. **Deckard BUMPED to full 256K (262144) 2026-07-08** (KV 8.7G q8_0). GPU0 layout now:
|
||||||
gen ~35G (256K/16-seq/util0.37), char-rp Magidonia ~30G (128K/4-slot), char-rp-reasoning Deckard ~28G (256K/4-slot),
|
gen ~35G (256K/16-seq/util0.37), char-rp Magidonia ~30G (128K/4-slot), char-rp-reasoning Deckard ~28G (256K/4-slot),
|
||||||
**~4G free** (all static allocations → stable, no OOM). **Canonical Deckard RP-reasoning samplers PENDING dvalin**
|
**~4G free** (all static allocations → stable, no OOM). **Canonical Deckard RP-reasoning samplers CONFIRMED by dvalin 2026-07-08**
|
||||||
(card has only generic base-Qwen3.5 profiles, no RP/DRY canon); running empirical temp1.0/top_p0.95/top_k40/min_p0.05
|
(thread 01KX1Y7P): the live empirical set IS canonical — temp1.0/top_p0.95/top_k40/min_p0.05, NO presence/rep
|
||||||
+ DRY0.8 (A/B-proven: 0 loops/0 refusals) until dvalin confirms — thread 01KX1Y7P.
|
penalty, DRY0.8 server-side (base1.75/len2, dry-after-temp), reasoning-budget400. Delta from card base-thinking:
|
||||||
|
top_k20→40, min_p0→0.05, presence1.5→0, DRY-on. Tuning ladder: flat prose→min_p0.08; loops→DRY0.9; over-damped→
|
||||||
|
DRY0.6/off. Do NOT import RpR/QwQ rules (QwQ hated DRY; Qwen3.5 benefits from DRY over rep_penalty).
|
||||||
**MAX CONTEXT (2026-07-08):** char-rp **128K** (Magidonia FULL native 131072), char-rp-reasoning **40K** (QwQ
|
**MAX CONTEXT (2026-07-08):** char-rp **128K** (Magidonia FULL native 131072), char-rp-reasoning **40K** (QwQ
|
||||||
native 40960, YaRN-free max), **q8_0 KV cache both** (near-lossless, ~2× ctx/GB; verified coherent, no Qwen
|
native 40960, YaRN-free max), **q8_0 KV cache both** (near-lossless, ~2× ctx/GB; verified coherent, no Qwen
|
||||||
gibberish). **Funded by gen util 0.40→0.37** (freed ~2.9G of gen's IDLE KV headroom — gen KV usage runs 0-2%,
|
gibberish). **Funded by gen util 0.40→0.37** (freed ~2.9G of gen's IDLE KV headroom — gen KV usage runs 0-2%,
|
||||||
|
|||||||
@@ -178,8 +178,10 @@ model_list:
|
|||||||
# clean prose, budget-capped. DRY server-side (sampler order = dry after temperature) tames looping.
|
# clean prose, budget-capped. DRY server-side (sampler order = dry after temperature) tames looping.
|
||||||
# A/B WINNER 2026-07-08: 0/30 loops + 0/30 refusals; beat RpR-v4 (1/30 loop, forbids DRY),
|
# A/B WINNER 2026-07-08: 0/30 loops + 0/30 refusals; beat RpR-v4 (1/30 loop, forbids DRY),
|
||||||
# Pantheon-27B (7/30 explicit refusals), Snowdrop + Gembrain (llama.cpp template-incompat).
|
# Pantheon-27B (7/30 explicit refusals), Snowdrop + Gembrain (llama.cpp template-incompat).
|
||||||
# Deckard decode: temp 1.0, top_p 0.95, top_k 40, min_p 0.05. NOT the same model as char-rp
|
# Deckard decode: temp 1.0, top_p 0.95, top_k 40, min_p 0.05 (dvalin-CONFIRMED canonical 2026-07-08;
|
||||||
# (best-of-breed per seat) — see stacks/char-rp-gguf/README.md.
|
# NO presence/rep penalty; DRY 0.8 server-side). Tuning ladder: flat prose→min_p 0.08, loops→DRY 0.9,
|
||||||
|
# over-damped→DRY 0.6/off. Do NOT import RpR/QwQ sampler rules (different family). NOT the same model as
|
||||||
|
# char-rp (best-of-breed per seat) — see stacks/char-rp-gguf/README.md.
|
||||||
- model_name: char-rp-reasoning
|
- model_name: char-rp-reasoning
|
||||||
litellm_params:
|
litellm_params:
|
||||||
model: hosted_vllm/deckard-pkd-27b
|
model: hosted_vllm/deckard-pkd-27b
|
||||||
|
|||||||
Reference in New Issue
Block a user