feat(char-rp-reasoning): Deckard-PKD (Qwen3.5) replaces RpR-v4 after autonomous A/B

Operator wanted a reasoning-RP model that tolerates DRY (RpR-v4 forbids rep/DRY -> a
1/30 loop tail). Ran the full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored):

- Deckard-PKD (Qwen3.5-27B, DavidAU creative tune) WON: 0/30 loops, 0/30 refusals, clean
  managed reasoning (native Qwen3.5 <think>/enable_thinking), DRY-tolerant, ~57 tok/s,
  runs on the base llama-swap b8840 image. -> now the char-rp-reasoning seat (:8018).
- RpR-v4: 0 refusals but 1/30 loop (no-DRY). Pantheon-27B: clean slop but 7/30 explicit
  refusals + needs the newer ggml-org/llama.cpp image (Qwen3.6 won't load on b8840).
  Snowdrop + Gembrain (Gemma-4): floored (llama.cpp can't manage their reasoning without
  the vetoed template hacks). Losers kept on disk as alternates.
- char-rp (Magidonia) unchanged; gen unchanged. gateway char-rp-reasoning -> Deckard
  sampler (temp 1.0/top_p 0.95/top_k 40/min_p 0.05; DRY server-side).
This commit is contained in:
vh
2026-07-08 15:07:04 -07:00
parent f49c4e40a3
commit 5f79b40982
4 changed files with 62 additions and 36 deletions
+17 -5
View File
@@ -120,13 +120,15 @@ _As of 2026-07-08 — OFF-THE-SHELF INFERENCE STACK is the active work (home-tra
elite literary prose, zero refusal on dark/explicit scenes, ~65 tok/s, tight POV/instruction adherence
(live-tested). Replaced the broken Angel NVFP4. Alt prose model (`.env` swap `CHARRP_MODEL`):
`MS3.2-PaintedFantasy-v4.1-24B` (more literary flair, looser POV). Both GGUFs pre-pulled at `/tank/aimodels/llm/rp/`.
- **char-rp-reasoning (:8018) = `QwQ-32B-ArliAI-RpR-v4-Q5_K_M` GGUF — LIVE.** QwQ reasoning RP tune (container
`llama-charrp-reasoning`). `--reasoning on` → managed CoT SURFACES in `reasoning_content`, content stays clean
prose, `--reasoning-budget 400` caps it. Reasoning data is QwQ-ABLITERATED → **no re-censor in `<think>`** (the
Pantheon failure mode). ~52 tok/s @ Q5 (46 @ Q6). Best-of-breed-per-seat (NOT the same model as char-rp).
- **char-rp-reasoning (:8018) = `Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M` GGUF — LIVE
(A/B WINNER 2026-07-08, REPLACED RpR-v4).** DavidAU creative (PKD) tune on Qwen3.5-27B (container
`llama-charrp-reasoning`). Native Qwen3.5 template → `--reasoning on` managed CoT SURFACES in `reasoning_content`,
content stays clean prose, `--reasoning-budget 400` caps it; **DRY server-side** (sampler order = dry-after-temp)
tames looping. A/B: **0/30 loops + 0/30 refusals**, ~57 tok/s. Runs on the base llama-swap b8840 image (Qwen3.5).
Best-of-breed-per-seat (NOT the same model as char-rp).
- **🦄 RP-SEAT UNICORN — RESOLVED + DEPLOYED (2026-07-08).** char-rp = **Magidonia-24B-v4.3** (Magistral prose,
65tps); char-rp-reasoning = **QwQ-32B-ArliAI-RpR-v4** (abliterated managed reasoning, 52tps @ Q5). Both GGUF via
65tps); char-rp-reasoning = **Deckard-PKD (Qwen3.5-27B)** (managed reasoning + DRY, ~57tps; A/B winner over RpR-v4 — see re-A/B note below). Both GGUF via
llama.cpp (`char-rp-gguf` stack, ana-ml2 GPU0, ~86/97G co-resident with gen, ~11G margin). Canonical stack in repo
`stacks/char-rp-gguf/`; gateway rewired (`char-rp`→:8016, `char-rp-reasoning`→:8018, Mistral/QwQ samplers, dropped
the Qwen `enable_thinking` kwarg). **KEY FINDINGS:** (a) no single dense 24-32B is BOTH an elite non-thinking prose
@@ -138,6 +140,16 @@ _As of 2026-07-08 — OFF-THE-SHELF INFERENCE STACK is the active work (home-tra
old AEON trace-not-surfacing gap). One-model fallback (Magidonia both, lighter reasoning) documented in the stack
header/README. Requirements met: prose#1, ≥50tps#2, low-refusal#3, dense#4, GGUF-not-Ollama#5, fit-GPU0#6, thinking#7.
Candidate GGUFs also on disk for A/B: Cydonia-R1-24B-v4.1, PaintedFantasy-v4.1-24B, RpR-v4 Q6_K.
**REASONING-SEAT RE-A/B (2026-07-08, later — RpR→Deckard):** operator wanted a reasoning model that TAKES DRY
(RpR forbids rep/DRY → 1/30 loop tail). Full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored):
**Deckard-PKD (Qwen3.5-27B) WON** (0/30 loops, 0/30 refusals, clean) → NOW the char-rp-reasoning seat.
RpR-v4: 0 refusals but 1/30 loop (no-DRY). **Pantheon-Reasoning-27B: 7/30 explicit refusals** (DeepSeek-distilled
re-censor IS real across the batch, milder than feared; NOT wholesale-rejected) — kept on disk as alt.
Snowdrop-v0.5-Type-S + Gembrain-31B (Gemma-4): FLOORED — llama.cpp can't do MANAGED reasoning on them (Snowdrop's
ChatML template has no `<think>`/`enable_thinking` hook; Gemma-4's reasoning parser splits content wrong). GATE for a
llama.cpp reasoning seat = STOCK template natively opens `<think>` or has `enable_thinking` (Qwen3.x/QwQ do; ChatML +
Gemma-4 don't). **INFRA:** llama-swap b8840 image can't load Qwen3.6/Gemma-4 archs → use
`ghcr.io/ggml-org/llama.cpp:server-cuda` (newer, pulled on ana-ml2) for those; Deckard (Qwen3.5) runs on b8840.
**MAX CONTEXT (2026-07-08):** char-rp **128K** (Magidonia FULL native 131072), char-rp-reasoning **40K** (QwQ
native 40960, YaRN-free max), **q8_0 KV cache both** (near-lossless, ~2× ctx/GB; verified coherent, no Qwen
gibberish). **Funded by gen util 0.40→0.37** (freed ~2.9G of gen's IDLE KV headroom — gen KV usage runs 0-2%,