feat(char-rp-reasoning): Deckard-PKD (Qwen3.5) replaces RpR-v4 after autonomous A/B
Operator wanted a reasoning-RP model that tolerates DRY (RpR-v4 forbids rep/DRY -> a 1/30 loop tail). Ran the full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored): - Deckard-PKD (Qwen3.5-27B, DavidAU creative tune) WON: 0/30 loops, 0/30 refusals, clean managed reasoning (native Qwen3.5 <think>/enable_thinking), DRY-tolerant, ~57 tok/s, runs on the base llama-swap b8840 image. -> now the char-rp-reasoning seat (:8018). - RpR-v4: 0 refusals but 1/30 loop (no-DRY). Pantheon-27B: clean slop but 7/30 explicit refusals + needs the newer ggml-org/llama.cpp image (Qwen3.6 won't load on b8840). Snowdrop + Gembrain (Gemma-4): floored (llama.cpp can't manage their reasoning without the vetoed template hacks). Losers kept on disk as alternates. - char-rp (Magidonia) unchanged; gen unchanged. gateway char-rp-reasoning -> Deckard sampler (temp 1.0/top_p 0.95/top_k 40/min_p 0.05; DRY server-side).
This commit is contained in:
+17
-5
@@ -120,13 +120,15 @@ _As of 2026-07-08 — OFF-THE-SHELF INFERENCE STACK is the active work (home-tra
|
||||
elite literary prose, zero refusal on dark/explicit scenes, ~65 tok/s, tight POV/instruction adherence
|
||||
(live-tested). Replaced the broken Angel NVFP4. Alt prose model (`.env` swap `CHARRP_MODEL`):
|
||||
`MS3.2-PaintedFantasy-v4.1-24B` (more literary flair, looser POV). Both GGUFs pre-pulled at `/tank/aimodels/llm/rp/`.
|
||||
- **char-rp-reasoning (:8018) = `QwQ-32B-ArliAI-RpR-v4-Q5_K_M` GGUF — LIVE.** QwQ reasoning RP tune (container
|
||||
`llama-charrp-reasoning`). `--reasoning on` → managed CoT SURFACES in `reasoning_content`, content stays clean
|
||||
prose, `--reasoning-budget 400` caps it. Reasoning data is QwQ-ABLITERATED → **no re-censor in `<think>`** (the
|
||||
Pantheon failure mode). ~52 tok/s @ Q5 (46 @ Q6). Best-of-breed-per-seat (NOT the same model as char-rp).
|
||||
- **char-rp-reasoning (:8018) = `Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M` GGUF — LIVE
|
||||
(A/B WINNER 2026-07-08, REPLACED RpR-v4).** DavidAU creative (PKD) tune on Qwen3.5-27B (container
|
||||
`llama-charrp-reasoning`). Native Qwen3.5 template → `--reasoning on` managed CoT SURFACES in `reasoning_content`,
|
||||
content stays clean prose, `--reasoning-budget 400` caps it; **DRY server-side** (sampler order = dry-after-temp)
|
||||
tames looping. A/B: **0/30 loops + 0/30 refusals**, ~57 tok/s. Runs on the base llama-swap b8840 image (Qwen3.5).
|
||||
Best-of-breed-per-seat (NOT the same model as char-rp).
|
||||
|
||||
- **🦄 RP-SEAT UNICORN — RESOLVED + DEPLOYED (2026-07-08).** char-rp = **Magidonia-24B-v4.3** (Magistral prose,
|
||||
65tps); char-rp-reasoning = **QwQ-32B-ArliAI-RpR-v4** (abliterated managed reasoning, 52tps @ Q5). Both GGUF via
|
||||
65tps); char-rp-reasoning = **Deckard-PKD (Qwen3.5-27B)** (managed reasoning + DRY, ~57tps; A/B winner over RpR-v4 — see re-A/B note below). Both GGUF via
|
||||
llama.cpp (`char-rp-gguf` stack, ana-ml2 GPU0, ~86/97G co-resident with gen, ~11G margin). Canonical stack in repo
|
||||
`stacks/char-rp-gguf/`; gateway rewired (`char-rp`→:8016, `char-rp-reasoning`→:8018, Mistral/QwQ samplers, dropped
|
||||
the Qwen `enable_thinking` kwarg). **KEY FINDINGS:** (a) no single dense 24-32B is BOTH an elite non-thinking prose
|
||||
@@ -138,6 +140,16 @@ _As of 2026-07-08 — OFF-THE-SHELF INFERENCE STACK is the active work (home-tra
|
||||
old AEON trace-not-surfacing gap). One-model fallback (Magidonia both, lighter reasoning) documented in the stack
|
||||
header/README. Requirements met: prose#1, ≥50tps#2, low-refusal#3, dense#4, GGUF-not-Ollama#5, fit-GPU0#6, thinking#7.
|
||||
Candidate GGUFs also on disk for A/B: Cydonia-R1-24B-v4.1, PaintedFantasy-v4.1-24B, RpR-v4 Q6_K.
|
||||
**REASONING-SEAT RE-A/B (2026-07-08, later — RpR→Deckard):** operator wanted a reasoning model that TAKES DRY
|
||||
(RpR forbids rep/DRY → 1/30 loop tail). Full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored):
|
||||
**Deckard-PKD (Qwen3.5-27B) WON** (0/30 loops, 0/30 refusals, clean) → NOW the char-rp-reasoning seat.
|
||||
RpR-v4: 0 refusals but 1/30 loop (no-DRY). **Pantheon-Reasoning-27B: 7/30 explicit refusals** (DeepSeek-distilled
|
||||
re-censor IS real across the batch, milder than feared; NOT wholesale-rejected) — kept on disk as alt.
|
||||
Snowdrop-v0.5-Type-S + Gembrain-31B (Gemma-4): FLOORED — llama.cpp can't do MANAGED reasoning on them (Snowdrop's
|
||||
ChatML template has no `<think>`/`enable_thinking` hook; Gemma-4's reasoning parser splits content wrong). GATE for a
|
||||
llama.cpp reasoning seat = STOCK template natively opens `<think>` or has `enable_thinking` (Qwen3.x/QwQ do; ChatML +
|
||||
Gemma-4 don't). **INFRA:** llama-swap b8840 image can't load Qwen3.6/Gemma-4 archs → use
|
||||
`ghcr.io/ggml-org/llama.cpp:server-cuda` (newer, pulled on ana-ml2) for those; Deckard (Qwen3.5) runs on b8840.
|
||||
**MAX CONTEXT (2026-07-08):** char-rp **128K** (Magidonia FULL native 131072), char-rp-reasoning **40K** (QwQ
|
||||
native 40960, YaRN-free max), **q8_0 KV cache both** (near-lossless, ~2× ctx/GB; verified coherent, no Qwen
|
||||
gibberish). **Funded by gen util 0.40→0.37** (freed ~2.9G of gen's IDLE KV headroom — gen KV usage runs 0-2%,
|
||||
|
||||
Reference in New Issue
Block a user