feat(char-rp-reasoning): Deckard-PKD (Qwen3.5) replaces RpR-v4 after autonomous A/B
Operator wanted a reasoning-RP model that tolerates DRY (RpR-v4 forbids rep/DRY -> a 1/30 loop tail). Ran the full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored): - Deckard-PKD (Qwen3.5-27B, DavidAU creative tune) WON: 0/30 loops, 0/30 refusals, clean managed reasoning (native Qwen3.5 <think>/enable_thinking), DRY-tolerant, ~57 tok/s, runs on the base llama-swap b8840 image. -> now the char-rp-reasoning seat (:8018). - RpR-v4: 0 refusals but 1/30 loop (no-DRY). Pantheon-27B: clean slop but 7/30 explicit refusals + needs the newer ggml-org/llama.cpp image (Qwen3.6 won't load on b8840). Snowdrop + Gembrain (Gemma-4): floored (llama.cpp can't manage their reasoning without the vetoed template hacks). Losers kept on disk as alternates. - char-rp (Magidonia) unchanged; gen unchanged. gateway char-rp-reasoning -> Deckard sampler (temp 1.0/top_p 0.95/top_k 40/min_p 0.05; DRY server-side).
This commit is contained in:
@@ -35,21 +35,25 @@ CHARRP_KV_TYPE=q8_0
|
||||
# ── REASONING seat (char-rp-reasoning) ──────────────────────────────────────
|
||||
CHARRP_REASONING_CONTAINER=llama-charrp-reasoning
|
||||
CHARRP_REASONING_PORT=8018
|
||||
# Default = QwQ-32B-ArliAI-RpR-v4 Q5_K_M (abliterated reasoning → no re-censor;
|
||||
# llama.cpp-managed CoT). ~50 tok/s @ Q5. Use Q6_K (~46 tok/s) for a touch more
|
||||
# quality if speed is not binding.
|
||||
CHARRP_REASONING_MODEL=rp/QwQ-32B-ArliAI-RpR-v4-Q5_K_M.gguf
|
||||
# QwQ native ctx = 40960 (its max without YaRN). ~5.5G VRAM @ q8_0 KV.
|
||||
# Default = Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking i1-Q5_K_M (DavidAU creative
|
||||
# tune, native Qwen3.5 managed reasoning, DRY-tolerant). A/B WINNER 2026-07-08: 0/30 loops,
|
||||
# 0/30 refusals, clean slop; beat RpR-v4 (1/30 loop, no-DRY), Pantheon (7/30 refusals),
|
||||
# Snowdrop + Gembrain (template-incompatible with llama.cpp managed reasoning).
|
||||
CHARRP_REASONING_MODEL=rp/Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M.gguf
|
||||
# Qwen3.5-27B native ctx is large; 40960 = a sane reasoning-seat cap. ~5G VRAM @ q8_0 KV.
|
||||
CHARRP_REASONING_CTX=40960
|
||||
# KV cache dtype (Qwen-arch): q8_0 verified coherent here; f16 if a future model gibbers.
|
||||
# KV cache dtype: q8_0 verified coherent; f16 if a future model gibbers.
|
||||
CHARRP_REASONING_KV_TYPE=q8_0
|
||||
# Thinking-token cap (QwQ over-thinks otherwise → starves the prose). 300-500 = a
|
||||
# concise, useful scene-plan before the response.
|
||||
# Thinking-token cap (concise scene-plan before the response). 300-500 is a good band.
|
||||
CHARRP_REASONING_BUDGET=400
|
||||
# DRY anti-repetition multiplier (0 disables). Deckard tolerates DRY; DRY is the correct
|
||||
# anti-loop tool for the Qwen/QwQ family (a repetition PENALTY worsens their looping).
|
||||
CHARRP_REASONING_DRY=0.8
|
||||
|
||||
# ── ONE-MODEL FALLBACK (consistent Mistral style, lighter reasoning) ─────────
|
||||
# To collapse both seats onto Magidonia (drop QwQ): set
|
||||
# CHARRP_REASONING_MODEL=rp/TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf
|
||||
# and remove the --reasoning* flags from the reasoning service in compose.yaml
|
||||
# (Magistral reasons only when the caller's system prompt contains "/think";
|
||||
# managed but LIGHT — see the compose header for why QwQ is the default).
|
||||
# ── ALTERNATES / FALLBACKS (all pre-pulled to /tank/aimodels/llm/rp/) ────────
|
||||
# Reasoning-seat A/B losers, kept on disk: QwQ-32B-ArliAI-RpR-v4-Q5_K_M (no-DRY → 1/30 loop
|
||||
# tail); Gryphe_Pantheon-Reasoning-27B-Q5_K_M (7/30 explicit refusals; also needs the newer
|
||||
# ggml-org/llama.cpp:server-cuda image — Qwen3.6 won't load on llama-swap b8840);
|
||||
# Gemma-4-Gembrain-31B (Gemma-4 reasoning-parser broken on llama.cpp — floored).
|
||||
# One-model fallback (collapse the reasoning seat onto Magidonia, lighter /think reasoning):
|
||||
# CHARRP_REASONING_MODEL=rp/TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf + drop the DRY/--reasoning flags.
|
||||
|
||||
@@ -94,7 +94,7 @@ services:
|
||||
- homepage.description=Dark-romantasy RP prose seat, non-thinking (llama.cpp, ana-ml2 GPU 0)
|
||||
- homepage.href=http://10.250.50.54:${CHARRP_PORT:-8016}
|
||||
|
||||
# ── REASONING seat — QwQ managed thinking. gateway char-rp-reasoning. ──
|
||||
# ── REASONING seat — Deckard-PKD (Qwen3.5) managed thinking + DRY. gateway char-rp-reasoning. ──
|
||||
llama-charrp-reasoning:
|
||||
image: ${LLAMA_IMAGE:-ghcr.io/mostlygeek/llama-swap:cuda}
|
||||
container_name: ${CHARRP_REASONING_CONTAINER:-llama-charrp-reasoning}
|
||||
@@ -113,7 +113,7 @@ services:
|
||||
entrypoint: ["/app/llama-server"]
|
||||
command:
|
||||
- --model
|
||||
- /models/${CHARRP_REASONING_MODEL:-rp/QwQ-32B-ArliAI-RpR-v4-Q5_K_M.gguf}
|
||||
- /models/${CHARRP_REASONING_MODEL:-rp/Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M.gguf}
|
||||
- --host
|
||||
- 0.0.0.0
|
||||
- --port
|
||||
@@ -124,23 +124,33 @@ services:
|
||||
- "${CHARRP_REASONING_CTX:-40960}"
|
||||
- --flash-attn
|
||||
- on
|
||||
# QwQ native ctx = 40960 (beyond needs YaRN → quality loss; don't). Qwen-arch can be
|
||||
# KV-quant-sensitive: q8_0 is verified coherent here, but flip to f16 in .env if a
|
||||
# future model shows gibberish.
|
||||
# Deckard = Qwen3.5-27B (native ctx large); 40960 is a sane reasoning-seat cap. q8_0 KV
|
||||
# verified coherent; flip to f16 in .env if a future model shows gibberish.
|
||||
- --cache-type-k
|
||||
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
|
||||
- --cache-type-v
|
||||
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
|
||||
- --jinja
|
||||
# QwQ reasoning is template-native → llama.cpp manages it. --reasoning on surfaces
|
||||
# the trace in reasoning_content (content stays clean prose); --reasoning-budget
|
||||
# caps the CoT so it can't run away and starve the prose (QwQ over-thinks otherwise).
|
||||
# Deckard's Qwen3.5 template natively opens <think> + has enable_thinking → llama.cpp
|
||||
# manages the reasoning (trace to reasoning_content, content stays clean prose);
|
||||
# --reasoning-budget caps the CoT. (A/B 2026-07-08: Deckard 0/30 loops + 0/30 refusals,
|
||||
# beat RpR-v4 (1/30 loop), Pantheon (7/30 refusals), Snowdrop + Gembrain (template-incompat).)
|
||||
- --reasoning
|
||||
- on
|
||||
- --reasoning-format
|
||||
- deepseek
|
||||
- --reasoning-budget
|
||||
- "${CHARRP_REASONING_BUDGET:-400}"
|
||||
# DRY anti-repetition (Deckard/Qwen3.5 tolerates it; sampler ORDER = dry AFTER temperature,
|
||||
# per the QwQ-family finding that a repetition PENALTY worsens looping but DRY fixes it).
|
||||
- --samplers
|
||||
- "top_k;top_p;min_p;temperature;dry"
|
||||
- --dry-multiplier
|
||||
- "${CHARRP_REASONING_DRY:-0.8}"
|
||||
- --dry-base
|
||||
- "1.75"
|
||||
- --dry-allowed-length
|
||||
- "2"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health >/dev/null || exit 1"]
|
||||
interval: 30s
|
||||
|
||||
@@ -173,23 +173,23 @@ model_list:
|
||||
model_info:
|
||||
mode: chat
|
||||
# char-rp-reasoning -> GGUF managed-REASONING seat (:8018, llama.cpp, char-rp-gguf stack).
|
||||
# ArliAI QwQ-32B-ArliAI-RpR-v4 Q5_K_M — QwQ reasoning RP tune. Reasoning is ON server-side
|
||||
# (--reasoning on): the CoT SURFACES in reasoning_content and content stays clean prose
|
||||
# (fixes the old trace-not-surfacing gap), CoT budget-capped so it can't starve the prose.
|
||||
# Its reasoning data is QwQ-ABLITERATED → no re-censor inside <think> (the failure mode
|
||||
# that disqualified Pantheon-Reasoning-27B). ~52 tok/s @ Q5_K_M. RpR card: temp 1.0,
|
||||
# top_k 40, min_p 0.02, and NO repetition / DRY / XTC penalties. NOT the same model as
|
||||
# char-rp (best-of-breed per seat) — see stacks/char-rp-gguf/README.md.
|
||||
# Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking i1-Q5_K_M — DavidAU creative tune.
|
||||
# Reasoning ON server-side (--reasoning on): CoT surfaces in reasoning_content, content stays
|
||||
# clean prose, budget-capped. DRY server-side (sampler order = dry after temperature) tames looping.
|
||||
# A/B WINNER 2026-07-08: 0/30 loops + 0/30 refusals; beat RpR-v4 (1/30 loop, forbids DRY),
|
||||
# Pantheon-27B (7/30 explicit refusals), Snowdrop + Gembrain (llama.cpp template-incompat).
|
||||
# Deckard decode: temp 1.0, top_p 0.95, top_k 40, min_p 0.05. NOT the same model as char-rp
|
||||
# (best-of-breed per seat) — see stacks/char-rp-gguf/README.md.
|
||||
- model_name: char-rp-reasoning
|
||||
litellm_params:
|
||||
model: hosted_vllm/qwq-32b-rpr-v4
|
||||
model: hosted_vllm/deckard-pkd-27b
|
||||
api_base: http://10.250.50.54:8018/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
extra_body:
|
||||
top_k: 40
|
||||
min_p: 0.02
|
||||
min_p: 0.05
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
|
||||
Reference in New Issue
Block a user