feat(char-rp-gguf): swap reasoning seat Deckard-PKD → NEO-CODE (Heretic2-Thinking Qwen3.6-27B)

R36 gate (2026-07-14) validated NEO-CODE ships on all axes: tool-calling 0.967
(attach_tool 1.00, 0 runaways — #355 eliminated), prose genre-artifact-fine
(less clichéd than gen), refusal uncensored-as-spec + CSAM-clean.

#355 root cause was MODEL-level, not the reasoning-budget-forcing bug: Deckard
emitted Qwen's native qwen3_coder XML tool format malformed -> llama.cpp leaked
the closing tags into the arg value -> Bifrost attach_tool schema error -> retry
-> reasoning runaway to max_tokens. NEO-CODE emits the same native format cleanly
on the same seat/parser -> no schema error -> no runaway. The fix was the model
swap; there was never a wrong parser (the XML is Qwen3.5/3.6-native).

- reasoning seat: Deckard-PKD (Qwen3.5) -> NEO-CODE=Heretic2-Thinking (Qwen3.6-27B) Q5
- samplers: card defaults (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0), DRY dropped
- ctx: 256K max; custom llama.cpp kept (qwen3_coder parse + PR#25544 belt-and-suspenders)
- persistent-memory ACTIVE 1 marked resolved
This commit is contained in:
vh
2026-07-13 22:21:10 -07:00
parent e0f1dbfae6
commit f960a73a79
2 changed files with 34 additions and 25 deletions
+26 -23
View File
@@ -94,13 +94,14 @@ services:
- homepage.description=Dark-romantasy RP prose seat, non-thinking (llama.cpp, ana-ml2 GPU 0)
- homepage.href=http://10.250.50.54:${CHARRP_PORT:-8016}
# ── REASONING seat — Deckard-PKD (Qwen3.5) managed thinking + DRY. gateway char-rp-reasoning. ──
# ── REASONING seat — NEO-CODE = Heretic2-Thinking (Qwen3.6-27B) managed thinking. gateway char-rp-reasoning. ──
llama-charrp-reasoning:
# ⚠️ CUSTOM llama.cpp build (master 6eddde0 + unmerged PR #25544) — the Worldtree
# #355 reasoning-budget-forcing fix (multi-terminator: handles Qwen3.5's <tool_call>
# reasoning end-tag, which stock b8840's single-</think> budget can't match → the seat
# loops in reasoning to max_tokens ≈ 22 min). DO NOT revert to a stock image until
# #25544 merges upstream. Build recipe + why + rollback: ./llamacpp-custom/README.md.
# ⚠️ CUSTOM llama.cpp build (master 6eddde0 + unmerged PR #25544). Needed for two reasons:
# (1) recent master parses Qwen3.6's native qwen3_coder tool-call format (stock b8840 predates it —
# the <tool_call><function=..><parameter=..> XML is Qwen3.5/3.6-native, NOT an OpenHands quirk);
# (2) PR #25544 multi-terminator reasoning-budget fix (Worldtree #355) — belt-and-suspenders now that
# NEO-CODE shows 0.0 runaway (R36 gate), but keep it. DO NOT revert to stock until #25544 merges.
# Build recipe + why + rollback: ./llamacpp-custom/README.md.
# Rollback: set LLAMA_REASONING_IMAGE=ghcr.io/mostlygeek/llama-swap:cuda in .env + recreate.
image: ${LLAMA_REASONING_IMAGE:-llamacpp-charrp:custom-latest}
container_name: ${CHARRP_REASONING_CONTAINER:-llama-charrp-reasoning}
@@ -119,7 +120,7 @@ services:
entrypoint: ["/app/llama-server"]
command:
- --model
- /models/${CHARRP_REASONING_MODEL:-rp/Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M.gguf}
- /models/${CHARRP_REASONING_MODEL:-rp/Qwen3.6-27B-NEO-CODE-HERE-2T-OT-Q5_K_M.gguf}
- --host
- 0.0.0.0
- --port
@@ -130,33 +131,35 @@ services:
- "${CHARRP_REASONING_CTX:-40960}"
- --flash-attn
- on
# Deckard = Qwen3.5-27B GDN-hybrid (16 KV-caching layers → KV cheap); native ctx 262144
# (256K, no YaRN). Full 256K @ q8_0 KV ≈ 8.5G. q8_0 verified coherent; f16 in .env if gibberish.
# NEO-CODE = Qwen3.6-27B GDN-hybrid (16 of 64 layers cache KV → KV cheap); native ctx 262144
# (256K). Full 256K @ q8_0 KV ≈ 8.6G, fits GPU0 w/ ~3.8G margin. q8_0 coherent; f16 in .env if gibberish.
- --cache-type-k
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
- --cache-type-v
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
- --jinja
# Deckard's Qwen3.5 template natively opens <think> + has enable_thinking → llama.cpp
# manages the reasoning (trace to reasoning_content, content stays clean prose);
# --reasoning-budget caps the CoT. (A/B 2026-07-08: Deckard 0/30 loops + 0/30 refusals,
# beat RpR-v4 (1/30 loop), Pantheon (7/30 refusals), Snowdrop + Gembrain (template-incompat).)
# NEO-CODE's Qwen3.6 template natively opens <think> → llama.cpp manages the reasoning
# (trace to reasoning_content, content stays clean prose); --reasoning-budget caps the CoT.
# (R36 gate 2026-07-14: NEO-CODE composite 0.922 tool-calling + 0.0 runaway — beat Deckard
# 0.08/0.80 and gen-reasoning 0.856. Budget held at 400: latency-coupled to soong's client timeout.)
- --reasoning
- on
- --reasoning-format
- deepseek
- --reasoning-budget
- "${CHARRP_REASONING_BUDGET:-400}"
# DRY anti-repetition (Deckard/Qwen3.5 tolerates it; sampler ORDER = dry AFTER temperature,
# per the QwQ-family finding that a repetition PENALTY worsens looping but DRY fixes it).
- --samplers
- "top_k;top_p;min_p;temperature;dry"
- --dry-multiplier
- "${CHARRP_REASONING_DRY:-0.8}"
- --dry-base
- "1.75"
- --dry-allowed-length
- "2"
# Sampler defaults per the DavidAU/Qwen3.6 model card (thinking-mode, general tasks): temp 1.0,
# top_p 0.95, top_k 20, min_p 0.0, no rep-penalty, no DRY (DRY was a QwQ/Deckard looping band-aid
# NEO-CODE doesn't need). All tunable via .env. NOTE: 0.922 tool-gate was on the OLD Deckard
# samplers (effective temp~0.8 + DRY); re-validate tools + slop on these card samplers.
- --temp
- "${CHARRP_REASONING_TEMP:-1.0}"
- --top-p
- "${CHARRP_REASONING_TOP_P:-0.95}"
- --top-k
- "${CHARRP_REASONING_TOP_K:-20}"
- --min-p
- "${CHARRP_REASONING_MIN_P:-0.0}"
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health >/dev/null || exit 1"]
interval: 30s