feat(char-rp-gguf): swap reasoning seat Deckard-PKD → NEO-CODE (Heretic2-Thinking Qwen3.6-27B)
R36 gate (2026-07-14) validated NEO-CODE ships on all axes: tool-calling 0.967 (attach_tool 1.00, 0 runaways — #355 eliminated), prose genre-artifact-fine (less clichéd than gen), refusal uncensored-as-spec + CSAM-clean. #355 root cause was MODEL-level, not the reasoning-budget-forcing bug: Deckard emitted Qwen's native qwen3_coder XML tool format malformed -> llama.cpp leaked the closing tags into the arg value -> Bifrost attach_tool schema error -> retry -> reasoning runaway to max_tokens. NEO-CODE emits the same native format cleanly on the same seat/parser -> no schema error -> no runaway. The fix was the model swap; there was never a wrong parser (the XML is Qwen3.5/3.6-native). - reasoning seat: Deckard-PKD (Qwen3.5) -> NEO-CODE=Heretic2-Thinking (Qwen3.6-27B) Q5 - samplers: card defaults (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0), DRY dropped - ctx: 256K max; custom llama.cpp kept (qwen3_coder parse + PR#25544 belt-and-suspenders) - persistent-memory ACTIVE 1 marked resolved
This commit is contained in:
@@ -94,13 +94,14 @@ services:
|
||||
- homepage.description=Dark-romantasy RP prose seat, non-thinking (llama.cpp, ana-ml2 GPU 0)
|
||||
- homepage.href=http://10.250.50.54:${CHARRP_PORT:-8016}
|
||||
|
||||
# ── REASONING seat — Deckard-PKD (Qwen3.5) managed thinking + DRY. gateway char-rp-reasoning. ──
|
||||
# ── REASONING seat — NEO-CODE = Heretic2-Thinking (Qwen3.6-27B) managed thinking. gateway char-rp-reasoning. ──
|
||||
llama-charrp-reasoning:
|
||||
# ⚠️ CUSTOM llama.cpp build (master 6eddde0 + unmerged PR #25544) — the Worldtree
|
||||
# #355 reasoning-budget-forcing fix (multi-terminator: handles Qwen3.5's <tool_call>
|
||||
# reasoning end-tag, which stock b8840's single-</think> budget can't match → the seat
|
||||
# loops in reasoning to max_tokens ≈ 22 min). DO NOT revert to a stock image until
|
||||
# #25544 merges upstream. Build recipe + why + rollback: ./llamacpp-custom/README.md.
|
||||
# ⚠️ CUSTOM llama.cpp build (master 6eddde0 + unmerged PR #25544). Needed for two reasons:
|
||||
# (1) recent master parses Qwen3.6's native qwen3_coder tool-call format (stock b8840 predates it —
|
||||
# the <tool_call><function=..><parameter=..> XML is Qwen3.5/3.6-native, NOT an OpenHands quirk);
|
||||
# (2) PR #25544 multi-terminator reasoning-budget fix (Worldtree #355) — belt-and-suspenders now that
|
||||
# NEO-CODE shows 0.0 runaway (R36 gate), but keep it. DO NOT revert to stock until #25544 merges.
|
||||
# Build recipe + why + rollback: ./llamacpp-custom/README.md.
|
||||
# Rollback: set LLAMA_REASONING_IMAGE=ghcr.io/mostlygeek/llama-swap:cuda in .env + recreate.
|
||||
image: ${LLAMA_REASONING_IMAGE:-llamacpp-charrp:custom-latest}
|
||||
container_name: ${CHARRP_REASONING_CONTAINER:-llama-charrp-reasoning}
|
||||
@@ -119,7 +120,7 @@ services:
|
||||
entrypoint: ["/app/llama-server"]
|
||||
command:
|
||||
- --model
|
||||
- /models/${CHARRP_REASONING_MODEL:-rp/Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-Thinking.i1-Q5_K_M.gguf}
|
||||
- /models/${CHARRP_REASONING_MODEL:-rp/Qwen3.6-27B-NEO-CODE-HERE-2T-OT-Q5_K_M.gguf}
|
||||
- --host
|
||||
- 0.0.0.0
|
||||
- --port
|
||||
@@ -130,33 +131,35 @@ services:
|
||||
- "${CHARRP_REASONING_CTX:-40960}"
|
||||
- --flash-attn
|
||||
- on
|
||||
# Deckard = Qwen3.5-27B GDN-hybrid (16 KV-caching layers → KV cheap); native ctx 262144
|
||||
# (256K, no YaRN). Full 256K @ q8_0 KV ≈ 8.5G. q8_0 verified coherent; f16 in .env if gibberish.
|
||||
# NEO-CODE = Qwen3.6-27B GDN-hybrid (16 of 64 layers cache KV → KV cheap); native ctx 262144
|
||||
# (256K). Full 256K @ q8_0 KV ≈ 8.6G, fits GPU0 w/ ~3.8G margin. q8_0 coherent; f16 in .env if gibberish.
|
||||
- --cache-type-k
|
||||
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
|
||||
- --cache-type-v
|
||||
- ${CHARRP_REASONING_KV_TYPE:-q8_0}
|
||||
- --jinja
|
||||
# Deckard's Qwen3.5 template natively opens <think> + has enable_thinking → llama.cpp
|
||||
# manages the reasoning (trace to reasoning_content, content stays clean prose);
|
||||
# --reasoning-budget caps the CoT. (A/B 2026-07-08: Deckard 0/30 loops + 0/30 refusals,
|
||||
# beat RpR-v4 (1/30 loop), Pantheon (7/30 refusals), Snowdrop + Gembrain (template-incompat).)
|
||||
# NEO-CODE's Qwen3.6 template natively opens <think> → llama.cpp manages the reasoning
|
||||
# (trace to reasoning_content, content stays clean prose); --reasoning-budget caps the CoT.
|
||||
# (R36 gate 2026-07-14: NEO-CODE composite 0.922 tool-calling + 0.0 runaway — beat Deckard
|
||||
# 0.08/0.80 and gen-reasoning 0.856. Budget held at 400: latency-coupled to soong's client timeout.)
|
||||
- --reasoning
|
||||
- on
|
||||
- --reasoning-format
|
||||
- deepseek
|
||||
- --reasoning-budget
|
||||
- "${CHARRP_REASONING_BUDGET:-400}"
|
||||
# DRY anti-repetition (Deckard/Qwen3.5 tolerates it; sampler ORDER = dry AFTER temperature,
|
||||
# per the QwQ-family finding that a repetition PENALTY worsens looping but DRY fixes it).
|
||||
- --samplers
|
||||
- "top_k;top_p;min_p;temperature;dry"
|
||||
- --dry-multiplier
|
||||
- "${CHARRP_REASONING_DRY:-0.8}"
|
||||
- --dry-base
|
||||
- "1.75"
|
||||
- --dry-allowed-length
|
||||
- "2"
|
||||
# Sampler defaults per the DavidAU/Qwen3.6 model card (thinking-mode, general tasks): temp 1.0,
|
||||
# top_p 0.95, top_k 20, min_p 0.0, no rep-penalty, no DRY (DRY was a QwQ/Deckard looping band-aid
|
||||
# NEO-CODE doesn't need). All tunable via .env. NOTE: 0.922 tool-gate was on the OLD Deckard
|
||||
# samplers (effective temp~0.8 + DRY); re-validate tools + slop on these card samplers.
|
||||
- --temp
|
||||
- "${CHARRP_REASONING_TEMP:-1.0}"
|
||||
- --top-p
|
||||
- "${CHARRP_REASONING_TOP_P:-0.95}"
|
||||
- --top-k
|
||||
- "${CHARRP_REASONING_TOP_K:-20}"
|
||||
- --min-p
|
||||
- "${CHARRP_REASONING_MIN_P:-0.0}"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health >/dev/null || exit 1"]
|
||||
interval: 30s
|
||||
|
||||
Reference in New Issue
Block a user