Operator-directed 2026-07-16. Moved the char-rp prose seat (Magidonia-24B,
llama-charrp) from GPU0 to GPU1 (CHARRP_GPU_ID 0->1; recreate llama-charrp
only -- the var is shared with the retired GGUF reasoning service), then
re-optimized every context-relevant seat on both cards to native/max context
with acceptable headroom:
GPU0 (both seats now 256K native, ~14 GB reserve):
- char-rp-reasoning 150K -> 256K (heretic2 stack, util 0.38->0.46, 1.56x)
- gen 256K, max-num-seqs 16 -> 32 (qwen36-27b-aeon, util 0.30->0.42, 5.43x)
GPU1 (~6.7 GB headroom):
- granite 64K -> 128K full-chapter (vllm stack, util 0.18->0.27, 1.50x)
- char-rp 128K native (4 slots), selene/reward/embed/rerank unchanged
All seats gateway-verified healthy. Live .env changes on ana-ml2 with per-stack
backups (*-20260716). Templates updated to match; the qwen36-27b-aeon template
carries a NOTE that its served-name/model still lag the 2026-07-08 gen model swap
(35B-A3B-heretic) -- separate reconciliation. persistent-memory records the full
layout + the util-floor / per-model-KV-cost lessons.
Note: the heretic2-charrp-reasoning stack (char-rp-reasoning's live config) is
still untracked in git -- standing open-loop, its .env change lives server-side only.
Move the ~22-service flat "AI Systems" group off the Main tab into a new
four-tab layout (Main / AI / Infrastructure / Toolchain). The AI tab sorts
the inference fleet by function into seven groups:
AI - Inference gen, char-rp, char-rp-reasoning, Granite summarizer
AI - Eval & Retrieval Selene, Skywork Reward, Qwen3 rerank/embed, image-bench
AI - Gateways & Chat LiteLLM, Asset Engine, Gateway Chat, Open WebUI, ...
AI - Speech (TTS) Chatterbox Fast, Kokoro, mOrpheus
AI - Audio Tools Parakeet ASR, YT Voice Clipper
AI - Image & Media ComfyUI, Arbo
AI - Dormant stopped rollback seats + retired auditions
Relabel each stack's homepage.group so canonical stacks/ matches the live
containers on ana-ml2, ana-docker, and irv-ml1. Dormant stacks were refreshed
with `docker compose up --no-start` so they carry the new label while staying
stopped (compose-start rollback preserved). settings.yaml drives tab/order/
columns; services.yaml and README updated to the new scheme.
R36 gate (2026-07-14) validated NEO-CODE ships on all axes: tool-calling 0.967
(attach_tool 1.00, 0 runaways — #355 eliminated), prose genre-artifact-fine
(less clichéd than gen), refusal uncensored-as-spec + CSAM-clean.
#355 root cause was MODEL-level, not the reasoning-budget-forcing bug: Deckard
emitted Qwen's native qwen3_coder XML tool format malformed -> llama.cpp leaked
the closing tags into the arg value -> Bifrost attach_tool schema error -> retry
-> reasoning runaway to max_tokens. NEO-CODE emits the same native format cleanly
on the same seat/parser -> no schema error -> no runaway. The fix was the model
swap; there was never a wrong parser (the XML is Qwen3.5/3.6-native).
- reasoning seat: Deckard-PKD (Qwen3.5) -> NEO-CODE=Heretic2-Thinking (Qwen3.6-27B) Q5
- samplers: card defaults (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0), DRY dropped
- ctx: 256K max; custom llama.cpp kept (qwen3_coder parse + PR#25544 belt-and-suspenders)
- persistent-memory ACTIVE 1 marked resolved
The char-rp-reasoning (Deckard) seat now runs llamacpp-charrp:custom-latest via a
new LLAMA_REASONING_IMAGE var (Magidonia char-rp stays on stock — no reasoning bug).
Fixes Worldtree #355 at the source (budget multi-terminator handles Qwen3.5's
<tool_call> reasoning end-tag). Live 2026-07-13: Deckard loads on Blackwell, serves
coherent, reasoning bounds at the 400 budget. Rollback via .env LLAMA_REASONING_IMAGE.
Durable record of the custom llama.cpp the char-rp-reasoning seat will run to
fix Worldtree #355 (reasoning-budget forcing broken in stock b8840 — single
end-tag </think> can't match Qwen3.5's <tool_call> reasoning terminator, so the
budget never force-closes and reasoning runs away to max_tokens). PR #25544
adds multiple terminating sequences; unmerged upstream, so we build it.
- build.sh: reproducible recipe (clone master@6eddde0 + merge PR #25544 +
resolve the 1 server-common.cpp conflict + CUDA build for Blackwell sm_120)
- README.md: why + acceptance test + rollback + REMOVE-WHEN-MERGED tracking
Image llamacpp-charrp:6eddde0-pr25544 BUILT + smoke-tested on ana-ml2; seat
swap pending. See also auto-memory reference_charrp_custom_llamacpp_pr25544.
Operator wanted a reasoning-RP model that tolerates DRY (RpR-v4 forbids rep/DRY -> a
1/30 loop tail). Ran the full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored):
- Deckard-PKD (Qwen3.5-27B, DavidAU creative tune) WON: 0/30 loops, 0/30 refusals, clean
managed reasoning (native Qwen3.5 <think>/enable_thinking), DRY-tolerant, ~57 tok/s,
runs on the base llama-swap b8840 image. -> now the char-rp-reasoning seat (:8018).
- RpR-v4: 0 refusals but 1/30 loop (no-DRY). Pantheon-27B: clean slop but 7/30 explicit
refusals + needs the newer ggml-org/llama.cpp image (Qwen3.6 won't load on b8840).
Snowdrop + Gembrain (Gemma-4): floored (llama.cpp can't manage their reasoning without
the vetoed template hacks). Losers kept on disk as alternates.
- char-rp (Magidonia) unchanged; gen unchanged. gateway char-rp-reasoning -> Deckard
sampler (temp 1.0/top_p 0.95/top_k 40/min_p 0.05; DRY server-side).
char-rp -> TheDrummer Magidonia-24B-v4.3 Q6_K (Magistral prose, ~65 tok/s,
zero refusal, tight POV) via llama.cpp (:8016).
char-rp-reasoning -> ArliAI QwQ-32B-RpR-v4 Q5_K_M (abliterated managed reasoning,
~52 tok/s, reasoning surfaces in reasoning_content) via llama.cpp (:8018).
- New canonical stack stacks/char-rp-gguf/ (llama-server x2, GPU0-pinned, ~86/97G
co-resident with gen). GGUF sidesteps the vLLM-NVFP4 + Mistral-tokenizer traps that
killed the Angel serve. Never Ollama.
- Best-of-breed per seat: no single dense 24-32B is both an elite non-thinking prose
seat AND a clean managed-reasoning seat on llama.cpp (Magidonia [THINK] boundary is
loose; Cydonia-R1 <think> runs away; QwQ is template-managed). Pantheon-Reasoning-27B
stays rejected (re-censors in <think>; RpR-v4 abliterated reasoning is the fix).
- Gateway rewired: char-rp->:8016, char-rp-reasoning->:8018, Mistral/QwQ samplers,
dropped the Qwen enable_thinking kwarg. One-model Magidonia fallback documented.
- Retired the ms32-24b-angel stack.
vh