feat(char-rp-gguf): replace broken Angel NVFP4 with dual GGUF RP seat on ana-ml2 GPU0

char-rp        -> TheDrummer Magidonia-24B-v4.3 Q6_K (Magistral prose, ~65 tok/s,
                  zero refusal, tight POV) via llama.cpp (:8016).
char-rp-reasoning -> ArliAI QwQ-32B-RpR-v4 Q5_K_M (abliterated managed reasoning,
                  ~52 tok/s, reasoning surfaces in reasoning_content) via llama.cpp (:8018).

- New canonical stack stacks/char-rp-gguf/ (llama-server x2, GPU0-pinned, ~86/97G
  co-resident with gen). GGUF sidesteps the vLLM-NVFP4 + Mistral-tokenizer traps that
  killed the Angel serve. Never Ollama.
- Best-of-breed per seat: no single dense 24-32B is both an elite non-thinking prose
  seat AND a clean managed-reasoning seat on llama.cpp (Magidonia [THINK] boundary is
  loose; Cydonia-R1 <think> runs away; QwQ is template-managed). Pantheon-Reasoning-27B
  stays rejected (re-censors in <think>; RpR-v4 abliterated reasoning is the fix).
- Gateway rewired: char-rp->:8016, char-rp-reasoning->:8018, Mistral/QwQ samplers,
  dropped the Qwen enable_thinking kwarg. One-model Magidonia fallback documented.
- Retired the ms32-24b-angel stack.

stacks/char-rp-gguf/.env.example

@@ -0,0 +1,46 @@
+# char-rp-gguf stack tunables. Copy to `.env` on ana-ml2 before deploying.
+#
+#   cp .env.example .env
+#   docker compose up -d
+#
+# Keep compose.yaml constant; edit values here.
+
+# llama.cpp server image (bundles /app/llama-server; already present on ana-ml2).
+LLAMA_IMAGE=ghcr.io/mostlygeek/llama-swap:cuda
+
+# GPU 0 = the on-demand large-model card (co-resident with `gen`). GPU 1 is the
+# always-on vLLM trio (granite/embed/rerank/reward) — do NOT put this stack there.
+CHARRP_GPU_ID=0
+
+# GGUF store on the host (mounted read-only at /models). Candidate GGUFs live in
+# /tank/aimodels/llm/rp/ (pre-pulled).
+MODELS_DIR=/tank/aimodels/llm
+
+# ── PROSE seat (char-rp) ────────────────────────────────────────────────────
+CHARRP_CONTAINER=llama-charrp
+CHARRP_PORT=8016
+# Model path relative to /models. Default = Magidonia-24B-v4.3 (Magistral RP tune):
+# elite dark-romantasy prose, zero refusal, ~65 tok/s, tight POV adherence.
+CHARRP_MODEL=rp/TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf
+# Alternate prose model (more literary flair, looser POV adherence) — uncomment to swap:
+# CHARRP_MODEL=rp/MS3.2-PaintedFantasy-v4.1-24B-Q6_K.gguf
+CHARRP_CTX=16384
+
+# ── REASONING seat (char-rp-reasoning) ──────────────────────────────────────
+CHARRP_REASONING_CONTAINER=llama-charrp-reasoning
+CHARRP_REASONING_PORT=8018
+# Default = QwQ-32B-ArliAI-RpR-v4 Q5_K_M (abliterated reasoning → no re-censor;
+# llama.cpp-managed CoT). ~50 tok/s @ Q5. Use Q6_K (~46 tok/s) for a touch more
+# quality if speed is not binding.
+CHARRP_REASONING_MODEL=rp/QwQ-32B-ArliAI-RpR-v4-Q5_K_M.gguf
+CHARRP_REASONING_CTX=16384
+# Thinking-token cap (QwQ over-thinks otherwise → starves the prose). 300-500 = a
+# concise, useful scene-plan before the response.
+CHARRP_REASONING_BUDGET=400
+
+# ── ONE-MODEL FALLBACK (consistent Mistral style, lighter reasoning) ─────────
+# To collapse both seats onto Magidonia (drop QwQ): set
+#   CHARRP_REASONING_MODEL=rp/TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf
+# and remove the --reasoning* flags from the reasoning service in compose.yaml
+# (Magistral reasons only when the caller's system prompt contains "/think";
+# managed but LIGHT — see the compose header for why QwQ is the default).

stacks/char-rp-gguf/README.md

@@ -0,0 +1,95 @@
+# char-rp-gguf
+
+Dedicated GGUF **character-RP seat** on ana-ml2 GPU 0, served via llama.cpp
+(`llama-server`) — **never Ollama**. Replaces the broken `ms32-24b-angel` NVFP4
+serve (self-quant produced garbage even at greedy).
+
+Two co-located instances, alongside the 35B-A3B heretic `gen` (`qwen36-27b-aeon`,
+:8015):
+
+| service | port | gateway alias | model | role |
+|---|---|---|---|---|
+| `llama-charrp` | 8016 | `char-rp` | TheDrummer **Magidonia-24B-v4.3** Q6_K | non-thinking **prose** seat |
+| `llama-charrp-reasoning` | 8018 | `char-rp-reasoning` | ArliAI **QwQ-32B-RpR-v4** Q5_K_M | managed-**reasoning** seat |
+
+## Why these two models
+
+Requirements (operator, 2026-07-08): highest prose quality; ≥50 tok/s (single-seat);
+very low refusal for dark explicit romantasy; dense; GGUF-via-llama.cpp; thinking
+strongly preferred; must fit alongside the 35B-A3B on GPU 0.
+
+- **Magidonia-24B-v4.3** (`char-rp`) — Magistral (Mistral) dark-romantasy RP tune.
+  Live-tested: elite literary prose, **zero refusal** on explicit scenes, precise
+  POV/instruction adherence, **~65 tok/s** at Q6_K. The prose star.
+- **QwQ-32B-RpR-v4** (`char-rp-reasoning`) — QwQ reasoning RP tune whose reasoning
+  data was generated with **QwQ-abliterated**, so it does **not re-censor inside the
+  think phase** — the failure mode that disqualified Pantheon-Reasoning-27B (its
+  DeepSeek-distilled traces reason themselves into refusals). llama.cpp **manages**
+  QwQ reasoning: `--reasoning on` surfaces the trace in `reasoning_content` (content
+  stays clean prose, no `<think>` leak) and `--reasoning-budget` caps the CoT.
+  ~50 tok/s at Q5_K_M (46 at Q6).
+
+**Why not one model for both:** no single dense 24–32B is both an elite non-thinking
+prose seat and a clean *managed*-reasoning seat on llama.cpp. Magidonia's Magistral
+`[THINK]` discipline is loose (won't reliably close `[/THINK]` on substantive
+reasoning → prose bleeds into `reasoning_content`, `content` empties). Cydonia-R1's
+`<think>` is emergent, so llama.cpp can't manage/cap it → runaway CoT that never
+reaches the prose. QwQ's template opens `<think>` natively → llama.cpp manages+caps
+it. Best-of-breed per seat. See the compose header for the one-model fallback.
+
+**Alternate prose model:** `MS3.2-PaintedFantasy-v4.1-24B` (Magistral; more literary
+flair, looser POV adherence) — swap via `CHARRP_MODEL` in `.env`. All candidate GGUFs
+are pre-pulled to `/tank/aimodels/llm/rp/`.
+
+## Deploy
+
+Canonical copies live here; authoritative copies live on the host under
+`/opt/docker/compose/char-rp-gguf/`.
+
+```bash
+# On ana-ml2 (models already pre-pulled to /tank/aimodels/llm/rp/):
+scripts/deploy-stack.sh ana-ml2 char-rp-gguf     # from this workspace
+# or, on the host:
+cd /opt/docker/compose/char-rp-gguf && cp .env.example .env && docker compose up -d
+```
+
+GGUFs are pulled with a detached `huggingface_hub` container, e.g.:
+
+```bash
+docker run -d --entrypoint python3 -v /tank:/tank vllm/vllm-openai:latest -c \
+  'from huggingface_hub import hf_hub_download as d; \
+   d(repo_id="bartowski/TheDrummer_Magidonia-24B-v4.3-GGUF", \
+     filename="TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf", local_dir="/tank/aimodels/llm/rp")'
+```
+
+## Gateway wiring (ana-docker LiteLLM `:4000`)
+
+`char-rp` → `http://10.250.50.54:8016/v1`, `char-rp-reasoning` →
+`http://10.250.50.54:8018/v1`. Sampler defaults (per the model cards):
+
+- **char-rp (Magidonia / Mistral):** `temperature 1.0, top_p 0.95, min_p 0.03`;
+  no repetition penalty (Magistral is stable without one). Callers may override.
+- **char-rp-reasoning (QwQ RpR-v4):** `temperature 1.0, top_k 40, min_p 0.02`;
+  **no repetition / DRY / XTC penalties** (RpR card is explicit about this).
+  Reasoning surfaces in `reasoning_content`.
+
+Do **not** carry the Qwen-specific `chat_template_kwargs.enable_thinking` on these
+aliases — it is meaningless to Mistral/QwQ templates. Reasoning is controlled
+server-side (`--reasoning on` on :8018).
+
+## Revert
+
+```bash
+docker compose -f /opt/docker/compose/char-rp-gguf/compose.yaml down
+```
+
+The retired `ms32-24b-angel` stack is left staged for reference (its NVFP4 quant is
+broken — do not re-serve it).
+
+## Verify
+
+```bash
+curl -s http://10.250.50.54:8016/v1/models        # char-rp up
+curl -s http://10.250.50.54:8018/v1/models        # char-rp-reasoning up
+nvidia-smi --query-gpu=index,memory.free --format=csv   # GPU 0 margin
+```

stacks/char-rp-gguf/compose.yaml

@@ -0,0 +1,149 @@
+# char-rp-gguf — dedicated GGUF character-RP seat on ana-ml2 GPU 0, REPLACING the
+# broken ms32-24b-angel NVFP4 serve (garbage output — bad self-quant W4A4).
+#
+# Two co-located llama.cpp (llama-server) instances on GPU 0, served alongside the
+# 35B-A3B heretic `gen` (qwen36-27b-aeon stack, :8015):
+#
+#   llama-charrp (:8016, gateway char-rp) — TheDrummer Magidonia-24B-v4.3 Q6_K.
+#       Magistral (Mistral) dark-romantasy RP tune. NON-thinking PROSE seat: elite
+#       literary prose, zero refusal, ~65 tok/s, precise POV/instruction adherence.
+#
+#   llama-charrp-reasoning (:8018, gateway char-rp-reasoning) — ArliAI QwQ-32B-RpR-v4 Q5_K_M.
+#       QwQ reasoning RP tune whose reasoning DATA was generated with QwQ-ABLITERATED
+#       → it does NOT re-censor in the think phase (the exact failure mode that killed
+#       the Pantheon/DeepSeek-distilled reasoners: they reason themselves into refusals
+#       inside <think>). llama.cpp MANAGES QwQ reasoning natively: --reasoning on
+#       surfaces the trace in reasoning_content (clean prose in content, no <think>
+#       leak), --reasoning-budget caps the chain-of-thought. ~50 tok/s @ Q5_K_M.
+#
+# WHY GGUF/llama.cpp (not vLLM NVFP4): sidesteps BOTH traps that killed the Angel serve
+# — the vLLM NVFP4 self-quant breakage AND the Mistral-tokenizer/vision crash. llama.cpp
+# handles Mistral + QwQ tokenizers natively. NEVER Ollama (banned fleet-wide).
+#
+# WHY TWO models (not one): no single dense 24-32B is BOTH an elite non-thinking prose
+# seat AND a clean managed-reasoning seat on llama.cpp. Magidonia's Magistral [THINK]
+# discipline is loose (won't reliably close [/THINK] on substantive reasoning → prose
+# bleeds into reasoning_content, content empties); Cydonia-R1's <think> is emergent, so
+# llama.cpp can't manage/cap it → runaway CoT that never reaches prose. QwQ's template
+# opens <think> natively → llama.cpp manages+caps it. So: best-of-breed per seat.
+# ONE-MODEL FALLBACK (consistent Mistral style, lighter reasoning): point both services
+# at Magidonia via CHARRP_REASONING_MODEL in .env and blank CHARRP_REASONING_EXTRA_*.
+#
+# ALTERNATE prose model: PaintedFantasy-v4.1-24B (also Magistral, more literary flair
+# but looser POV adherence) — set CHARRP_MODEL in .env. All candidate GGUFs are
+# pre-pulled to /tank/aimodels/llm/rp/.
+#
+# VRAM (GPU 0, co-resident with gen ~38G): Magidonia Q6 ~19G + RpR-v4 Q5 ~23G + KV/
+# compute ~6-8G = ~85-88G / 97G (~9-12G margin). Keep ctx modest; drop CHARRP_*_CTX
+# to 8192 in .env if warmup bites. depends_on sequences char-rp first.
+#
+# API auth: blank (LAN-internal on the GPU host; matches API_KEY= in the AEON stack /
+# gateway VLLM_API_KEY). llama-server ignores the gateway's api_key when none is set.
+#
+# All tunables live in .env — edit that, not this file.
+
+name: char-rp-gguf
+
+services:
+  # ── PROSE seat — non-thinking. gateway char-rp. ──
+  llama-charrp:
+    image: ${LLAMA_IMAGE:-ghcr.io/mostlygeek/llama-swap:cuda}
+    container_name: ${CHARRP_CONTAINER:-llama-charrp}
+    restart: unless-stopped
+    runtime: nvidia
+    ports:
+      - "${CHARRP_PORT:-8016}:8080"
+    volumes:
+      - ${MODELS_DIR:-/tank/aimodels/llm}:/models:ro
+    environment:
+      # Pin to GPU 0 (the on-demand large-model card; the always-on vLLM trio owns GPU 1).
+      - NVIDIA_VISIBLE_DEVICES=${CHARRP_GPU_ID:-0}
+    entrypoint: ["/app/llama-server"]
+    command:
+      - --model
+      - /models/${CHARRP_MODEL:-rp/TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf}
+      - --host
+      - 0.0.0.0
+      - --port
+      - "8080"
+      - --n-gpu-layers
+      - "999"
+      - --ctx-size
+      - "${CHARRP_CTX:-16384}"
+      - --flash-attn
+      - on
+      - --jinja
+    healthcheck:
+      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health >/dev/null || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 240s
+    networks:
+      - tnet
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=char-rp (Magidonia-24B GGUF)
+      - homepage.icon=mdi-drama-masks
+      - homepage.description=Dark-romantasy RP prose seat, non-thinking (llama.cpp, ana-ml2 GPU 0)
+      - homepage.href=http://10.250.50.54:${CHARRP_PORT:-8016}
+
+  # ── REASONING seat — QwQ managed thinking. gateway char-rp-reasoning. ──
+  llama-charrp-reasoning:
+    image: ${LLAMA_IMAGE:-ghcr.io/mostlygeek/llama-swap:cuda}
+    container_name: ${CHARRP_REASONING_CONTAINER:-llama-charrp-reasoning}
+    restart: unless-stopped
+    runtime: nvidia
+    # Sequence AFTER the prose seat is healthy so the two GPU-0 allocations don't race.
+    depends_on:
+      llama-charrp:
+        condition: service_healthy
+    ports:
+      - "${CHARRP_REASONING_PORT:-8018}:8080"
+    volumes:
+      - ${MODELS_DIR:-/tank/aimodels/llm}:/models:ro
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=${CHARRP_GPU_ID:-0}
+    entrypoint: ["/app/llama-server"]
+    command:
+      - --model
+      - /models/${CHARRP_REASONING_MODEL:-rp/QwQ-32B-ArliAI-RpR-v4-Q5_K_M.gguf}
+      - --host
+      - 0.0.0.0
+      - --port
+      - "8080"
+      - --n-gpu-layers
+      - "999"
+      - --ctx-size
+      - "${CHARRP_REASONING_CTX:-16384}"
+      - --flash-attn
+      - on
+      - --jinja
+      # QwQ reasoning is template-native → llama.cpp manages it. --reasoning on surfaces
+      # the trace in reasoning_content (content stays clean prose); --reasoning-budget
+      # caps the CoT so it can't run away and starve the prose (QwQ over-thinks otherwise).
+      - --reasoning
+      - on
+      - --reasoning-format
+      - deepseek
+      - --reasoning-budget
+      - "${CHARRP_REASONING_BUDGET:-400}"
+    healthcheck:
+      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health >/dev/null || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 300s
+    networks:
+      - tnet
+    labels:
+      - homepage.group=AI Systems
+      - homepage.name=char-rp-reasoning (QwQ-32B RpR-v4 GGUF)
+      - homepage.icon=mdi-brain
+      - homepage.description=Dark-romantasy RP reasoning seat, managed CoT (llama.cpp, ana-ml2 GPU 0)
+      - homepage.href=http://10.250.50.54:${CHARRP_REASONING_PORT:-8018}
+
+networks:
+  tnet:
+    name: traefik-net
+    external: true

stacks/litellm/conf/config.yaml

@@ -148,49 +148,42 @@ model_list:
           enable_thinking: true
     model_info:
       mode: chat
-  # char-rp -> the native-MTP single-seat RP twin (:8016, served qwen3.6-27b-aeon-rp).
-  #   SAME weights as gen, MTP ON (qwen3_5_mtp n=3) for single-stream RP latency. RP-shaped
-  #   sampler defaults (Worldtree character-rp role / callers override); thinking OFF.
-  #   ⚠️ MTP silently drops min_p/logit_bias — don't rely on those through char-rp.
+  # char-rp -> GGUF dark-romantasy PROSE seat (:8016, llama.cpp, char-rp-gguf stack on
+  #   ana-ml2 GPU 0). TheDrummer Magidonia-24B-v4.3 Q6_K — Magistral (Mistral) RP tune.
+  #   NON-thinking: elite literary prose, zero refusal on dark/explicit scenes, ~65 tok/s,
+  #   tight POV/instruction adherence (live-tested 2026-07-08). Replaced the broken Angel
+  #   NVFP4 serve AND the earlier AEON-rp MTP twin. Magistral is stable WITHOUT a repetition
+  #   penalty (dropped the old 1.05); min_p 0.03 is the anti-slop knob. No enable_thinking
+  #   kwarg — meaningless to the Mistral template. Alt prose model (swap via the stack .env):
+  #   MS3.2-PaintedFantasy-v4.1-24B. Callers may override the sampler.
   - model_name: char-rp
     litellm_params:
-      model: hosted_vllm/qwen3.6-27b-aeon-rp
+      model: hosted_vllm/magidonia-24b-v4.3
       api_base: http://10.250.50.54:8016/v1
       api_key: os.environ/VLLM_API_KEY
-      # Qwen3.x non-thinking RP recs (operator 2026-07-05, adapted from Qwen + community
-      # RP testing). presence_penalty light (0.1) to reduce topic drift; repetition_penalty
-      # 1.05. min_p skipped (Qwen rec + MTP drops it anyway). DRY off (Qwen3.x artifacts).
-      temperature: 0.7
-      top_p: 0.8
-      presence_penalty: 0.1
+      temperature: 1.0
+      top_p: 0.95
       extra_body:
-        top_k: 20
-        repetition_penalty: 1.05
-        chat_template_kwargs:
-          enable_thinking: false
+        min_p: 0.03
     model_info:
       mode: chat
-  # char-rp-reasoning -> same RP seat (:8016, MTP), thinking ON. Reasoning-profile
-  #   sampler (lower temp than char-rp for coherent thought); tunable. ⚠️ The thinking
-  #   TRACE does not yet surface in reasoning_content (chat_template injects <think> in the
-  #   prompt → parser drops the span); pending a template fix, NOT a parser swap.
+  # char-rp-reasoning -> GGUF managed-REASONING seat (:8018, llama.cpp, char-rp-gguf stack).
+  #   ArliAI QwQ-32B-ArliAI-RpR-v4 Q5_K_M — QwQ reasoning RP tune. Reasoning is ON server-side
+  #   (--reasoning on): the CoT SURFACES in reasoning_content and content stays clean prose
+  #   (fixes the old trace-not-surfacing gap), CoT budget-capped so it can't starve the prose.
+  #   Its reasoning data is QwQ-ABLITERATED → no re-censor inside <think> (the failure mode
+  #   that disqualified Pantheon-Reasoning-27B). ~52 tok/s @ Q5_K_M. RpR card: temp 1.0,
+  #   top_k 40, min_p 0.02, and NO repetition / DRY / XTC penalties. NOT the same model as
+  #   char-rp (best-of-breed per seat) — see stacks/char-rp-gguf/README.md.
   - model_name: char-rp-reasoning
     litellm_params:
-      # Distinct served-name so a thinking-off char-rp request can't clobber this to
-      # enable_thinking:false (the bug that broke it). Same backend :8016, different model id.
-      model: hosted_vllm/qwen3.6-27b-aeon-rp-thinking
-      api_base: http://10.250.50.54:8016/v1
+      model: hosted_vllm/qwq-32b-rpr-v4
+      api_base: http://10.250.50.54:8018/v1
       api_key: os.environ/VLLM_API_KEY
-      # Same RP profile as char-rp but thinking-mode top_p 0.95 (Qwen thinking rec) + focused
-      # temp 0.6. (Reasoning-trace surfacing still pending the parser/template fix.)
-      temperature: 0.6
-      top_p: 0.95
-      presence_penalty: 0.1
+      temperature: 1.0
       extra_body:
-        top_k: 20
-        repetition_penalty: 1.05
-        chat_template_kwargs:
-          enable_thinking: true
+        top_k: 40
+        min_p: 0.02
     model_info:
       mode: chat