vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(pfi): add SOTA eRP thinking-finetunes research (gecko-65 booth → reference)

Browse Source 
56KB deep-research report on thinking-capable eRP finetunes 15-30B, weighted for
spatial/state coherence, targeting RTX PRO 6000 Blackwell (sm_120) NVFP4. Preserved
from an ephemeral Booth (gecko-65) into durable reference for the quant decision.
This commit is contained in:
Vuong Hoang
2026-08-12 02:05:22 -07:00
parent 69597cb686
commit 7bd7375d65
1 changed files with 426 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/pfi/erp-thinking-finetunes-2026.md
+426
View File
@@ -0,0 +1,426 @@
# Thinking-Capable eRP Finetunes, 15–30B — Deep Research
**Compiled 2026-08-12 · Window: Feb–Aug 2026 · Weighted for spatial/state coherence · Target: RTX PRO 6000 Blackwell (sm_120), NVFP4, throughput**
---
## 0. Read this first — three findings that should change your shortlist
**1. The 24B Mistral era is over.** Everything worth running in this band now sits on one of four bases, all of which ship native thinking out of the box: **Qwen3.6-27B** (Apr 2026), **Qwen3.5-27B** (Feb 2026), **Gemma-4-31B / Gemma-4-26B-A4B** (Mar 31 2026, now **Apache 2.0**), and **arcee-ai/Trinity-Mini** (26B-A3B). Mistral has shipped *nothing* in your band in 2026 — Mistral Small 4 is a 119B-A6B MoE that absorbed the Magistral line. Magistral-Small-2509 (Sep 2025) is still the newest in-range Mistral reasoning model, and the 24B tunes built on it are now a legacy tier.
**2. The evidence says heavy eRP finetuning actively damages the thing you care about most.** This is the uncomfortable core of this report and it's covered in §2. Short version: reasoning-native models buy real long-context state tracking, but bolting RP-tuning *and* reasoning-tuning on top degrades both prose and world-modeling. The single most respected merger in the space says flatly that 24B "will struggle with details of logical/physical continuity at times — which is probably inescapable for a 24B model." **If spatial coherence is your #1 criterion, bias toward light-touch tunes on smart bases, not heavy eRP tunes.**
**3. MTP and best-in-class RP tuning are currently mutually exclusive — with exactly one escape hatch.** Every dedicated RP brand (Cydonia, Skyfall, Dark-Scarlett, MeroMero, Artemis, Magistry) sits on Mistral or Gemma bases that **have no MTP heads at all**. Only Qwen3.5/3.6-27B ships MTP in your band — and `from_pretrained` **silently drops the MTP heads during finetuning**, so almost every Qwen-based community tune has lost them too. The escape hatch is the `Native-MTP-Preserved` lineage (§5.2), which grafts the 15 MTP tensors back post-hoc, and already has NVFP4 checkpoints.
> **Also worth knowing up front:** at temp 0.8–1.2 (normal RP sampling), speculative decoding acceptance collapses to ~38–52%, and vLLM's own guidance is to disable it below 0.5. On a *shared, batched* box it is likely a net throughput **loss**. Details and the one contradicting measurement in §5.4.
---
## 1. Ranked picks
Ranked for **spatial/state coherence first**, prose second, with your NVFP4 + throughput constraints factored in.
| # | Model | Params | Base | Thinking | NVFP4 today? | MTP? |
|---|---|---|---|---|---|---|
| 1 | [zerofata/G4-MeroMero-v2-31B](https://huggingface.co/zerofata/G4-MeroMero-v2-31B) | 31.27B | Gemma-4-31B | Dual (Think/NoThink presets) | v1 only — must quantize v2 | ✗ |
| 2 | [Gryphe/Pantheon-Reasoning-27B](https://huggingface.co/Gryphe/Pantheon-Reasoning-27B) | ~27.8B | Qwen3.6-27B (MTP-preserved heretic) | **Always-on** | ✗ — must quantize | ✗ (re-graftable) |
| 3 | [llmfan46/…-Native-MTP-Preserved-NVFP4](https://huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-NVFP4) | ~27.8B | Qwen3.6-27B | Native | **✓ shipped** | **✓ intact** |
| 4 | [allura-org/Qwen3.5-27B-Anko](https://huggingface.co/allura-org/Qwen3.5-27B-Anko) | ~27.4B | ArliAI Qwen3.5-27B-Derestricted | Dual-mode (trained both ways) | ✗ | ✗ |
| 5 | [ReadyArt/Dark-Scarlett-v1.0-27B](https://huggingface.co/ReadyArt/Dark-Scarlett-v1.0-27B) | ~27.8B | Qwen3.6-27B | `enable_thinking` flag | ✗ (W4A16/W8A16 PTQ only) | ✗ |
| 6 | [TheDrummer/Artemis-31B-v1.1](https://huggingface.co/TheDrummer/Artemis-31B-v1.1) | 31.27B | Gemma-4-31B | Dual + custom tags | ✗ | ✗ |
| 7 | [Gryphe/Pantheon-Reasoning-26B-A4B-1.1](https://huggingface.co/Gryphe/Pantheon-Reasoning-26B-A4B-1.1) | 26.5B MoE (A4B) | Gemma-4-26B-A4B | **Always-on** | ✗ | ✗ |
| 8 | [zerofata/G4-MeroMero-26B-A4B](https://huggingface.co/zerofata/G4-MeroMero-26B-A4B) | 25.8B MoE (A4B) | Gemma-4-26B-A4B | Dual | **✓** (2 quantizers) | ✗ |
| 9 | [sophosympatheia/Magistry-24B-v1.1](https://huggingface.co/sophosympatheia/Magistry-24B-v1.1) | 23.6B | Magistral-2509-24B | `<think>` prefill | MLX only | ✗ |
| 10 | [zerofata/Q3.5-BlueStar-v2-27B](https://huggingface.co/zerofata/Q3.5-BlueStar-v2-27B) | ~27.4B | Qwen3.5-27B | `<think>\n` prefill (trained) | MLX only | ✗ |
**Wildcard worth a slot on your test rig:** [Gryphe/WorldSim-Opus-3.6-35B-A3B](https://huggingface.co/Gryphe/WorldSim-Opus-3.6-35B-A3B) — 35B-A3B, over your band but only ~3B active so it's cheap. It is the closest thing anyone has built to a model *designed* for the state-tracking problem: trained on three datasets that all carry full thinking traces, with reasoning persisting per-turn. The author calls it a research release whose "practical effectiveness remains uncertain."
**Actively avoid for your criterion:** [LatitudeGames/Equinox-31B](https://huggingface.co/LatitudeGames/Equinox-31B) — card states verbatim "No reasoning datasets were included during training," thinking suppressed by default. [TheDrummer/Rocinante-XL-16B-v1](https://huggingface.co/TheDrummer/Rocinante-XL-16B-v1) — user reports of degradation past 16k and noticeable decline past 20k; you can't track scene state in a window that small.
---
## 2. Does thinking actually help spatial coherence? — the evidence
This deserves its own section because the answer is **"yes for state tracking, no for prose, and only if the model was pretrained for reasoning."**
### 2.1 Thinking clearly helps long-context state tracking — for reasoning-native models
- **Fiction.liveBench** (narrative comprehension, theory of mind, chronological reasoning at length) is the single strongest datapoint. At 16k context: **QwQ-32B 83.3%** vs Gemma-3-27B 33.3% vs dolphin-Mistral-24B 25.0% — a reasoning-native 32B beating a *70B* non-reasoning model (Llama-3.3-70B, 33.3%) by 50 points. Same-model toggle: claude-3-7-sonnet thinking **83.3%** vs non-thinking **50.0%** at 16k. [[data]](https://raw.githubusercontent.com/mnismt/llms-long-context-benchmark/main/src/data/benchmark.ts) [[Epoch]](https://epoch.ai/benchmarks/fictionlivebench)
- **LongBench Pro** (8k–256k, includes consistency-checking and dialogue-tracking): thinking mode adds **+11 to +16 points** for reasoning-native models (Claude-4-Sonnet 56.07→69.87; DeepSeek-V3.2 51.67→67.82). But models *not trained* for thinking gain nothing — Llama-3.1-405B **+0.59**, Gemma-3-12B **−0.24**. Paper's own conclusion: "models without thinking training may fail to effectively leverage test-time compute." [[arXiv 2601.02872]](https://arxiv.org/html/2601.02872v1)
- **MuSR** (multi-step narrative state tracking): Ministral 3 14B Reasoning **70%** vs base **64%**; consistent +6 to +9 at every size down to 1.2B. [[BenchLM]](https://benchlm.ai/benchmarks/musr)
- **UGI "World Model"** column, same-model toggles: Qwen3-32B **21.25 → 23.80**, Qwen3-30B-A3B **13.10 → 16.67** with thinking on.
### 2.2 Thinking reliably damages prose and *destroys* instruction-following
Every same-model pair in the UGI dataset shows the `Writing` score dropping when thinking is on: Qwen3-14B **34.76 → 29.64**, Qwen3-32B 32.95 → 30.34, Qwen3-30B-A3B 30.24 → 28.54, Qwen3-8B 27.96 → 23.87. gpt-oss-20b degrades monotonically with reasoning effort — Writing **24.62 (low) → 24.50 (med) → 10.94 (high)** with repetition interrupts rising 2 → 1 → **8**.
The instruction-following collapse is the most reproducible effect in the entire dataset. `creative_writing_wc_exceeded_pct` — the share of creative tasks where the model blew the requested word limit:
| Model | Thinking off | Thinking on |
|---|---|---|
| Qwen3-14B | 1% | **99%** |
| Qwen3-32B | 0% | **100%** |
| Qwen3-30B-A3B | 10% | **99%** |
| Qwen3-8B | 4% | **100%** |
If you've ever wondered why a thinking model ignores your "keep replies to two paragraphs" instruction — that's this.
### 2.3 The warning case: bolting reasoning onto an RP finetune
`Cydonia-R1-24B-v4` vs `Cydonia-24B-v4` — same trainer, same base lineage, one reasoning-tuned:
| Metric | Cydonia-24B-v4 | Cydonia-R1-24B-v4 |
|---|---|---|
| Writing | 30.91 | **20.38** (−34% rel.) |
| World Model | 23.30 | **19.33** (−17%) |
| NatInt | 26.64 | 24.27 |
| Length error | 22% | **80%** |
| W/10 (willingness) | 7.8 | 8.2 ✓ |
Reasoning-tuning bought willingness and cost everything else, *including the world-model score*. Caveat: separate training runs, not a toggle, so recipe differences are confounded. But it's the closest analogue to "what happens when an RP finetuner adds thinking."
### 2.4 Mechanistic support for why
- **Visual vs Textual CoT diagnostic** (ACL 2026): textual chain-of-thought **degrades spatial transformation by up to 16.5%** and **multi-object tracking by 12.7%** vs direct answering, measured across GPT-5, Claude Opus 4.6, Gemini 2.5 Pro, Qwen3-VL-72B. [[pdf]](https://aclanthology.org/2026.alvr-main.1.pdf) That is *literally your criterion*, and CoT made it worse.
- **"Mind Your Step (by Step)"**: CoT reduces performance on implicit statistical learning by up to **−36.3%** absolute, framed as verbal overshadowing — narrating a scene in a scratchpad makes the model worse at *feeling* the scene. [[arXiv 2410.21333]](https://arxiv.org/html/2410.21333v4)
- **Contrary evidence worth weighing** — "Thinking in Character" found *role-aware* reasoning beats naive reasoning (CharacterBench 3.69 RAR vs 3.57 distill), but note the third term: **undirected extra thinking scored worst at 3.05**. The claim is not "reasoning helps," it's "reasoning helps only if its style is constrained to the character." [[arXiv 2506.01748]](https://arxiv.org/html/2506.01748v1)
### 2.5 And at the frontier, reasoning doesn't fix narrative consistency at all
- **NarrativeWorldBench**: frontier + reasoning models all cluster at **F1 0.78–0.81** at horizon 50 with no significant difference (p>0.13); everything loses ~0.20 F1 from h=10 to h=200. A purpose-built 8B latent world model holds **F1 ≥ 0.84 across all horizons** at ~4× lower cost. [[arXiv 2606.17391]](https://arxiv.org/html/2606.17391v1)
- **NCP-Bench** (Aug 2026) is the benchmark you were hoping existed — it explicitly scores *spatial consistency* ("character described on the bridge later appearing in a doorway"), *object state tracking* ("a raft inflated→deflated without justification"), and character knowledge leakage. Results are humbling: **GPT-5.2 survives 20 turns only 42% of the time**, near-zero survival by 100 turns, fact conflicts at 40–68% across all models. It tests no sub-32B models. [[arXiv 2608.08160]](https://arxiv.org/abs/2608.08160)
- **RP-Bench** found reasoning models (GLM 5.1, Gemini 3.1 Pro, Kimi K2.5/K2.6) *underperformed* frontier non-reasoning models on roleplay dimensions, with severe latency costs (Kimi K2.6 p95 **173s**, **17% truncation at length limit** — truncation is itself a coherence failure). Its verdict on the category: "**The RP-specialist finetunes — the models marketed for exactly this — rank last.**" [[repo]](https://github.com/LeviTheWeasel/rp-benchmark)
### 2.6 What I'd actually do with this
The defensible synthesis: **use thinking sparingly and structurally, not as an always-on prefix to prose.** A gated pattern — reasoning enabled for scene-state checks, scene transitions, and complex multi-character blocking; disabled for straight prose continuation — captures the state-tracking gain without paying the prose and length-adherence tax. Every model in §1 that supports *dual* mode (MeroMero, Artemis, BlueStar, Dark-Scarlett) lets you do this at the request level. The always-on models (Pantheon, WorldSim) do not.
---
## 3. Per-model breakdowns
Metadata below is from the HuggingFace API, verified individually. Download counts are trailing-30-day and are **unreliable as a quality signal** — most users pull the GGUF mirror repos, not the BF16 originals.
### 3.1 zerofata/G4-MeroMero-v2-31B — best-shaped training for your criterion
[huggingface.co/zerofata/G4-MeroMero-v2-31B](https://huggingface.co/zerofata/G4-MeroMero-v2-31B) · 31.27B · Gemma-4-31B · Apache-2.0 · **2026-08-03** · 258 dl / 43 likes
The reason this is #1: it is the **only model in the entire survey whose training explicitly optimizes reasoning against a coherence judge.** Verbatim from the card, the pipeline is `SFT > Merge > GRPO > GRPO > on-policy SFT`:
1. Diversity SFT — ~4,000 curated stories, 0.5 blend merge-back
2. **Creative GRPO** — 8 rollouts/prompt, 300 steps, *thinking disabled*
3. **RP Logic GRPO** — 100 steps, *thinking enabled*, scored by "a logic-defect judge (DeepSeek-V4 Flash with a rubric)", with a `reward_judge_coherence` reward term
4. On-policy SFT — ~3,300 self-generated RP samples, diversity-filtered
Stage 3 is the mechanism that should produce state tracking. **Honest caveat:** the card does *not* claim improved spatial coherence as an outcome, and I could not confirm the stage-3 prompts were multi-turn (an earlier source claimed this; it's unverified). You're buying a plausible training signal, not a measured result.
Author's own metrics vs stock Gemma 4: swipe diversity **0.72 vs 0.43**, story slop **7.4 vs 8.8 per 1k words**, bare-prompt attractor hit rate **66% vs 99%**, no regression on IFEval / GSM8K / MMLU-Pro.
- **Thinking:** dual, via `Gemma4-Think.json` / `Gemma4-NoThink.json` SillyTavern presets. Reasoning is longer than stock Gemma 4, shorter than MeroMero v1.
- **Samplers:** temp 0.8–1.0, MinP 0.05
- **Quants:** GGUF (official + mradermacher), FP8 W8A16 ([hoborific](https://huggingface.co/hoborific/G4-MeroMero-v2-31B-W8A16-FP8)), exl3, MLX. **NVFP4 exists only for v1** ([pekkAi](https://huggingface.co/pekkAi/G4-MeroMero-31B-NVFP4), [heretic variant](https://huggingface.co/pekkAi/G4-MeroMero-31B-uncensored-heretic-NVFP4)). You'll quantize v2 yourself.
- **Note:** 31.27B is marginally over your stated band. There is a true in-band sibling, [G4-MeroMero-26B-A4B](https://huggingface.co/zerofata/G4-MeroMero-26B-A4B) (25.8B MoE, A4B, May 2), which *does* have NVFP4 ([Deaquay](https://huggingface.co/Deaquay/G4-MeroMero-26B-A4B-NVFP4), [pekkAi heretic](https://huggingface.co/pekkAi/G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4)) and claims "reasoning is more structured, using less tokens during RP." But the 26B's card is candid that "logic and repetition I think are roughly on par with the original" — v2-31B is where the coherence work actually happened.
### 3.2 Gryphe/Pantheon-Reasoning-27B — best methodology, and it sits on the MTP-preserved base
[huggingface.co/Gryphe/Pantheon-Reasoning-27B](https://huggingface.co/Gryphe/Pantheon-Reasoning-27B) · ~27.8B · Apache-2.0 · **2026-05-30** · 232 dl / 27 likes
Base is `llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved` — **verified**, and that matters enormously for your rig (§5.2).
Two things make this the most methodologically interesting tune in the set:
- **Always-on reasoning.** Verbatim: "The model was trained with `preserve_thinking: true`, so thinking tags remain active across all assistant turns in multi-turn conversations, not just the first." Almost every other model reasons once and then stops.
- **The thinking traces were generated as *planning*, not annotation.** DeepSeek 3.2 produced them under the instruction to "think as a writer planning their next response — before writing — rather than annotating a response," then judge-model validated. This is the "role-aware reasoning" pattern that the CharacterBench work found is the *only* kind that helps.
Data mix: Pantheon RP corpus ~28%, Opus-4.6-Reasoning-24k ~21%, WorldSim narrative ~16%, text adventure/IF ~16%, general RP ~16%, Tiamat ~3%.
- **Samplers:** temp 1.0, **rep_pen 1.0**, min_p 0.05. The rep-pen point is emphatic and now consensus among reasoning-RP authors: repetition penalties corrupt thinking content. **Any thinking model whose card recommends rep_pen > 1.0 is a red flag.**
- **Template:** ChatML (Qwen3.6 chat template)
- **Author's own framing:** a research release, with the stated open question being "does reasoning actually help roleplay, or does it just add latency?" Respect that honesty.
- **Quants:** GGUF only. No NVFP4, no FP8. You will quantize this one.
- **Sibling:** [Pantheon-Reasoning-26B-A4B-1.1](https://huggingface.co/Gryphe/Pantheon-Reasoning-26B-A4B-1.1) (26.5B MoE, Gemma-4-26B-A4B, Jun 8) — same methodology, stricter trace QA, genuinely in-band, and the **most-reused merge donor in the whole 26B-A4B ecosystem**. SillyTavern gotcha: character-name prefixes break reasoning compatibility on this one — disable them.
### 3.3 llmfan46 Native-MTP-Preserved (NVFP4) — the throughput play
[huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-NVFP4](https://huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-NVFP4)
Not an RP finetune — a decensored Qwen3.6-27B. It's on this list because it's the **only 15–30B option that is simultaneously NVFP4, MTP-intact, and uncensored**, and because §2 argues that a smart, lightly-touched base may outperform a heavy eRP tune on exactly the axis you're prioritizing.
Parent repo: 7,580 dl / 41 likes, created May 6, modified May 25. Made with Heretic v1.3.0 using a variant of Magnitude-Preserving Orthogonal Ablation (MPOA), ablating only `attn.o_proj`, `attn.out_proj`, `mlp.down_proj`. Claimed: **94% fewer refusals (6/100 vs 92/100) at 0.0021 KL divergence**, MMLU 85.67% vs 86.65% original.
The load-bearing detail is `model-auxiliary.safetensors` in the repo — that's where Qwen stores the MTP heads, and its presence is hard proof the claim isn't marketing. The card enumerates all 15 preserved tensors.
**Pair it with a style fix.** Its weakness vs a proper RP tune is voice, not intelligence. [Gryphe/Gemma-4-26B-A4B-StyleTune-V2](https://huggingface.co/Gryphe/Gemma-4-26B-A4B-StyleTune-V2) demonstrates the approach on the Gemma side and is the most quantitatively-supported claim in this whole survey: it trains **precisely one tensor** — "the `lm_head` output projection… freeze everything else. All 30 transformer layers, all the attention heads, all the MLPs — completely untouched" — and measures **52% fewer clichés per 100 words (1.141 → 0.551)** over 200 RP prompts with only 19.9% shared trigram vocabulary. Reasoning capability is untouched by construction. There's no Qwen equivalent published yet, but the recipe is simple enough to replicate.
### 3.4 allura-org/Qwen3.5-27B-Anko
[huggingface.co/allura-org/Qwen3.5-27B-Anko](https://huggingface.co/allura-org/Qwen3.5-27B-Anko) · ~27.4B · Apache-2.0 · **2026-04-08** · 40 dl / 11 likes
**Correction to circulating claims:** the base is **`ArliAI/Qwen3.5-27B-Derestricted`**, not stock Qwen3.5-27B. LoRA r=64 / α=512 on Doubao Seed 2.0 Pro reasoning traces, trained on both reasoning *and* non-reasoning responses, so it's dual-mode by construction. Stated goal, verbatim: "increase the quality of reasoning and decrease looping, and fix slop in outputs."
Why it ranks well for you: Qwen3.5-27B is the best state-tracking base in the band by measurement — **MuSR 95, the best open-weight score overall**, and LongBench v2 60.6%.
- **Samplers, verbatim and shouted:** "**DO NOT USE QWEN'S SAMPLERS. THEY ARE AWFUL.**" Use **temp 1.25, min_p 0.05–0.1**.
- **Odd but documented:** recommended system prompt is `You are Claude, a helpful and harmless language model created by Anthropic.` It was trained to work with Claude-style system prompt formatting.
- **Quants:** GGUF only (bartowski, mradermacher). No NVFP4/FP8/AWQ/exl3.
- **Warning:** ArliAI's Derestricted line **drops MTP** — I verified the file manifest, there is no `model-auxiliary.safetensors`. So Anko has no MTP.
### 3.5 ReadyArt/Dark-Scarlett-v1.0-27B — cleanest eRP with flag-based thinking
[huggingface.co/ReadyArt/Dark-Scarlett-v1.0-27B](https://huggingface.co/ReadyArt/Dark-Scarlett-v1.0-27B) · ~27.8B · Qwen3.6-27B · Apache-2.0 (personal use, 18+) · **2026-06-16**
The most explicitly eRP-targeted model here with a properly documented thinking toggle:
```
chat_template_kwargs: {"enable_thinking": true, "reasoning_effort": "medium"}
```
That `reasoning_effort` knob is unusually useful for the gated-thinking pattern in §2.6 — you can dial it per-request rather than binary on/off.
Training: LoRA r=32, 2 epochs, **text layers only**, on 12,211 curated adult-RP prompts, with multi-turn generation, refusal filtering, and group-chat support in the pipeline.
- **Samplers:** top_p 0.92, temp 1.0, freq_pen 0, pres_pen 0
- **Real limitation:** the card states it's optimized for Male(user)→Female(AI) perspective. Narrow.
- **Quants:** GGUF + ReadyArt's own W4A16/W8A16 PTQ. **No NVFP4, no FP8.**
- Family context: ReadyArt shipped a dense June burst — `Dark-Scarlett-v2.0-31B` (Gemma-4), `v1.0-26B-A4B`, `v1.0-31B`, `v0.4-2509-24B`, `Heimdallr-v0.02-31B`. Download signal favors the MoEs.
### 3.6 TheDrummer/Artemis-31B-v1.1 — freshest, longest bake
[huggingface.co/TheDrummer/Artemis-31B-v1.1](https://huggingface.co/TheDrummer/Artemis-31B-v1.1) · 31.27B · Gemma-4-31B · **2026-08-06** · 7 likes · **no license set**
Four months of public iteration through BeaverAI test builds (`v1a` Apr 8 → `v1n` Jul 22), which is unusually thorough for this scene. **Use v1.1, not v1** — v1 has "strong writing potential but requires manual adjustments"; v1.1 "improves stability while maintaining v1's creative strengths," specifically fixing **"dash spiraling."**
- **Thinking:** the most flexible activation of any model here — "standard thinking gemma template or `<thinking></thinking>` blocks on non-thinking gemma template," and "`<think></think>` should work too, along with tricks like `<evil_think></evil_think>`."
- **Samplers:** not fixed in the card; Drummer points to a crowdsourced sampler spreadsheet.
- **Too new for consensus** as of Aug 12 — one enthusiastic but content-free feedback thread.
- Predecessor if you want something proven: [Skyfall-31B-v4.2](https://huggingface.co/TheDrummer/Skyfall-31B-v4.2) (Apr 3, Magistral-Small-2509 upscaled, Mistral v7 Tekken template) is the established workhorse of this window and **has an NVFP4 quant already** ([ealexeev, v4.1](https://huggingface.co/ealexeev/TheDrummer-Skyfall-31B-v4.1-NVFP4)).
### 3.7 sophosympatheia/Magistry-24B-v1.1 — the honest one
[huggingface.co/sophosympatheia/Magistry-24B-v1.1](https://huggingface.co/sophosympatheia/Magistry-24B-v1.1) · 23.6B · Apache-2.0 · **2026-03-22** · 35 likes (highest like count in-band)
A mergekit DELLA merge (not a finetune) on `Darkhn/Magistral-2509-24B-Text-Only`, so it inherits Magistral's native reasoning. Donors: `Casual-Autopsy/Maginum-Cydoms-24B`, `DarkArtsForge/Magistaroth-24B-v1`, plus `Huihui-Devstral-Small-2-24B-Instruct-2512-abliterated` at 0.3.
I'm listing it partly because its card contains the **single most on-point statement anyone in this scene has made about your criterion**, verbatim:
> "This model is fun, but it will struggle with details of logical/physical continuity at times — which is probably inescapable for a 24B model."
That is a respected merger saying 24B sits below the threshold where physical continuity holds. Take it seriously as a floor: **if spatial coherence is your top priority, 27B+ is the entry point, not 24B.**
- **Thinking:** prefill-based — force the reply to start with `<think>` plus basic instructions. Card notes `<think></think>` works better than Mistral's `[THINK][/THINK]` tags. (Related gotcha: on Mistral models `<think>` is *not* a special token; `[THINK]` is.)
- **Samplers:** three named presets — Conservative (temp 0.7, MinP 0.05, Top-N σ 0.75), Balanced (temp 1.0, Adaptive-P target 0.6 / decay 0.9), Wild (temp 0.9, Adaptive-P target 0.35 / decay 0.45). Also ships a SillyTavern Master Import JSON.
- **It is NOT gated** (a claim to the contrary is circulating; the API says `gated: false`).
- **Quants:** GGUF, exl3, MLX MXFP4/MXFP8. **No NVFP4.**
### 3.8 zerofata/Q3.5-BlueStar-v2-27B — best-documented anti-slop SFT
[huggingface.co/zerofata/Q3.5-BlueStar-v2-27B](https://huggingface.co/zerofata/Q3.5-BlueStar-v2-27B) · ~27.4B · Qwen3.5-27B · **MIT** · **2026-03-20** · 42 likes
The interesting technical contribution here is **custom loss masking on slop phrases** — "most common phrases of slop are masked out, so the model doesn't get rewarded for learning these patterns." That lets you train on otherwise-useful RP data without absorbing its clichés. SFT ~27M tokens via Axolotl + LoRA on 4×H200.
- **Thinking:** prefill `<think>\n` — and importantly, "it is required to prefill the `<think>\n` **as that is how it was trained**." This is a trained-for prefill, not a bolted-on hack. Ships separate think/no-think ChatML instruct JSONs.
- **Samplers:** temp 0.8–1.0, MinP 0.05–0.075
- **⚠️ Trained at 10,756 token sequence length** despite the 262k base. See §4 on why this matters more than anything else in the card.
- **Quants:** GGUF. The two "NVFP4" BlueStar repos you'll find are **MLX** (Apple silicon) — useless on Blackwell.
### 3.9 Also verified, lower priority
- **[Vortex5/G4-Moonlight-Dusk-26B-A4B](https://huggingface.co/Vortex5/G4-Moonlight-Dusk-26B-A4B)** (26.5B MoE, Jul 14, 1016 dl) — merge of Animus-V14.1-FFT + G4-MeroMero-26B-A4B + Esmeralda + **Pantheon-Reasoning-26B-A4B-1.1**. Highest download count of the Gemma-4 MoE merges. Thinking activation is **undocumented** — merge card only, no sampler guidance. Good candidate, poor paperwork.
- **[ArliAI/Qwen3.5-27B-RpRMax-v1](https://huggingface.co/ArliAI/Qwen3.5-27B-RpRMax-v1)** (Apr 28) — successor to the well-regarded QwQ-32B-ArliAI-RpR line, in a collection literally titled "Thinking-trained RP specialized models." **Confirmed to have no model card at all** — training method, datasets, template, samplers, context all unverified. Heavy third-party GGUF activity (bartowski et al.) suggests real pickup. High risk, possibly high reward.
- **[NewEden/Trinity-Mini-Ichthyo](https://huggingface.co/NewEden/Trinity-Mini-Ichthyo)** (26.1B-A3B, Jul 10, 2,489 dl — highest of any in-band RP repo) — trained with **actual RL** (Prime RL run, step-100 checkpoint, 32,768 ctx). Base is `NewEden/Trinity-Mini-Futaba`, not stock Trinity-Mini. **Gated behind a contact-info agreement and the README returns 401** — I could read nothing. Zero third-party quants, consistent with the gating. Interesting, unassessable.
- **[Nimbz/Gemma-4-Gembrain-31B](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B)** (~Aug 2) — 5-phase Gemma-4 merge, `<|think|>` reasoning, targets "enhanced logical and lateral thinking." Samplers: temp 1.0, Top-P 0.95, Min-P 0.03, DRY 0.8/1.75. Trending but unproven.
- **[ReadyArt/gemma-4-31B-it-scotoma-2](https://huggingface.co/ReadyArt/gemma-4-31B-it-scotoma-2)** (Aug 6) — not an RP tune, the most rigorous **anti-slop** work of the window: γ-fold refusal-edit projection + 3 rounds of preference training on 9.3k pairs. Measured over 480 RP continuations: stacked adjectives **↓21×**, "Not X. But Y." **↓4×**, em-dash asides **↓4×**. ⚠️ Explicitly **"not uncensored"** — refusal behavior matches base. Useful as a merge donor or style reference, not as a driver.
### 3.10 Confirmed dormant — stop waiting on these
Checked directly; **no 2026 releases in this band**: **anthracite-org / Magnum** (last: Nov 2024) · **Sao10K** (Mar 2025) · **Nitral-AI** (Sep 2025) · **PocketDoc / Dans-PersonalityEngine** (May 2025) · **Undi95** (Mar 2025) · **aixonlab** (May 2025) · **knifeayumu** (Aug 2025) · **TareksLab** (70B only, Aug 2025) · **Doctor-Shotgun** (quant-only in 2026) · **Delta-Vector** (moved to 399B Trinity-Large) · **inflatebot** · **Tesslate** (never RP).
**Steelskull correction:** `Steelskull/CWT-V5.6` (Apr 2026) is **not** an RP model — it's "Cognitive Workspace Transformer," a **57.8M-parameter** from-scratch research architecture trained on FineWeb-Edu. Steelskull's RP line (Electra / Nevoria / Broken-Tutu) has shipped nothing since L3.3-Shakudo-70B in Jul 2025.
**One to watch:** `TheDrummer/Orion-26B-A4B` exists only as BeaverAI test builds (`v1a` May 24 → `v1c` Jul 10). Dead center of your band. Likely the next official release after Artemis.
---
## 4. The thing nobody puts in the headline: training context length
This is buried in the model cards and it undercuts a lot of the spatial-coherence story:
| Model | Base context | **Actually trained at** |
|---|---|---|
| Q3.5-BlueStar-v2-27B | 262k | **10,756 tokens** |
| MS3.2-PaintedFantasy-v4.1-24B | 128k | **10,756 tokens** |
| Trinity-Mini-Futaba | 128k | **32,768 tokens** |
| Rocinante-XL-16B-v1 | — | user reports drift past **16–20k** |
You cannot track scene state across a 60k-token roleplay with a model whose RP behavior was only ever reinforced at 10k. Base-model long-context ability degrades gracefully in benchmarks, but the *RP-specific* behavior these tunes install has a much shorter effective horizon. **When you evaluate, test at your real session length, not at 8k.** This is probably the highest-leverage thing in this report that no leaderboard captures.
Related: **Gemma-4 degrades far more gracefully with context than Qwen3.6** on throughput — 32k→128k loss of **−32%** vs Qwen3.6-35B-A3B's **−65%** (dual RTX 4070 Ti). That's throughput only, not accuracy, but it's consistent with the architecture: Gemma-4 is full-attention dense; Qwen3.5/3.6 are hybrid Gated-DeltaNet linear-attention designs (3 linear blocks per 1 full-attention block), which are theoretically weaker at exact long-range state tracking despite the bigger advertised window.
---
## 5. Deployment on your rig
### 5.1 NVFP4 on sm_120 — the headline is W4A16, not W4A4
**Do not ship plain W4A4 NVFP4 for long-context RP.** NVIDIA's own guidance flipped to recommending **W4A16 (`NVFP4A16`)** for sm_120/121, citing **KLD 2–4× worse for W4A4, "especially past ~10K context where activation quantization noise compounds with KV-cache lookups."** [[NVIDIA forum]](https://forums.developer.nvidia.com/t/update-for-nvfp4-model-conversion-to-use-w4a16-instead-of-w4a4/370403) That is precisely the failure mode you'd care about and it's the only source I found measuring KLD rather than MMLU at RP-relevant context lengths.
Cheap experiment: **NVFP4 weight storage is identical between W4A4 and W4A16** — only the activation scales differ. Flipping is a `config.json` patch (set `config_groups.group_0.input_activations` to `null`), not a re-quantization.
**The tension you should be aware of:** W4A16 gives up the FP4 tensor-core compute path, so the gain becomes pure weight-compression/bandwidth — and Benjamin Marie's comparison found NVFP4A16 shows *minimal throughput gain over INT4 AWQ*, with AWQ/AutoRound scoring slightly *better* on accuracy and ~7GB smaller on disk. The counterargument for your box: freed VRAM converts to KV cache, which converts to concurrency, which is what you actually want on a shared rig.
**Quality at 24–32B — the size gradient is real.** Red Hat's aggregate NVFP4 recovery: 70B–235B ~99%, **~30B 97–99%**, 7B–14B ~95–98%. Per-model, the damage concentrates in reasoning: Qwen3-32B-NVFP4 scores 99.83% OpenLLM v1 but only **94.21% reasoning avg**; Qwen3-14B drops to **91.45% reasoning, 86.34% on AIME24**. NVIDIA's own QAD report states it plainly: *"for small LLMs, the accuracy drop from PTQ is often non-negligible."*
**sm_120-specific caveats (all confirmed against upstream issues):**
- **Silent Marlin fallback.** Backend selectors check `is_device_capability(100)` only; sm_120 fails and falls back to Marlin dequant, logging *"Your GPU does not have native support for FP4 computation."* [vLLM #47749](https://github.com/vllm-project/vllm/issues/47749) was **still open as of Jul 6 2026**. **Always grep your startup log for that warning** — if it's there, the whole exercise is moot.
- **Dense is the healthy path.** [CUTLASS #3096](https://github.com/NVIDIA/cutlass/issues/3096) explicitly states dense FP4 GEMM works correctly on sm_120; the broken path was **grouped (MoE) GEMM**. Nearly every sm_120 NVFP4 horror story you'll read is a MoE story. This is a real argument for **dense 27B over 26B-A4B MoE** on your hardware, at least until the FlashInfer 0.6.5 / `compute_120f` path is more settled.
- `compute_120f` (needs **CUDA 13.0**) vs `compute_120a`: ~2.7× throughput difference (39.0 vs 14.6 tok/s in the CUTLASS issue's own table).
- `flashinfer_cutlass` has a reported **race condition causing silent memory corruption at high concurrency**; `flashinfer_cudnn` is reported safer. **Directly relevant to you as a multi-tenant operator** — toy prompts won't surface it, only soak testing will.
- FP8 KV cache is not universally safe on sm_120 (GLM-5 requires BF16 KV). Test yours.
Env vars people actually set:
```bash
export FLASHINFER_CUDA_ARCH_LIST=12.0f
export FLASHINFER_FORCE_SM=120f
export VLLM_NVFP4_GEMM_BACKEND=cutlass
```
**Toolchain choice matters more than it looks:** llm-compressor emits `compressed-tensors` but **does not calibrate KV-cache scales by default**, so you fall back to BF16 KV — **2× KV memory, roughly half the concurrent sessions.** ModelOpt emits per-layer `k_scale`/`v_scale` and gets you real FP8 KV. On a shared box that's the deciding factor.
### 5.2 MTP — the one lineage that keeps it
The failure chain is three-deep and every stage is silent:
1. **Loading.** `Qwen3_5ForConditionalGeneration.from_pretrained` **drops the MTP heads**. Finetune → `save_pretrained` → heads gone, no warning. I verified ArliAI's Derestricted and RpRMax file manifests: **no `model-auxiliary.safetensors`, no MTP tensors.** This is why almost no community Qwen tune has MTP.
2. **Quantization.** Converters use allowlists and skip unknown tensor prefixes silently; GPTQ-style quantizers preserve the weights but never calibrate them, leaving effectively random values.
3. **Serving.** Even when present, `mtp.*` / `mtp.fc` must be in `quantization_config.ignore` or vLLM runs a quantized MTP head against differently-scaled activations.
**The fix is unglamorous:** copy the 15 MTP tensors out of the original `Qwen/Qwen3.6-27B` checkpoint and graft them onto your output shard. Published pipelines: [lna-lab/GGUF-to-NVFP4-SM120](https://github.com/lna-lab/GGUF-to-NVFP4-SM120) and AEON-7's variant. **This means you can graft MTP back onto Pantheon-Reasoning-27B**, since it descends from an MTP-preserved base — probably the single highest-value move available to you.
**Two caveats on grafted MTP for eRP specifically:**
- You're bolting the *base* model's draft head onto a *finetuned* target. Acceptance drops by however much your finetune moved the distribution — for an RP tune, a lot.
- The rtx6kpro notes warn explicitly: **"abliterated models: MTP heads were trained on censored content; avoid with abliterated models."** The head predicts what the *aligned* model would say, so acceptance collapses precisely on the content that differs. Mechanism is sound; generality is my inference.
- They also measured MTP causing a **−22% throughput regression** on sm_120 when Marlin fallback was active, because the draft heads expect native FP4 activations.
### 5.3 Existing NVFP4 checkpoints of RP finetunes — more than you'd expect
Two quantizers specialize in exactly this:
- **[ealexeev](https://huggingface.co/ealexeev)** — a pure TheDrummer shop, 9 repos, **ships `recipe.yaml` in-repo** so the recipe is reproducible: [Skyfall-31B-v4.1](https://huggingface.co/ealexeev/TheDrummer-Skyfall-31B-v4.1-NVFP4), [Cydonia-24B-v4.3](https://huggingface.co/ealexeev/TheDrummer-Cydonia-24B-v4.3-NVFP4), [Snowpiercer-15B-v4](https://huggingface.co/ealexeev/TheDrummer-Snowpiercer-15B-v4-NVFP4), [Magidonia-24B-v4.2.0](https://huggingface.co/ealexeev/The-Drummer-Magidonia-24B-v4.2.0-NVFP4)
- **[Firworks](https://huggingface.co/Firworks)** — ~100 NVFP4 repos incl. [Cydonia-24B-v4.3-heretic](https://huggingface.co/Firworks/Cydonia-24B-v4.3-heretic-nvfp4), [Magidonia-24B-v4.3](https://huggingface.co/Firworks/Magidonia-24B-v4.3-nvfp4), [WeirdCompound-v1.7-24b](https://huggingface.co/Firworks/WeirdCompound-v1.7-24b-nvfp4)
- **[AEON-7](https://huggingface.co/AEON-7)** — the MTP-grafting specialists. ModelOpt 0.43.0, `NVFP4_DEFAULT_CFG`, 15 MTP tensors grafted post-quantization, GatedDeltaNet layers kept BF16 (432 keys across 48 GDN layers), calibrated on `neuralmagic/calibration` 20 samples × 8192 tokens. **Publishes an RTX PRO 6000 number: 92 tok/s median, 124.7 peak, 67.7% acceptance.**
- **[sakamakismile](https://huggingface.co/sakamakismile)** — highest volume (~57 repos), explicit `-MTP` naming convention, incl. actual creative tunes: [Carnice-V2-27b-NVFP4-TEXT-MTP](https://huggingface.co/sakamakismile/Carnice-V2-27b-NVFP4-TEXT-MTP), [Qwen3.6-27B-Fable-Fusion-MTP-NVFP4](https://huggingface.co/sakamakismile/Qwen3.6-27B-Fable-Fusion-MTP-NVFP4). Also ships `DSv4-Flash-FP8-SM120-Configs`.
**Gemma-4 NVFP4 works** — the catastrophic vLLM bug ([#39407](https://github.com/vllm-project/vllm/issues/39407), logits saturating at the bf16 softcap ceiling and emitting `" a a a a"` forever) is in the **FP8_BLOCK** path, not NVFP4. Existing Gemma-4-*finetune* NVFP4 checkpoints: [pekkAi/G4-MeroMero-31B-NVFP4](https://huggingface.co/pekkAi/G4-MeroMero-31B-NVFP4), [AEON-7/Gemma-4-31B-it-DECKARD-HERETIC-Uncensored-NVFP4](https://huggingface.co/AEON-7/Gemma-4-31B-it-DECKARD-HERETIC-Uncensored-NVFP4), [Deaquay/G4-MeroMero-26B-A4B-NVFP4](https://huggingface.co/Deaquay/G4-MeroMero-26B-A4B-NVFP4). Gemma-4 quirks: exclude vision tower / `embed_vision` / `multi_modal_projector`, and note heterogeneous attention head dims (`head_dim=256`, `global_head_dim=512`) need multi-group KV support if you use spec decode. Gemma-4 has **no MTP** — spec decode there is EAGLE-based.
### 5.4 Speculative decoding at RP temperatures — probably don't
Measured acceptance vs temperature [[DigitalOcean vLLM guide]](https://www.digitalocean.com/community/tutorials/speculative-decoding-vllm-configuration-guide):
| Temperature | Acceptance |
|---|---|
| 0.0 | ~81% |
| 0.4 | ~71% |
| **0.8** | **~52%** |
| **1.0** | **~38%** |
The stated rule: below 0.5 acceptance, spec decode is net-negative. **Your RP sampling sits at 0.8–1.25.**
Corroborating, from AEON-7's own Qwen3.5-27B NVFP4 card with a DFlash drafter: greedy **~80% acceptance → ~91 tok/s**; **sampled ~5% acceptance → ~38 tok/s** against a ~50 tok/s no-spec baseline. That's a **~24% throughput loss** from turning it on.
And batching compounds it: spec decode gives 1.5–2.8× at low QPS but **1.4–1.8× slowdown at high QPS** when the GPU is compute-saturated. Every impressive DFlash/EAGLE number you'll see quoted is greedy decoding at concurrency 1 — the exact opposite of your regime on both axes.
**One contradicting measurement worth replicating:** [loFT LLC](https://loftllc.dev/en/docs/tech/llm-research/qwen3-6-27b-nvfp4-mtp-vllm-benchmark/) reports Qwen3.6-27B NVFP4 + MTP=3 at **87.9% acceptance, accept length 3.64, 161 tok/s mean at temp 1.0, top_p 0.95, top_k 20** on 2× RTX PRO 6000 Max-Q. If true, native MTP heads degrade far more gracefully under sampling than external drafters do — which would be a meaningfully different conclusion. Verify before believing it.
If you do use spec decode, vLLM ships [Dynamic Speculative Decoding](https://docs.vllm.ai/en/latest/features/speculative_decoding/dynamic_speculative_decoding/) to auto-disable under load — but note [vLLM #25112](https://github.com/vllm-project/vllm/issues/25112): *"Spec decoding is not disabled at/after configured batch size."* Verify the disable actually fires.
Free alternative worth trying: **n-gram / prompt-lookup decoding**. RP genuinely echoes its input — character cards, world info, prior turns get re-quoted — so it may pick up real acceptance at zero VRAM cost. Set `prompt_lookup_min=8`; the default of 2 causes structured-output corruption on Qwen3-class models ([vLLM #40875](https://github.com/vllm-project/vllm/issues/40875)).
### 5.5 Throughput reference points (all single RTX PRO 6000 unless noted)
| Model | Precision | Single-stream | Batched |
|---|---|---|---|
| Gemma-4-31B | NVFP4 + FP8 KV | 40.7 tok/s @1k, 38.3 @128k | 126.0 @ 4 req |
| Qwen3.6-27B | FP8 | 46.1 @1k, 30.4 @256k | peak 189.3 @ 5 concurrent |
| Qwen3.6-27B | NVFP4, 256k ctx, FP8 KV | ~58 tok/s | ~119 @ 2-parallel; 64.8 GiB left for KV |
| Qwen3.6-27B | NVFP4 + grafted MTP=3 | median ~92, peak 124.7 | 67.7% acceptance |
| Qwen3-32B | NVFP4 vs BF16 | — | **2,050 tok/s @ conc 128** (vs 1,156 BF16 = 1.77×) |
Note the NVFP4-over-BF16 advantage **narrows** from 2.1× at conc 64 to 1.77× at conc 128 — consistent with the argument that NVFP4's dense-model gain is weight compression (bandwidth), not FP4 math. For your throughput-first shared box: NVFP4 buys less raw compute than marketed, but a lot of freed VRAM → KV cache → concurrency.
### 5.6 A starting stack
```bash
pip install -U llmcompressor==0.13.0 # released 2026-08-11
# Recipe changes that matter for RP:
# scheme="NVFP4A16" (weight-only, NOT plain "NVFP4")
# ignore=["lm_head"]
# calibration: your OWN RP/creative corpus, or Opus-WritingPrompts
# num_calibration_samples=256-512, max_seq_length=8192
#
# UltraChat calibration is assistant-y and sanitized — RP finetune activations
# are out-of-distribution relative to it. The one published NVFP4 RP quant used
# 64 samples of Opus-WritingPrompts at seq len 8192. Long sequences matter more
# than sample count here.
#
# Cost on your card: ~45-60 min for a 27B; GPU-trivial (layers onloaded one at
# a time), CPU-RAM-bound at roughly 2GB per 1B params -> ~55GB system RAM.
# llm-compressor does NOT support tensor parallelism for quantization.
export FLASHINFER_CUDA_ARCH_LIST=12.0f
export FLASHINFER_FORCE_SM=120f
export VLLM_NVFP4_GEMM_BACKEND=cutlass
vllm serve /models/rp-27b-nvfp4a16 \
--quantization compressed-tensors \
--kv-cache-dtype fp8 \
--max-model-len 32768 \
--gpu-memory-utilization 0.90 \
--enable-chunked-prefill \
--enable-prefix-caching \
--max-num-seqs 32
# NO --speculative-config initially. Add only after measuring
# draft_acceptance_rate at your real production temperature.
```
**Validation gates before you trust any of it:**
1. `grep` the startup log for `"does not have native support for FP4"` → if present you're silently on Marlin.
2. **KLD against the BF16 parent at 16k and 32k context**, not MMLU. This is the only test that catches the failure mode you care about.
3. If spec decode is on, log `draft_acceptance_rate` **at production temperature**. Below 0.5, turn it off.
4. Soak-test at real concurrency — the `flashinfer_cutlass` corruption is silent and load-dependent.
---
## 6. How I'd actually evaluate these
Nobody publishes spatial-coherence numbers for these models. Across the entire survey the only quantitative claims that exist are Gryphe's StyleTune slop metrics and zerofata's swipe-diversity numbers. **You will have to measure this yourself**, and it's not hard:
Build ~20 adversarial scenes that bait the specific failures you care about, run each model 5× per scene at your production sampler settings, and score:
- **Position tracking** — 3+ characters in a room, someone moves, someone leaves. Does the model place them correctly 10 turns later?
- **Clothing/object state** — an item is removed, moved, or destroyed. Does it reappear?
- **Anatomy/limb count** — the classic failure. Score explicit impossibilities.
- **Knowledge partition** — character A learns something in private. Does character B act on it? (OmniToM found "Knowledge Access" is the weakest dimension across all models at 56–75% macro-F1 — this is a real, measurable, near-universal weakness.)
- **Context depth** — run every test at 8k, 32k, and your real session length. Per §4, this is where the tunes will separate, and where none of them are trained.
- **Thinking on vs off, same seed, same scene.** Given §2, this is the highest-information single comparison you can run, and no published benchmark has done it for RP.
RP-Bench's own validation is a useful warning about scoring: LLM-judge methods showed **negative correlation** with community Bayesian Elo (ρ between −0.31 and −0.07), and its automated "Flaw Hunter" disagreed with human users more often than it agreed (50.7% vs 38.7%). **Use rule-based checks for state tracking** (did the model say "left hand" when the character's left arm was established as pinned?) rather than asking an LLM judge whether the scene was coherent.
---
## 7. What I could not verify
Stated plainly so you can weigh the rest:
- **Reddit is hard-blocked by this environment's egress policy** (403 on `reddit.com`, `old.reddit.com`, the JSON API, and domain-filtered search). The r/SillyTavernAI weekly megathreads are the single best source for practitioner reports on spatial coherence, and I got none of it. Everything here comes from HuggingFace, benchmark sites, papers, and blog coverage. **The community-consensus layer of this report is missing** — treat the rankings as evidence-based rather than user-validated.
- **No model card in this survey makes an affirmative spatial-coherence or state-tracking claim.** I checked all of them explicitly. What exists is MeroMero-v2's training-side coherence judge, and Magistry's *disclaimer*. Any source telling you these models advertise state tracking is fabricating.
- **Trinity-Mini-Ichthyo's card is unreadable** (gated, 401). It has the highest download count in-band and I can tell you nothing about it.
- **Artemis-31B-v1.1 has no license set** — no tag in the API, nothing in the README. Matters if this is going anywhere commercial.
- **The Qwen-27B-family exact parameter counts** were inconsistent across API calls (27,781,427,952 / 27,781,419,504 / 27,356,728,560 in mutually contradictory slots). The ~27.4B / ~27.8B magnitudes are safe; exact digits are not.
- **MeroMero-v2 stage 3 being "multi-turn"** — steps, thinking-enabled, and the DeepSeek-V4-Flash logic-defect judge are all confirmed verbatim; the multi-turn detail is not.
- **`heretic` does not preserve MTP natively.** I checked PyPI, GitHub, and the docs for any mention of MTP, auxiliary weights, or draft heads — absent from all three. The `Native-MTP-Preserved` repos are doing a manual post-hoc graft the tool doesn't do for you. Whether heretic 1.4.0 (Jun 2026) added passthrough is unverified.
- **UGI Leaderboard's live 2026 data** — the CSV is 653kB and only the first chunk is fetchable; the visible slice runs to Nov 2025. The 2026 entries (`Huihui-Qwen3-VL-32B-Thinking`, `Ayla-Light-v2`) are unverified.
- **EQ-Bench carries essentially no 15–32B RP finetunes** — only 9–12B Gemma derivatives. There is no Cydonia/MeroMero/Pantheon Elo, so cross-referencing UGI willingness against EQ-Bench writing quality is not currently possible for any model in this report.
- `arxiv.org/html/2607.22732` ("Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning") — rate-limited on 6 attempts. Likely the single most on-point paper for your question. Worth retrying.
---
## Sources
**Models:** [zerofata/G4-MeroMero-v2-31B](https://huggingface.co/zerofata/G4-MeroMero-v2-31B) · [zerofata/G4-MeroMero-26B-A4B](https://huggingface.co/zerofata/G4-MeroMero-26B-A4B) · [zerofata/Q3.5-BlueStar-v2-27B](https://huggingface.co/zerofata/Q3.5-BlueStar-v2-27B) · [Gryphe/Pantheon-Reasoning-27B](https://huggingface.co/Gryphe/Pantheon-Reasoning-27B) · [Gryphe/Pantheon-Reasoning-26B-A4B-1.1](https://huggingface.co/Gryphe/Pantheon-Reasoning-26B-A4B-1.1) · [Gryphe/Gemma-4-26B-A4B-StyleTune-V2](https://huggingface.co/Gryphe/Gemma-4-26B-A4B-StyleTune-V2) · [Gryphe/WorldSim-Opus-3.6-35B-A3B](https://huggingface.co/Gryphe/WorldSim-Opus-3.6-35B-A3B) · [allura-org/Qwen3.5-27B-Anko](https://huggingface.co/allura-org/Qwen3.5-27B-Anko) · [ReadyArt/Dark-Scarlett-v1.0-27B](https://huggingface.co/ReadyArt/Dark-Scarlett-v1.0-27B) · [ReadyArt/gemma-4-31B-it-scotoma-2](https://huggingface.co/ReadyArt/gemma-4-31B-it-scotoma-2) · [TheDrummer/Artemis-31B-v1.1](https://huggingface.co/TheDrummer/Artemis-31B-v1.1) · [TheDrummer/Skyfall-31B-v4.2](https://huggingface.co/TheDrummer/Skyfall-31B-v4.2) · [TheDrummer/Rocinante-XL-16B-v1](https://huggingface.co/TheDrummer/Rocinante-XL-16B-v1) · [sophosympatheia/Magistry-24B-v1.1](https://huggingface.co/sophosympatheia/Magistry-24B-v1.1) · [ArliAI/Qwen3.5-27B-RpRMax-v1](https://huggingface.co/ArliAI/Qwen3.5-27B-RpRMax-v1) · [Vortex5/G4-Moonlight-Dusk-26B-A4B](https://huggingface.co/Vortex5/G4-Moonlight-Dusk-26B-A4B) · [NewEden/Trinity-Mini-Ichthyo](https://huggingface.co/NewEden/Trinity-Mini-Ichthyo) · [Nimbz/Gemma-4-Gembrain-31B](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B) · [LatitudeGames/Equinox-31B](https://huggingface.co/LatitudeGames/Equinox-31B) · [llmfan46/…-Native-MTP-Preserved](https://huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved)
**Bases:** [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) · [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) · [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) · [google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it) · [arcee-ai/Trinity-Mini](https://huggingface.co/arcee-ai/Trinity-Mini) · [mistralai/Magistral-Small-2509](https://huggingface.co/mistralai/Magistral-Small-2509) · [Gemma 4 blog](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/) · [Mistral Small 4](https://mistral.ai/news/mistral-small-4/)
**Benchmarks:** [UGI Leaderboard](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard) · [EQ-Bench](https://eqbench.com/) · [Fiction.liveBench @ Epoch](https://epoch.ai/benchmarks/fictionlivebench) · [Fiction.liveBench data](https://raw.githubusercontent.com/mnismt/llms-long-context-benchmark/main/src/data/benchmark.ts) · [NCP-Bench (arXiv 2608.08160)](https://arxiv.org/abs/2608.08160) · [NarrativeWorldBench (arXiv 2606.17391)](https://arxiv.org/html/2606.17391v1) · [RP-Bench](https://github.com/LeviTheWeasel/rp-benchmark) · [PlotPoints](https://plotlightstudios.com/plotpoints) · [MuSR](https://benchlm.ai/benchmarks/musr) · [LongBench Pro (arXiv 2601.02872)](https://arxiv.org/html/2601.02872v1) · [SpatialEval](https://spatialeval.github.io/) · [OmniToM (arXiv 2605.26322)](https://arxiv.org/html/2605.26322) · [Visual vs Textual CoT (ACL 2026)](https://aclanthology.org/2026.alvr-main.1.pdf) · [Mind Your Step (arXiv 2410.21333)](https://arxiv.org/html/2410.21333v4) · [Thinking in Character (arXiv 2506.01748)](https://arxiv.org/html/2506.01748v1)
**Deployment:** [NVIDIA forum: W4A16 over W4A4](https://forums.developer.nvidia.com/t/update-for-nvfp4-model-conversion-to-use-w4a16-instead-of-w4a4/370403) · [Red Hat NVFP4 accuracy](https://developers.redhat.com/articles/2026/02/04/accelerating-large-language-models-nvfp4-quantization) · [NVIDIA NVFP4-QAD report](https://research.nvidia.com/labs/nemotron/files/NVFP4-QAD-Report.pdf) · [llm-compressor NVFP4 example](https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_w4a4_fp4/) · [llm-compressor Gemma 4](https://docs.vllm.ai/projects/llm-compressor/en/latest/key-models/gemma4/) · [ModelOpt hf_ptq](https://github.com/NVIDIA/Model-Optimizer/blob/main/examples/hf_ptq/README.md) · [vLLM #47749](https://github.com/vllm-project/vllm/issues/47749) · [vLLM #39407 (Gemma 4)](https://github.com/vllm-project/vllm/issues/39407) · [vLLM #40875](https://github.com/vllm-project/vllm/issues/40875) · [vLLM #25112](https://github.com/vllm-project/vllm/issues/25112) · [CUTLASS #3096](https://github.com/NVIDIA/cutlass/issues/3096) · [SGLang #19637](https://github.com/sgl-project/sglang/issues/19637) · [vLLM recipe Qwen3.6-27B](https://recipes.vllm.ai/Qwen/Qwen3.6-27B) · [DigitalOcean spec-decode guide](https://www.digitalocean.com/community/tutorials/speculative-decoding-vllm-configuration-guide) · [vLLM EAGLE 3.1](https://vllm.ai/blog/2026-05-26-eagle-3-1) · [Why quantized LLMs lose MTP heads](https://dev.to/alanwest/why-your-quantized-llm-loses-its-mtp-heads-and-how-to-keep-them-m7h) · [lna-lab GGUF-to-NVFP4-SM120](https://github.com/lna-lab/GGUF-to-NVFP4-SM120) · [rtx6kpro NVFP4 guide](https://github.com/local-inference-lab/rtx6kpro/blob/master/optimization/nvfp4-quantization.md) · [Jarvislabs NVFP4 on RTX PRO 6000](https://jarvislabs.ai/blog/nvfp4-rtxpro-6000) · [Millstone Gemma-4-31B NVFP4](https://www.millstoneai.com/inference-benchmark/gemma-4-31b-nvfp4-1x-rtx-pro-6000-blackwell) · [loFT Qwen3.6-27B NVFP4+MTP](https://loftllc.dev/en/docs/tech/llm-research/qwen3-6-27b-nvfp4-mtp-vllm-benchmark/) · [Unsloth Dynamic NVFP4](https://unsloth.ai/docs/basics/nvfp4) · [Benjamin Marie NVFP4 vs INT4](https://medium.com/data-science-collective/nvfp4-same-accuracy-with-2-3x-higher-throughput-for-4-bit-llms-03518ecba108) · [heretic-llm](https://pypi.org/project/heretic-llm/)