esh-pfi-infrastructure/docs/pfi/erp-thinking-finetunes-2026.md

Thinking-Capable eRP Finetunes, 15–30B — Deep Research

Compiled 2026-08-12 · Window: Feb–Aug 2026 · Weighted for spatial/state coherence · Target: RTX PRO 6000 Blackwell (sm_120), NVFP4, throughput

---

0. Read this first — three findings that should change your shortlist

1. The 24B Mistral era is over. Everything worth running in this band now sits on one of four bases, all of which ship native thinking out of the box: Qwen3.6-27B (Apr 2026), Qwen3.5-27B (Feb 2026), Gemma-4-31B / Gemma-4-26B-A4B (Mar 31 2026, now Apache 2.0), and arcee-ai/Trinity-Mini (26B-A3B). Mistral has shipped nothing in your band in 2026 — Mistral Small 4 is a 119B-A6B MoE that absorbed the Magistral line. Magistral-Small-2509 (Sep 2025) is still the newest in-range Mistral reasoning model, and the 24B tunes built on it are now a legacy tier.

2. The evidence says heavy eRP finetuning actively damages the thing you care about most. This is the uncomfortable core of this report and it's covered in §2. Short version: reasoning-native models buy real long-context state tracking, but bolting RP-tuning and reasoning-tuning on top degrades both prose and world-modeling. The single most respected merger in the space says flatly that 24B "will struggle with details of logical/physical continuity at times — which is probably inescapable for a 24B model." If spatial coherence is your #1 criterion, bias toward light-touch tunes on smart bases, not heavy eRP tunes.

3. MTP and best-in-class RP tuning are currently mutually exclusive — with exactly one escape hatch. Every dedicated RP brand (Cydonia, Skyfall, Dark-Scarlett, MeroMero, Artemis, Magistry) sits on Mistral or Gemma bases that have no MTP heads at all. Only Qwen3.5/3.6-27B ships MTP in your band — and from_pretrained silently drops the MTP heads during finetuning, so almost every Qwen-based community tune has lost them too. The escape hatch is the Native-MTP-Preserved lineage (§5.2), which grafts the 15 MTP tensors back post-hoc, and already has NVFP4 checkpoints.

Also worth knowing up front: at temp 0.8–1.2 (normal RP sampling), speculative decoding acceptance collapses to ~38–52%, and vLLM's own guidance is to disable it below 0.5. On a shared, batched box it is likely a net throughput loss. Details and the one contradicting measurement in §5.4.

---

1. Ranked picks

Ranked for spatial/state coherence first, prose second, with your NVFP4 + throughput constraints factored in.

#	Model	Params	Base	Thinking	NVFP4 today?	MTP?
1	zerofata/G4-MeroMero-v2-31B	31.27B	Gemma-4-31B	Dual (Think/NoThink presets)	v1 only — must quantize v2	✗
2	Gryphe/Pantheon-Reasoning-27B	~27.8B	Qwen3.6-27B (MTP-preserved heretic)	Always-on	✗ — must quantize	✗ (re-graftable)
3	llmfan46/…-Native-MTP-Preserved-NVFP4	~27.8B	Qwen3.6-27B	Native	✓ shipped	✓ intact
4	allura-org/Qwen3.5-27B-Anko	~27.4B	ArliAI Qwen3.5-27B-Derestricted	Dual-mode (trained both ways)	✗	✗
5	ReadyArt/Dark-Scarlett-v1.0-27B	~27.8B	Qwen3.6-27B	enable_thinking flag	✗ (W4A16/W8A16 PTQ only)	✗
6	TheDrummer/Artemis-31B-v1.1	31.27B	Gemma-4-31B	Dual + custom tags	✗	✗
7	Gryphe/Pantheon-Reasoning-26B-A4B-1.1	26.5B MoE (A4B)	Gemma-4-26B-A4B	Always-on	✗	✗
8	zerofata/G4-MeroMero-26B-A4B	25.8B MoE (A4B)	Gemma-4-26B-A4B	Dual	✓ (2 quantizers)	✗
9	sophosympatheia/Magistry-24B-v1.1	23.6B	Magistral-2509-24B	<think> prefill	MLX only	✗
10	zerofata/Q3.5-BlueStar-v2-27B	~27.4B	Qwen3.5-27B	<think>\n prefill (trained)	MLX only	✗

Wildcard worth a slot on your test rig: Gryphe/WorldSim-Opus-3.6-35B-A3B — 35B-A3B, over your band but only ~3B active so it's cheap. It is the closest thing anyone has built to a model designed for the state-tracking problem: trained on three datasets that all carry full thinking traces, with reasoning persisting per-turn. The author calls it a research release whose "practical effectiveness remains uncertain."

Actively avoid for your criterion: LatitudeGames/Equinox-31B — card states verbatim "No reasoning datasets were included during training," thinking suppressed by default. TheDrummer/Rocinante-XL-16B-v1 — user reports of degradation past 16k and noticeable decline past 20k; you can't track scene state in a window that small.

---

2. Does thinking actually help spatial coherence? — the evidence

This deserves its own section because the answer is "yes for state tracking, no for prose, and only if the model was pretrained for reasoning."

2.1 Thinking clearly helps long-context state tracking — for reasoning-native models

• Fiction.liveBench (narrative comprehension, theory of mind, chronological reasoning at length) is the single strongest datapoint. At 16k context: QwQ-32B 83.3% vs Gemma-3-27B 33.3% vs dolphin-Mistral-24B 25.0% — a reasoning-native 32B beating a 70B non-reasoning model (Llama-3.3-70B, 33.3%) by 50 points. Same-model toggle: claude-3-7-sonnet thinking 83.3% vs non-thinking 50.0% at 16k. [data] [Epoch]
• LongBench Pro (8k–256k, includes consistency-checking and dialogue-tracking): thinking mode adds +11 to +16 points for reasoning-native models (Claude-4-Sonnet 56.07→69.87; DeepSeek-V3.2 51.67→67.82). But models not trained for thinking gain nothing — Llama-3.1-405B +0.59, Gemma-3-12B −0.24. Paper's own conclusion: "models without thinking training may fail to effectively leverage test-time compute." [arXiv 2601.02872]
• MuSR (multi-step narrative state tracking): Ministral 3 14B Reasoning 70% vs base 64%; consistent +6 to +9 at every size down to 1.2B. [BenchLM]
• UGI "World Model" column, same-model toggles: Qwen3-32B 21.25 → 23.80, Qwen3-30B-A3B 13.10 → 16.67 with thinking on.

2.2 Thinking reliably damages prose and destroys instruction-following

Every same-model pair in the UGI dataset shows the Writing score dropping when thinking is on: Qwen3-14B 34.76 → 29.64, Qwen3-32B 32.95 → 30.34, Qwen3-30B-A3B 30.24 → 28.54, Qwen3-8B 27.96 → 23.87. gpt-oss-20b degrades monotonically with reasoning effort — Writing 24.62 (low) → 24.50 (med) → 10.94 (high) with repetition interrupts rising 2 → 1 → 8.

The instruction-following collapse is the most reproducible effect in the entire dataset. creative_writing_wc_exceeded_pct — the share of creative tasks where the model blew the requested word limit:

Model	Thinking off	Thinking on
Qwen3-14B	1%	99%
Qwen3-32B	0%	100%
Qwen3-30B-A3B	10%	99%
Qwen3-8B	4%	100%

If you've ever wondered why a thinking model ignores your "keep replies to two paragraphs" instruction — that's this.

2.3 The warning case: bolting reasoning onto an RP finetune

Cydonia-R1-24B-v4 vs Cydonia-24B-v4 — same trainer, same base lineage, one reasoning-tuned:

Metric	Cydonia-24B-v4	Cydonia-R1-24B-v4
Writing	30.91	20.38 (−34% rel.)
World Model	23.30	19.33 (−17%)
NatInt	26.64	24.27
Length error	22%	80%
W/10 (willingness)	7.8	8.2 ✓

Reasoning-tuning bought willingness and cost everything else, including the world-model score. Caveat: separate training runs, not a toggle, so recipe differences are confounded. But it's the closest analogue to "what happens when an RP finetuner adds thinking."

2.4 Mechanistic support for why

• Visual vs Textual CoT diagnostic (ACL 2026): textual chain-of-thought degrades spatial transformation by up to 16.5% and multi-object tracking by 12.7% vs direct answering, measured across GPT-5, Claude Opus 4.6, Gemini 2.5 Pro, Qwen3-VL-72B. [pdf] That is literally your criterion, and CoT made it worse.
• "Mind Your Step (by Step)": CoT reduces performance on implicit statistical learning by up to −36.3% absolute, framed as verbal overshadowing — narrating a scene in a scratchpad makes the model worse at feeling the scene. [arXiv 2410.21333]
• Contrary evidence worth weighing — "Thinking in Character" found role-aware reasoning beats naive reasoning (CharacterBench 3.69 RAR vs 3.57 distill), but note the third term: undirected extra thinking scored worst at 3.05. The claim is not "reasoning helps," it's "reasoning helps only if its style is constrained to the character." [arXiv 2506.01748]

2.5 And at the frontier, reasoning doesn't fix narrative consistency at all

• NarrativeWorldBench: frontier + reasoning models all cluster at F1 0.78–0.81 at horizon 50 with no significant difference (p>0.13); everything loses ~0.20 F1 from h=10 to h=200. A purpose-built 8B latent world model holds F1 ≥ 0.84 across all horizons at ~4× lower cost. [arXiv 2606.17391]
• NCP-Bench (Aug 2026) is the benchmark you were hoping existed — it explicitly scores spatial consistency ("character described on the bridge later appearing in a doorway"), object state tracking ("a raft inflated→deflated without justification"), and character knowledge leakage. Results are humbling: GPT-5.2 survives 20 turns only 42% of the time, near-zero survival by 100 turns, fact conflicts at 40–68% across all models. It tests no sub-32B models. [arXiv 2608.08160]
• RP-Bench found reasoning models (GLM 5.1, Gemini 3.1 Pro, Kimi K2.5/K2.6) underperformed frontier non-reasoning models on roleplay dimensions, with severe latency costs (Kimi K2.6 p95 173s, 17% truncation at length limit — truncation is itself a coherence failure). Its verdict on the category: "The RP-specialist finetunes — the models marketed for exactly this — rank last." [repo]

2.6 What I'd actually do with this

The defensible synthesis: use thinking sparingly and structurally, not as an always-on prefix to prose. A gated pattern — reasoning enabled for scene-state checks, scene transitions, and complex multi-character blocking; disabled for straight prose continuation — captures the state-tracking gain without paying the prose and length-adherence tax. Every model in §1 that supports dual mode (MeroMero, Artemis, BlueStar, Dark-Scarlett) lets you do this at the request level. The always-on models (Pantheon, WorldSim) do not.

---

3. Per-model breakdowns

Metadata below is from the HuggingFace API, verified individually. Download counts are trailing-30-day and are unreliable as a quality signal — most users pull the GGUF mirror repos, not the BF16 originals.

3.1 zerofata/G4-MeroMero-v2-31B — best-shaped training for your criterion

huggingface.co/zerofata/G4-MeroMero-v2-31B · 31.27B · Gemma-4-31B · Apache-2.0 · 2026-08-03 · 258 dl / 43 likes

The reason this is #1: it is the only model in the entire survey whose training explicitly optimizes reasoning against a coherence judge. Verbatim from the card, the pipeline is SFT > Merge > GRPO > GRPO > on-policy SFT:

1. Diversity SFT — ~4,000 curated stories, 0.5 blend merge-back
2. Creative GRPO — 8 rollouts/prompt, 300 steps, thinking disabled
3. RP Logic GRPO — 100 steps, thinking enabled, scored by "a logic-defect judge (DeepSeek-V4 Flash with a rubric)", with a reward_judge_coherence reward term
4. On-policy SFT — ~3,300 self-generated RP samples, diversity-filtered

Stage 3 is the mechanism that should produce state tracking. Honest caveat: the card does not claim improved spatial coherence as an outcome, and I could not confirm the stage-3 prompts were multi-turn (an earlier source claimed this; it's unverified). You're buying a plausible training signal, not a measured result.

Author's own metrics vs stock Gemma 4: swipe diversity 0.72 vs 0.43, story slop 7.4 vs 8.8 per 1k words, bare-prompt attractor hit rate 66% vs 99%, no regression on IFEval / GSM8K / MMLU-Pro.

• Thinking: dual, via Gemma4-Think.json / Gemma4-NoThink.json SillyTavern presets. Reasoning is longer than stock Gemma 4, shorter than MeroMero v1.
• Samplers: temp 0.8–1.0, MinP 0.05
• Quants: GGUF (official + mradermacher), FP8 W8A16 (hoborific), exl3, MLX. NVFP4 exists only for v1 (pekkAi, heretic variant). You'll quantize v2 yourself.
• Note: 31.27B is marginally over your stated band. There is a true in-band sibling, G4-MeroMero-26B-A4B (25.8B MoE, A4B, May 2), which does have NVFP4 (Deaquay, pekkAi heretic) and claims "reasoning is more structured, using less tokens during RP." But the 26B's card is candid that "logic and repetition I think are roughly on par with the original" — v2-31B is where the coherence work actually happened.

3.2 Gryphe/Pantheon-Reasoning-27B — best methodology, and it sits on the MTP-preserved base

huggingface.co/Gryphe/Pantheon-Reasoning-27B · ~27.8B · Apache-2.0 · 2026-05-30 · 232 dl / 27 likes

Base is llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved — verified, and that matters enormously for your rig (§5.2).

Two things make this the most methodologically interesting tune in the set:

• Always-on reasoning. Verbatim: "The model was trained with preserve_thinking: true, so thinking tags remain active across all assistant turns in multi-turn conversations, not just the first." Almost every other model reasons once and then stops.
• The thinking traces were generated as planning, not annotation. DeepSeek 3.2 produced them under the instruction to "think as a writer planning their next response — before writing — rather than annotating a response," then judge-model validated. This is the "role-aware reasoning" pattern that the CharacterBench work found is the only kind that helps.

Data mix: Pantheon RP corpus ~28%, Opus-4.6-Reasoning-24k ~21%, WorldSim narrative ~16%, text adventure/IF ~16%, general RP ~16%, Tiamat ~3%.

• Samplers: temp 1.0, rep_pen 1.0, min_p 0.05. The rep-pen point is emphatic and now consensus among reasoning-RP authors: repetition penalties corrupt thinking content. Any thinking model whose card recommends rep_pen > 1.0 is a red flag.
• Template: ChatML (Qwen3.6 chat template)
• Author's own framing: a research release, with the stated open question being "does reasoning actually help roleplay, or does it just add latency?" Respect that honesty.
• Quants: GGUF only. No NVFP4, no FP8. You will quantize this one.
• Sibling: Pantheon-Reasoning-26B-A4B-1.1 (26.5B MoE, Gemma-4-26B-A4B, Jun 8) — same methodology, stricter trace QA, genuinely in-band, and the most-reused merge donor in the whole 26B-A4B ecosystem. SillyTavern gotcha: character-name prefixes break reasoning compatibility on this one — disable them.

3.3 llmfan46 Native-MTP-Preserved (NVFP4) — the throughput play

huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-NVFP4

Not an RP finetune — a decensored Qwen3.6-27B. It's on this list because it's the only 15–30B option that is simultaneously NVFP4, MTP-intact, and uncensored, and because §2 argues that a smart, lightly-touched base may outperform a heavy eRP tune on exactly the axis you're prioritizing.

Parent repo: 7,580 dl / 41 likes, created May 6, modified May 25. Made with Heretic v1.3.0 using a variant of Magnitude-Preserving Orthogonal Ablation (MPOA), ablating only attn.o_proj, attn.out_proj, mlp.down_proj. Claimed: 94% fewer refusals (6/100 vs 92/100) at 0.0021 KL divergence, MMLU 85.67% vs 86.65% original.

The load-bearing detail is model-auxiliary.safetensors in the repo — that's where Qwen stores the MTP heads, and its presence is hard proof the claim isn't marketing. The card enumerates all 15 preserved tensors.

Pair it with a style fix. Its weakness vs a proper RP tune is voice, not intelligence. Gryphe/Gemma-4-26B-A4B-StyleTune-V2 demonstrates the approach on the Gemma side and is the most quantitatively-supported claim in this whole survey: it trains precisely one tensor — "the lm_head output projection… freeze everything else. All 30 transformer layers, all the attention heads, all the MLPs — completely untouched" — and measures 52% fewer clichés per 100 words (1.141 → 0.551) over 200 RP prompts with only 19.9% shared trigram vocabulary. Reasoning capability is untouched by construction. There's no Qwen equivalent published yet, but the recipe is simple enough to replicate.

3.4 allura-org/Qwen3.5-27B-Anko

huggingface.co/allura-org/Qwen3.5-27B-Anko · ~27.4B · Apache-2.0 · 2026-04-08 · 40 dl / 11 likes

Correction to circulating claims: the base is ArliAI/Qwen3.5-27B-Derestricted, not stock Qwen3.5-27B. LoRA r=64 / α=512 on Doubao Seed 2.0 Pro reasoning traces, trained on both reasoning and non-reasoning responses, so it's dual-mode by construction. Stated goal, verbatim: "increase the quality of reasoning and decrease looping, and fix slop in outputs."

Why it ranks well for you: Qwen3.5-27B is the best state-tracking base in the band by measurement — MuSR 95, the best open-weight score overall, and LongBench v2 60.6%.

• Samplers, verbatim and shouted: "DO NOT USE QWEN'S SAMPLERS. THEY ARE AWFUL." Use temp 1.25, min_p 0.05–0.1.
• Odd but documented: recommended system prompt is You are Claude, a helpful and harmless language model created by Anthropic. It was trained to work with Claude-style system prompt formatting.
• Quants: GGUF only (bartowski, mradermacher). No NVFP4/FP8/AWQ/exl3.
• Warning: ArliAI's Derestricted line drops MTP — I verified the file manifest, there is no model-auxiliary.safetensors. So Anko has no MTP.

3.5 ReadyArt/Dark-Scarlett-v1.0-27B — cleanest eRP with flag-based thinking

huggingface.co/ReadyArt/Dark-Scarlett-v1.0-27B · ~27.8B · Qwen3.6-27B · Apache-2.0 (personal use, 18+) · 2026-06-16

The most explicitly eRP-targeted model here with a properly documented thinking toggle:

chat_template_kwargs: {"enable_thinking": true, "reasoning_effort": "medium"}

That reasoning_effort knob is unusually useful for the gated-thinking pattern in §2.6 — you can dial it per-request rather than binary on/off.

Training: LoRA r=32, 2 epochs, text layers only, on 12,211 curated adult-RP prompts, with multi-turn generation, refusal filtering, and group-chat support in the pipeline.

• Samplers: top_p 0.92, temp 1.0, freq_pen 0, pres_pen 0
• Real limitation: the card states it's optimized for Male(user)→Female(AI) perspective. Narrow.
• Quants: GGUF + ReadyArt's own W4A16/W8A16 PTQ. No NVFP4, no FP8.
• Family context: ReadyArt shipped a dense June burst — Dark-Scarlett-v2.0-31B (Gemma-4), v1.0-26B-A4B, v1.0-31B, v0.4-2509-24B, Heimdallr-v0.02-31B. Download signal favors the MoEs.

3.6 TheDrummer/Artemis-31B-v1.1 — freshest, longest bake

huggingface.co/TheDrummer/Artemis-31B-v1.1 · 31.27B · Gemma-4-31B · 2026-08-06 · 7 likes · no license set

Four months of public iteration through BeaverAI test builds (v1a Apr 8 → v1n Jul 22), which is unusually thorough for this scene. Use v1.1, not v1 — v1 has "strong writing potential but requires manual adjustments"; v1.1 "improves stability while maintaining v1's creative strengths," specifically fixing "dash spiraling."

• Thinking: the most flexible activation of any model here — "standard thinking gemma template or <thinking></thinking> blocks on non-thinking gemma template," and "<think></think> should work too, along with tricks like <evil_think></evil_think>."
• Samplers: not fixed in the card; Drummer points to a crowdsourced sampler spreadsheet.
• Too new for consensus as of Aug 12 — one enthusiastic but content-free feedback thread.
• Predecessor if you want something proven: Skyfall-31B-v4.2 (Apr 3, Magistral-Small-2509 upscaled, Mistral v7 Tekken template) is the established workhorse of this window and has an NVFP4 quant already (ealexeev, v4.1).

3.7 sophosympatheia/Magistry-24B-v1.1 — the honest one

huggingface.co/sophosympatheia/Magistry-24B-v1.1 · 23.6B · Apache-2.0 · 2026-03-22 · 35 likes (highest like count in-band)

A mergekit DELLA merge (not a finetune) on Darkhn/Magistral-2509-24B-Text-Only, so it inherits Magistral's native reasoning. Donors: Casual-Autopsy/Maginum-Cydoms-24B, DarkArtsForge/Magistaroth-24B-v1, plus Huihui-Devstral-Small-2-24B-Instruct-2512-abliterated at 0.3.

I'm listing it partly because its card contains the single most on-point statement anyone in this scene has made about your criterion, verbatim:

"This model is fun, but it will struggle with details of logical/physical continuity at times — which is probably inescapable for a 24B model."

That is a respected merger saying 24B sits below the threshold where physical continuity holds. Take it seriously as a floor: if spatial coherence is your top priority, 27B+ is the entry point, not 24B.

• Thinking: prefill-based — force the reply to start with <think> plus basic instructions. Card notes <think></think> works better than Mistral's [THINK][/THINK] tags. (Related gotcha: on Mistral models <think> is not a special token; [THINK] is.)
• Samplers: three named presets — Conservative (temp 0.7, MinP 0.05, Top-N σ 0.75), Balanced (temp 1.0, Adaptive-P target 0.6 / decay 0.9), Wild (temp 0.9, Adaptive-P target 0.35 / decay 0.45). Also ships a SillyTavern Master Import JSON.
• It is NOT gated (a claim to the contrary is circulating; the API says gated: false).
• Quants: GGUF, exl3, MLX MXFP4/MXFP8. No NVFP4.

3.8 zerofata/Q3.5-BlueStar-v2-27B — best-documented anti-slop SFT

huggingface.co/zerofata/Q3.5-BlueStar-v2-27B · ~27.4B · Qwen3.5-27B · MIT · 2026-03-20 · 42 likes

The interesting technical contribution here is custom loss masking on slop phrases — "most common phrases of slop are masked out, so the model doesn't get rewarded for learning these patterns." That lets you train on otherwise-useful RP data without absorbing its clichés. SFT ~27M tokens via Axolotl + LoRA on 4×H200.

• Thinking: prefill <think>\n — and importantly, "it is required to prefill the <think>\n as that is how it was trained." This is a trained-for prefill, not a bolted-on hack. Ships separate think/no-think ChatML instruct JSONs.
• Samplers: temp 0.8–1.0, MinP 0.05–0.075
• ⚠️ Trained at 10,756 token sequence length despite the 262k base. See §4 on why this matters more than anything else in the card.
• Quants: GGUF. The two "NVFP4" BlueStar repos you'll find are MLX (Apple silicon) — useless on Blackwell.

3.9 Also verified, lower priority

• Vortex5/G4-Moonlight-Dusk-26B-A4B (26.5B MoE, Jul 14, 1016 dl) — merge of Animus-V14.1-FFT + G4-MeroMero-26B-A4B + Esmeralda + Pantheon-Reasoning-26B-A4B-1.1. Highest download count of the Gemma-4 MoE merges. Thinking activation is undocumented — merge card only, no sampler guidance. Good candidate, poor paperwork.
• ArliAI/Qwen3.5-27B-RpRMax-v1 (Apr 28) — successor to the well-regarded QwQ-32B-ArliAI-RpR line, in a collection literally titled "Thinking-trained RP specialized models." Confirmed to have no model card at all — training method, datasets, template, samplers, context all unverified. Heavy third-party GGUF activity (bartowski et al.) suggests real pickup. High risk, possibly high reward.
• NewEden/Trinity-Mini-Ichthyo (26.1B-A3B, Jul 10, 2,489 dl — highest of any in-band RP repo) — trained with actual RL (Prime RL run, step-100 checkpoint, 32,768 ctx). Base is NewEden/Trinity-Mini-Futaba, not stock Trinity-Mini. Gated behind a contact-info agreement and the README returns 401 — I could read nothing. Zero third-party quants, consistent with the gating. Interesting, unassessable.
• Nimbz/Gemma-4-Gembrain-31B (~Aug 2) — 5-phase Gemma-4 merge, <|think|> reasoning, targets "enhanced logical and lateral thinking." Samplers: temp 1.0, Top-P 0.95, Min-P 0.03, DRY 0.8/1.75. Trending but unproven.
• ReadyArt/gemma-4-31B-it-scotoma-2 (Aug 6) — not an RP tune, the most rigorous anti-slop work of the window: γ-fold refusal-edit projection + 3 rounds of preference training on 9.3k pairs. Measured over 480 RP continuations: stacked adjectives ↓21×, "Not X. But Y." ↓4×, em-dash asides ↓4×. ⚠️ Explicitly "not uncensored" — refusal behavior matches base. Useful as a merge donor or style reference, not as a driver.

3.10 Confirmed dormant — stop waiting on these

Checked directly; no 2026 releases in this band: anthracite-org / Magnum (last: Nov 2024) · Sao10K (Mar 2025) · Nitral-AI (Sep 2025) · PocketDoc / Dans-PersonalityEngine (May 2025) · Undi95 (Mar 2025) · aixonlab (May 2025) · knifeayumu (Aug 2025) · TareksLab (70B only, Aug 2025) · Doctor-Shotgun (quant-only in 2026) · Delta-Vector (moved to 399B Trinity-Large) · inflatebot · Tesslate (never RP).

Steelskull correction: Steelskull/CWT-V5.6 (Apr 2026) is not an RP model — it's "Cognitive Workspace Transformer," a 57.8M-parameter from-scratch research architecture trained on FineWeb-Edu. Steelskull's RP line (Electra / Nevoria / Broken-Tutu) has shipped nothing since L3.3-Shakudo-70B in Jul 2025.

One to watch: TheDrummer/Orion-26B-A4B exists only as BeaverAI test builds (v1a May 24 → v1c Jul 10). Dead center of your band. Likely the next official release after Artemis.

---

4. The thing nobody puts in the headline: training context length

This is buried in the model cards and it undercuts a lot of the spatial-coherence story:

Model	Base context	Actually trained at
Q3.5-BlueStar-v2-27B	262k	10,756 tokens
MS3.2-PaintedFantasy-v4.1-24B	128k	10,756 tokens
Trinity-Mini-Futaba	128k	32,768 tokens
Rocinante-XL-16B-v1	—	user reports drift past 16–20k

You cannot track scene state across a 60k-token roleplay with a model whose RP behavior was only ever reinforced at 10k. Base-model long-context ability degrades gracefully in benchmarks, but the RP-specific behavior these tunes install has a much shorter effective horizon. When you evaluate, test at your real session length, not at 8k. This is probably the highest-leverage thing in this report that no leaderboard captures.

Related: Gemma-4 degrades far more gracefully with context than Qwen3.6 on throughput — 32k→128k loss of −32% vs Qwen3.6-35B-A3B's −65% (dual RTX 4070 Ti). That's throughput only, not accuracy, but it's consistent with the architecture: Gemma-4 is full-attention dense; Qwen3.5/3.6 are hybrid Gated-DeltaNet linear-attention designs (3 linear blocks per 1 full-attention block), which are theoretically weaker at exact long-range state tracking despite the bigger advertised window.

---

5. Deployment on your rig

5.1 NVFP4 on sm_120 — the headline is W4A16, not W4A4

Do not ship plain W4A4 NVFP4 for long-context RP. NVIDIA's own guidance flipped to recommending W4A16 (NVFP4A16) for sm_120/121, citing KLD 2–4× worse for W4A4, "especially past ~10K context where activation quantization noise compounds with KV-cache lookups." [NVIDIA forum] That is precisely the failure mode you'd care about and it's the only source I found measuring KLD rather than MMLU at RP-relevant context lengths.

Cheap experiment: NVFP4 weight storage is identical between W4A4 and W4A16 — only the activation scales differ. Flipping is a config.json patch (set config_groups.group_0.input_activations to null), not a re-quantization.

The tension you should be aware of: W4A16 gives up the FP4 tensor-core compute path, so the gain becomes pure weight-compression/bandwidth — and Benjamin Marie's comparison found NVFP4A16 shows minimal throughput gain over INT4 AWQ, with AWQ/AutoRound scoring slightly better on accuracy and ~7GB smaller on disk. The counterargument for your box: freed VRAM converts to KV cache, which converts to concurrency, which is what you actually want on a shared rig.

Quality at 24–32B — the size gradient is real. Red Hat's aggregate NVFP4 recovery: 70B–235B ~99%, ~30B 97–99%, 7B–14B ~95–98%. Per-model, the damage concentrates in reasoning: Qwen3-32B-NVFP4 scores 99.83% OpenLLM v1 but only 94.21% reasoning avg; Qwen3-14B drops to 91.45% reasoning, 86.34% on AIME24. NVIDIA's own QAD report states it plainly: "for small LLMs, the accuracy drop from PTQ is often non-negligible."

sm_120-specific caveats (all confirmed against upstream issues):

• Silent Marlin fallback. Backend selectors check is_device_capability(100) only; sm_120 fails and falls back to Marlin dequant, logging "Your GPU does not have native support for FP4 computation." vLLM #47749 was still open as of Jul 6 2026. Always grep your startup log for that warning — if it's there, the whole exercise is moot.
• Dense is the healthy path. CUTLASS #3096 explicitly states dense FP4 GEMM works correctly on sm_120; the broken path was grouped (MoE) GEMM. Nearly every sm_120 NVFP4 horror story you'll read is a MoE story. This is a real argument for dense 27B over 26B-A4B MoE on your hardware, at least until the FlashInfer 0.6.5 / compute_120f path is more settled.
• compute_120f (needs CUDA 13.0) vs compute_120a: ~2.7× throughput difference (39.0 vs 14.6 tok/s in the CUTLASS issue's own table).
• flashinfer_cutlass has a reported race condition causing silent memory corruption at high concurrency; flashinfer_cudnn is reported safer. Directly relevant to you as a multi-tenant operator — toy prompts won't surface it, only soak testing will.
• FP8 KV cache is not universally safe on sm_120 (GLM-5 requires BF16 KV). Test yours.

Env vars people actually set:

export FLASHINFER_CUDA_ARCH_LIST=12.0f
export FLASHINFER_FORCE_SM=120f
export VLLM_NVFP4_GEMM_BACKEND=cutlass

Toolchain choice matters more than it looks: llm-compressor emits compressed-tensors but does not calibrate KV-cache scales by default, so you fall back to BF16 KV — 2× KV memory, roughly half the concurrent sessions. ModelOpt emits per-layer k_scale/v_scale and gets you real FP8 KV. On a shared box that's the deciding factor.

5.2 MTP — the one lineage that keeps it

The failure chain is three-deep and every stage is silent:

1. Loading. Qwen3_5ForConditionalGeneration.from_pretrained drops the MTP heads. Finetune → save_pretrained → heads gone, no warning. I verified ArliAI's Derestricted and RpRMax file manifests: no model-auxiliary.safetensors, no MTP tensors. This is why almost no community Qwen tune has MTP.
2. Quantization. Converters use allowlists and skip unknown tensor prefixes silently; GPTQ-style quantizers preserve the weights but never calibrate them, leaving effectively random values.
3. Serving. Even when present, mtp.* / mtp.fc must be in quantization_config.ignore or vLLM runs a quantized MTP head against differently-scaled activations.

The fix is unglamorous: copy the 15 MTP tensors out of the original Qwen/Qwen3.6-27B checkpoint and graft them onto your output shard. Published pipelines: lna-lab/GGUF-to-NVFP4-SM120 and AEON-7's variant. This means you can graft MTP back onto Pantheon-Reasoning-27B, since it descends from an MTP-preserved base — probably the single highest-value move available to you.

Two caveats on grafted MTP for eRP specifically:

• You're bolting the base model's draft head onto a finetuned target. Acceptance drops by however much your finetune moved the distribution — for an RP tune, a lot.
• The rtx6kpro notes warn explicitly: "abliterated models: MTP heads were trained on censored content; avoid with abliterated models." The head predicts what the aligned model would say, so acceptance collapses precisely on the content that differs. Mechanism is sound; generality is my inference.
• They also measured MTP causing a −22% throughput regression on sm_120 when Marlin fallback was active, because the draft heads expect native FP4 activations.

5.3 Existing NVFP4 checkpoints of RP finetunes — more than you'd expect

Two quantizers specialize in exactly this:

• ealexeev — a pure TheDrummer shop, 9 repos, ships recipe.yaml in-repo so the recipe is reproducible: Skyfall-31B-v4.1, Cydonia-24B-v4.3, Snowpiercer-15B-v4, Magidonia-24B-v4.2.0
• Firworks — ~100 NVFP4 repos incl. Cydonia-24B-v4.3-heretic, Magidonia-24B-v4.3, WeirdCompound-v1.7-24b
• AEON-7 — the MTP-grafting specialists. ModelOpt 0.43.0, NVFP4_DEFAULT_CFG, 15 MTP tensors grafted post-quantization, GatedDeltaNet layers kept BF16 (432 keys across 48 GDN layers), calibrated on neuralmagic/calibration 20 samples × 8192 tokens. Publishes an RTX PRO 6000 number: 92 tok/s median, 124.7 peak, 67.7% acceptance.
• sakamakismile — highest volume (~57 repos), explicit -MTP naming convention, incl. actual creative tunes: Carnice-V2-27b-NVFP4-TEXT-MTP, Qwen3.6-27B-Fable-Fusion-MTP-NVFP4. Also ships DSv4-Flash-FP8-SM120-Configs.

Gemma-4 NVFP4 works — the catastrophic vLLM bug (#39407, logits saturating at the bf16 softcap ceiling and emitting " a a a a" forever) is in the FP8_BLOCK path, not NVFP4. Existing Gemma-4-finetune NVFP4 checkpoints: pekkAi/G4-MeroMero-31B-NVFP4, AEON-7/Gemma-4-31B-it-DECKARD-HERETIC-Uncensored-NVFP4, Deaquay/G4-MeroMero-26B-A4B-NVFP4. Gemma-4 quirks: exclude vision tower / embed_vision / multi_modal_projector, and note heterogeneous attention head dims (head_dim=256, global_head_dim=512) need multi-group KV support if you use spec decode. Gemma-4 has no MTP — spec decode there is EAGLE-based.

5.4 Speculative decoding at RP temperatures — probably don't

Measured acceptance vs temperature [DigitalOcean vLLM guide]:

Temperature	Acceptance
0.0	~81%
0.4	~71%
0.8	~52%
1.0	~38%

The stated rule: below 0.5 acceptance, spec decode is net-negative. Your RP sampling sits at 0.8–1.25.

Corroborating, from AEON-7's own Qwen3.5-27B NVFP4 card with a DFlash drafter: greedy ~80% acceptance → ~91 tok/s; sampled ~5% acceptance → ~38 tok/s against a ~50 tok/s no-spec baseline. That's a ~24% throughput loss from turning it on.

And batching compounds it: spec decode gives 1.5–2.8× at low QPS but 1.4–1.8× slowdown at high QPS when the GPU is compute-saturated. Every impressive DFlash/EAGLE number you'll see quoted is greedy decoding at concurrency 1 — the exact opposite of your regime on both axes.

One contradicting measurement worth replicating: loFT LLC reports Qwen3.6-27B NVFP4 + MTP=3 at 87.9% acceptance, accept length 3.64, 161 tok/s mean at temp 1.0, top_p 0.95, top_k 20 on 2× RTX PRO 6000 Max-Q. If true, native MTP heads degrade far more gracefully under sampling than external drafters do — which would be a meaningfully different conclusion. Verify before believing it.

If you do use spec decode, vLLM ships Dynamic Speculative Decoding to auto-disable under load — but note vLLM #25112: "Spec decoding is not disabled at/after configured batch size." Verify the disable actually fires.

Free alternative worth trying: n-gram / prompt-lookup decoding. RP genuinely echoes its input — character cards, world info, prior turns get re-quoted — so it may pick up real acceptance at zero VRAM cost. Set prompt_lookup_min=8; the default of 2 causes structured-output corruption on Qwen3-class models (vLLM #40875).

5.5 Throughput reference points (all single RTX PRO 6000 unless noted)

Model	Precision	Single-stream	Batched
Gemma-4-31B	NVFP4 + FP8 KV	40.7 tok/s @1k, 38.3 @128k	126.0 @ 4 req
Qwen3.6-27B	FP8	46.1 @1k, 30.4 @256k	peak 189.3 @ 5 concurrent
Qwen3.6-27B	NVFP4, 256k ctx, FP8 KV	~58 tok/s	~119 @ 2-parallel; 64.8 GiB left for KV
Qwen3.6-27B	NVFP4 + grafted MTP=3	median ~92, peak 124.7	67.7% acceptance
Qwen3-32B	NVFP4 vs BF16	—	2,050 tok/s @ conc 128 (vs 1,156 BF16 = 1.77×)

Note the NVFP4-over-BF16 advantage narrows from 2.1× at conc 64 to 1.77× at conc 128 — consistent with the argument that NVFP4's dense-model gain is weight compression (bandwidth), not FP4 math. For your throughput-first shared box: NVFP4 buys less raw compute than marketed, but a lot of freed VRAM → KV cache → concurrency.

5.6 A starting stack

pip install -U llmcompressor==0.13.0        # released 2026-08-11

# Recipe changes that matter for RP:
#   scheme="NVFP4A16"  (weight-only, NOT plain "NVFP4")
#   ignore=["lm_head"]
#   calibration: your OWN RP/creative corpus, or Opus-WritingPrompts
#   num_calibration_samples=256-512, max_seq_length=8192
#
# UltraChat calibration is assistant-y and sanitized — RP finetune activations
# are out-of-distribution relative to it. The one published NVFP4 RP quant used
# 64 samples of Opus-WritingPrompts at seq len 8192. Long sequences matter more
# than sample count here.
#
# Cost on your card: ~45-60 min for a 27B; GPU-trivial (layers onloaded one at
# a time), CPU-RAM-bound at roughly 2GB per 1B params -> ~55GB system RAM.
# llm-compressor does NOT support tensor parallelism for quantization.

export FLASHINFER_CUDA_ARCH_LIST=12.0f
export FLASHINFER_FORCE_SM=120f
export VLLM_NVFP4_GEMM_BACKEND=cutlass

vllm serve /models/rp-27b-nvfp4a16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --max-num-seqs 32
  # NO --speculative-config initially. Add only after measuring
  # draft_acceptance_rate at your real production temperature.

Validation gates before you trust any of it:

1. grep the startup log for "does not have native support for FP4" → if present you're silently on Marlin.
2. KLD against the BF16 parent at 16k and 32k context, not MMLU. This is the only test that catches the failure mode you care about.
3. If spec decode is on, log draft_acceptance_rate at production temperature. Below 0.5, turn it off.
4. Soak-test at real concurrency — the flashinfer_cutlass corruption is silent and load-dependent.

---

6. How I'd actually evaluate these

Nobody publishes spatial-coherence numbers for these models. Across the entire survey the only quantitative claims that exist are Gryphe's StyleTune slop metrics and zerofata's swipe-diversity numbers. You will have to measure this yourself, and it's not hard:

Build ~20 adversarial scenes that bait the specific failures you care about, run each model 5× per scene at your production sampler settings, and score:

• Position tracking — 3+ characters in a room, someone moves, someone leaves. Does the model place them correctly 10 turns later?
• Clothing/object state — an item is removed, moved, or destroyed. Does it reappear?
• Anatomy/limb count — the classic failure. Score explicit impossibilities.
• Knowledge partition — character A learns something in private. Does character B act on it? (OmniToM found "Knowledge Access" is the weakest dimension across all models at 56–75% macro-F1 — this is a real, measurable, near-universal weakness.)
• Context depth — run every test at 8k, 32k, and your real session length. Per §4, this is where the tunes will separate, and where none of them are trained.
• Thinking on vs off, same seed, same scene. Given §2, this is the highest-information single comparison you can run, and no published benchmark has done it for RP.

RP-Bench's own validation is a useful warning about scoring: LLM-judge methods showed negative correlation with community Bayesian Elo (ρ between −0.31 and −0.07), and its automated "Flaw Hunter" disagreed with human users more often than it agreed (50.7% vs 38.7%). Use rule-based checks for state tracking (did the model say "left hand" when the character's left arm was established as pinned?) rather than asking an LLM judge whether the scene was coherent.

---

7. What I could not verify

Stated plainly so you can weigh the rest:

• Reddit is hard-blocked by this environment's egress policy (403 on reddit.com, old.reddit.com, the JSON API, and domain-filtered search). The r/SillyTavernAI weekly megathreads are the single best source for practitioner reports on spatial coherence, and I got none of it. Everything here comes from HuggingFace, benchmark sites, papers, and blog coverage. The community-consensus layer of this report is missing — treat the rankings as evidence-based rather than user-validated.
• No model card in this survey makes an affirmative spatial-coherence or state-tracking claim. I checked all of them explicitly. What exists is MeroMero-v2's training-side coherence judge, and Magistry's disclaimer. Any source telling you these models advertise state tracking is fabricating.
• Trinity-Mini-Ichthyo's card is unreadable (gated, 401). It has the highest download count in-band and I can tell you nothing about it.
• Artemis-31B-v1.1 has no license set — no tag in the API, nothing in the README. Matters if this is going anywhere commercial.
• The Qwen-27B-family exact parameter counts were inconsistent across API calls (27,781,427,952 / 27,781,419,504 / 27,356,728,560 in mutually contradictory slots). The ~27.4B / ~27.8B magnitudes are safe; exact digits are not.
• MeroMero-v2 stage 3 being "multi-turn" — steps, thinking-enabled, and the DeepSeek-V4-Flash logic-defect judge are all confirmed verbatim; the multi-turn detail is not.
• heretic does not preserve MTP natively. I checked PyPI, GitHub, and the docs for any mention of MTP, auxiliary weights, or draft heads — absent from all three. The Native-MTP-Preserved repos are doing a manual post-hoc graft the tool doesn't do for you. Whether heretic 1.4.0 (Jun 2026) added passthrough is unverified.
• UGI Leaderboard's live 2026 data — the CSV is 653kB and only the first chunk is fetchable; the visible slice runs to Nov 2025. The 2026 entries (Huihui-Qwen3-VL-32B-Thinking, Ayla-Light-v2) are unverified.
• EQ-Bench carries essentially no 15–32B RP finetunes — only 9–12B Gemma derivatives. There is no Cydonia/MeroMero/Pantheon Elo, so cross-referencing UGI willingness against EQ-Bench writing quality is not currently possible for any model in this report.
• arxiv.org/html/2607.22732 ("Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning") — rate-limited on 6 attempts. Likely the single most on-point paper for your question. Worth retrying.

---

Sources

Models: zerofata/G4-MeroMero-v2-31B · zerofata/G4-MeroMero-26B-A4B · zerofata/Q3.5-BlueStar-v2-27B · Gryphe/Pantheon-Reasoning-27B · Gryphe/Pantheon-Reasoning-26B-A4B-1.1 · Gryphe/Gemma-4-26B-A4B-StyleTune-V2 · Gryphe/WorldSim-Opus-3.6-35B-A3B · allura-org/Qwen3.5-27B-Anko · ReadyArt/Dark-Scarlett-v1.0-27B · ReadyArt/gemma-4-31B-it-scotoma-2 · TheDrummer/Artemis-31B-v1.1 · TheDrummer/Skyfall-31B-v4.2 · TheDrummer/Rocinante-XL-16B-v1 · sophosympatheia/Magistry-24B-v1.1 · ArliAI/Qwen3.5-27B-RpRMax-v1 · Vortex5/G4-Moonlight-Dusk-26B-A4B · NewEden/Trinity-Mini-Ichthyo · Nimbz/Gemma-4-Gembrain-31B · LatitudeGames/Equinox-31B · llmfan46/…-Native-MTP-Preserved

Bases: Qwen/Qwen3.6-27B · Qwen/Qwen3.5-27B · google/gemma-4-31B-it · google/gemma-4-26B-A4B-it · arcee-ai/Trinity-Mini · mistralai/Magistral-Small-2509 · Gemma 4 blog · Mistral Small 4

Benchmarks: UGI Leaderboard · EQ-Bench · Fiction.liveBench @ Epoch · Fiction.liveBench data · NCP-Bench (arXiv 2608.08160) · NarrativeWorldBench (arXiv 2606.17391) · RP-Bench · PlotPoints · MuSR · LongBench Pro (arXiv 2601.02872) · SpatialEval · OmniToM (arXiv 2605.26322) · Visual vs Textual CoT (ACL 2026) · Mind Your Step (arXiv 2410.21333) · Thinking in Character (arXiv 2506.01748)

Deployment: NVIDIA forum: W4A16 over W4A4 · Red Hat NVFP4 accuracy · NVIDIA NVFP4-QAD report · llm-compressor NVFP4 example · llm-compressor Gemma 4 · ModelOpt hf_ptq · vLLM #47749 · vLLM #39407 (Gemma 4) · vLLM #40875 · vLLM #25112 · CUTLASS #3096 · SGLang #19637 · vLLM recipe Qwen3.6-27B · DigitalOcean spec-decode guide · vLLM EAGLE 3.1 · Why quantized LLMs lose MTP heads · lna-lab GGUF-to-NVFP4-SM120 · rtx6kpro NVFP4 guide · Jarvislabs NVFP4 on RTX PRO 6000 · Millstone Gemma-4-31B NVFP4 · loFT Qwen3.6-27B NVFP4+MTP · Unsloth Dynamic NVFP4 · Benjamin Marie NVFP4 vs INT4 · heretic-llm