docs(litellm): correct stale char-rp description -- it is MeroMero/Gemma-4, not the retired GGUF Magidonia seat

The comment still described TheDrummer Magidonia-24B-v4.3 Q6_K on
llama.cpp, which was replaced by the vLLM MeroMero-v2 NVFP4A16 seat on
2026-08-12. Routing was already correct (:8016 is MeroMero); only the
prose was wrong, so anyone reading the config got the wrong model family
entirely.

Records why the seat exists: char-rp-reasoning is a Qwen3.x derivative and
emits ~5-6k chars of CoT per turn regardless of which Qwen RP tune is
loaded. Measured 2026-08-16 on identical prompts -- Dark-Scarlett 6036 ch
vs Fable-Fusion 5323 ch -- so that is the base family, not the finetune,
and no swap within it fixes it. Gemma-4 is the non-thinking seat.

Also pins the mandatory --default-chat-template-kwargs
'{"enable_thinking": false}' rationale from b8f0f4c, and flags that the
temp 1.1 / min_p 0.10 samplers were tuned against the retired
Mistral-family seat and never re-tuned for Gemma-4.

Docs-only: no litellm_params touched, no routing change.
This commit is contained in:
2026-08-16 09:57:48 -07:00
parent ee2b678bcb
commit 53096bffdc
+24 -11
View File
@@ -161,17 +161,30 @@ model_list:
enable_thinking: true
model_info:
mode: chat
# char-rp -> GGUF dark-romantasy PROSE seat (:8016, llama.cpp, char-rp-gguf stack on
# ana-ml2 GPU 0). TheDrummer Magidonia-24B-v4.3 Q6_K — Magistral (Mistral) RP tune.
# NON-thinking: elite literary prose, zero refusal on dark/explicit scenes, ~65 tok/s,
# tight POV/instruction adherence (live-tested 2026-07-08). Replaced the broken Angel
# NVFP4 serve AND the earlier AEON-rp MTP twin. Sampler A/B-tuned 2026-07-08 vs the
# dvalin-smithy canonical: temp 1.1 / top_p 0.95 / min_p 0.10 / top_k 0, NO repetition
# penalty. min_p 0.10 richened imagery vs 0.03; rep 1.05 REJECTED (injected a stray
# markdown title in a grief scene — rep-style penalties hurt Drummer RP, matching the
# card). No enable_thinking kwarg — meaningless to the Mistral template. Alt prose model
# (swap via the stack .env): MS3.2-PaintedFantasy-v4.1-24B. Callers may override.
# docs/pfi/model-sampler-defaults.md.
# char-rp -> MeroMero-v2 NON-THINKING prose seat (:8016, vLLM, meromero-charrp stack on
# ana-ml2 GPU 0). G4-MeroMero-v2-31B NVFP4A16, **Gemma-4 base** (google/gemma-4-31B-it),
# 256K ctx, in-house quant. Replaced the GGUF/llama.cpp Magidonia-24B seat 2026-08-12.
#
# THIS SEAT EXISTS BECAUSE THE QWEN BASE THINKS INCESSANTLY. char-rp-reasoning is a
# Qwen3.x derivative and emits ~5-6k chars of CoT per turn no matter which Qwen RP tune
# is loaded — that is the base family, not the finetune, and no swap within it fixes it
# (measured 2026-08-16: Dark-Scarlett 6036 ch vs Fable-Fusion 5323 ch on the same
# prompts). Gemma-4 gives a genuinely non-thinking prose seat. Reach for THIS one when
# you want prose without a reasoning trace; reach for char-rp-reasoning when you want
# the deliberation. Best-of-breed per seat — deliberately NOT the same model.
#
# Serving flags are load-bearing (commit b8f0f4c): `--tool-call-parser gemma4
# --enable-auto-tool-choice --reasoning-parser gemma4` AND
# `--default-chat-template-kwargs '{"enable_thinking": false}'`. That last flag is
# MANDATORY, not decorative — the gemma4 parser defaults enable_thinking to True, which
# pre-initialises the engine to REASONING and returns null `content` for all plain RP
# prose. Before it was set, every tools-bearing request also 400'd (no parser at all).
# Verified 0 chars reasoning / clean prose end-to-end 2026-08-16.
#
# Sampler note: the temp 1.1 / min_p 0.10 / top_k 0 values below were A/B-tuned
# 2026-07-08 against the retired Mistral-family Magidonia seat, NOT against MeroMero.
# They have not been re-tuned for Gemma-4 — treat as inherited, not canonical.
# Callers may override. docs/pfi/model-sampler-defaults.md; stacks/meromero-charrp/.
- model_name: char-rp
litellm_params:
model: hosted_vllm/char-rp