docs(litellm): correct stale char-rp description -- it is MeroMero/Gemma-4, not the retired GGUF Magidonia seat
The comment still described TheDrummer Magidonia-24B-v4.3 Q6_K on
llama.cpp, which was replaced by the vLLM MeroMero-v2 NVFP4A16 seat on
2026-08-12. Routing was already correct (:8016 is MeroMero); only the
prose was wrong, so anyone reading the config got the wrong model family
entirely.
Records why the seat exists: char-rp-reasoning is a Qwen3.x derivative and
emits ~5-6k chars of CoT per turn regardless of which Qwen RP tune is
loaded. Measured 2026-08-16 on identical prompts -- Dark-Scarlett 6036 ch
vs Fable-Fusion 5323 ch -- so that is the base family, not the finetune,
and no swap within it fixes it. Gemma-4 is the non-thinking seat.
Also pins the mandatory --default-chat-template-kwargs
'{"enable_thinking": false}' rationale from b8f0f4c, and flags that the
temp 1.1 / min_p 0.10 samplers were tuned against the retired
Mistral-family seat and never re-tuned for Gemma-4.
Docs-only: no litellm_params touched, no routing change.
This commit is contained in:
@@ -161,17 +161,30 @@ model_list:
|
||||
enable_thinking: true
|
||||
model_info:
|
||||
mode: chat
|
||||
# char-rp -> GGUF dark-romantasy PROSE seat (:8016, llama.cpp, char-rp-gguf stack on
|
||||
# ana-ml2 GPU 0). TheDrummer Magidonia-24B-v4.3 Q6_K — Magistral (Mistral) RP tune.
|
||||
# NON-thinking: elite literary prose, zero refusal on dark/explicit scenes, ~65 tok/s,
|
||||
# tight POV/instruction adherence (live-tested 2026-07-08). Replaced the broken Angel
|
||||
# NVFP4 serve AND the earlier AEON-rp MTP twin. Sampler A/B-tuned 2026-07-08 vs the
|
||||
# dvalin-smithy canonical: temp 1.1 / top_p 0.95 / min_p 0.10 / top_k 0, NO repetition
|
||||
# penalty. min_p 0.10 richened imagery vs 0.03; rep 1.05 REJECTED (injected a stray
|
||||
# markdown title in a grief scene — rep-style penalties hurt Drummer RP, matching the
|
||||
# card). No enable_thinking kwarg — meaningless to the Mistral template. Alt prose model
|
||||
# (swap via the stack .env): MS3.2-PaintedFantasy-v4.1-24B. Callers may override.
|
||||
# docs/pfi/model-sampler-defaults.md.
|
||||
# char-rp -> MeroMero-v2 NON-THINKING prose seat (:8016, vLLM, meromero-charrp stack on
|
||||
# ana-ml2 GPU 0). G4-MeroMero-v2-31B NVFP4A16, **Gemma-4 base** (google/gemma-4-31B-it),
|
||||
# 256K ctx, in-house quant. Replaced the GGUF/llama.cpp Magidonia-24B seat 2026-08-12.
|
||||
#
|
||||
# THIS SEAT EXISTS BECAUSE THE QWEN BASE THINKS INCESSANTLY. char-rp-reasoning is a
|
||||
# Qwen3.x derivative and emits ~5-6k chars of CoT per turn no matter which Qwen RP tune
|
||||
# is loaded — that is the base family, not the finetune, and no swap within it fixes it
|
||||
# (measured 2026-08-16: Dark-Scarlett 6036 ch vs Fable-Fusion 5323 ch on the same
|
||||
# prompts). Gemma-4 gives a genuinely non-thinking prose seat. Reach for THIS one when
|
||||
# you want prose without a reasoning trace; reach for char-rp-reasoning when you want
|
||||
# the deliberation. Best-of-breed per seat — deliberately NOT the same model.
|
||||
#
|
||||
# Serving flags are load-bearing (commit b8f0f4c): `--tool-call-parser gemma4
|
||||
# --enable-auto-tool-choice --reasoning-parser gemma4` AND
|
||||
# `--default-chat-template-kwargs '{"enable_thinking": false}'`. That last flag is
|
||||
# MANDATORY, not decorative — the gemma4 parser defaults enable_thinking to True, which
|
||||
# pre-initialises the engine to REASONING and returns null `content` for all plain RP
|
||||
# prose. Before it was set, every tools-bearing request also 400'd (no parser at all).
|
||||
# Verified 0 chars reasoning / clean prose end-to-end 2026-08-16.
|
||||
#
|
||||
# Sampler note: the temp 1.1 / min_p 0.10 / top_k 0 values below were A/B-tuned
|
||||
# 2026-07-08 against the retired Mistral-family Magidonia seat, NOT against MeroMero.
|
||||
# They have not been re-tuned for Gemma-4 — treat as inherited, not canonical.
|
||||
# Callers may override. docs/pfi/model-sampler-defaults.md; stacks/meromero-charrp/.
|
||||
- model_name: char-rp
|
||||
litellm_params:
|
||||
model: hosted_vllm/char-rp
|
||||
|
||||
Reference in New Issue
Block a user