feat(litellm): wire canonical sampler defaults for all 4 gateway seats

dvalin-smithy canonical set, infra-ops triaged + char-rp A/B-validated on the live serve. - gen (+summarizer-large twin): presence_penalty 1.0 -> 1.5 (Qwen3.6 non-thinking rec). - gen-reasoning: temp 0.6 -> 1.0, presence 1.0 -> 1.5 (Qwen general-thinking profile; the old 0.6 was the coding sub-profile). - char-rp: temp 1.0 -> 1.1, min_p 0.03 -> 0.10, top_k 0, NO rep. A/B on 2 dark-romantasy prompts: min_p 0.10 richened imagery; repeat_penalty 1.05 REJECTED (injected a stray markdown title, hurts Drummer/Magistral RP creativity per the card + dvalin's own note). - char-rp-reasoning: add explicit top_p 0.95 (else per the RpR card: no rep/DRY/XTC). Canonical reference: docs/pfi/model-sampler-defaults.md (mirrors dvalin's derivation).
2026-07-08 11:10:28 -07:00
parent f5706046b1
commit aac4bcfa3e
3 changed files with 101 additions and 19 deletions
@@ -0,0 +1,74 @@
+# Canonical sampler defaults — PFI/VastBlue LiteLLM gateway seats
+
+**Applied:** 2026-07-08 · **Gateway:** `ana-docker:4000` · **Config:** `stacks/litellm/conf/config.yaml` → `/opt/docker/conf/litellm/config.yaml`
+
+Canonical high-quality sampler defaults for the four model seats, **derived by
+dvalin-smithy-dev** (full rationale + sources: `dvalin-smithy/hoard-drafts/pfi-gateway-sampler-defaults-20260708.md`),
+**triaged + A/B-validated by infra-ops**, and wired into the gateway. These are the
+gateway *defaults*; callers may override per request.
+
+Optimized for **output / prose quality** (not throughput or determinism).
+
+## Engine surfaces
+
+- **gen / gen-reasoning** — vLLM 0.24 (OpenAI sampler surface). No native DRY/XTC → anti-repetition via `presence_penalty`. Thinking split via `chat_template_kwargs.enable_thinking` on distinct `--served-model-name`s (avoids the shared-config-mutation footgun).
+- **char-rp / char-rp-reasoning** — llama.cpp / llama-server (supports `min_p`, `top_k`, DRY, XTC, dynatemp). `min_p` + `top_p` do the tail work; `top_k 0` disables top-k.
+
+## The four seats (applied values)
+
+### 1. gen — Qwen3.6-35B-A3B heretic (vLLM, non-thinking)
+Also governs **summarizer-large** (shares the same `qwen3.6-27b-aeon` @ :8015 deployment → kept identical).
+
+| param | value |
+|---|---|
+| temperature | 0.7 |
+| top_p | 0.80 |
+| top_k | 20 |
+| presence_penalty | **1.5** |
+| repetition/frequency | 1.0 / 0.0 |
+| enable_thinking | false |
+
+*Source:* Qwen3.6 README instruct/non-thinking rec. *Change:* presence_penalty 1.0 → 1.5.
+
+### 2. gen-reasoning — same model (vLLM, thinking)
+
+| param | value |
+|---|---|
+| temperature | **1.0** |
+| top_p | 0.95 |
+| top_k | 20 |
+| presence_penalty | **1.5** |
+| repetition/frequency | 1.0 / 0.0 |
+| enable_thinking | true |
+
+*Source:* Qwen3.6 README **general** thinking profile (NOT the temp-0.6 coding sub-profile — the prior default was that coding profile by mistake). *Changes:* temperature 0.6 → 1.0, presence_penalty 1.0 → 1.5. Reasoning is verbose (~9k chars) → callers set generous `max_tokens` (catalog default 32768). Optional per-route coding override: temp 0.6 / presence 0.0.
+
+### 3. char-rp — Magidonia-24B-v4.3 (llama.cpp, non-thinking prose RP)
+
+| param | value |
+|---|---|
+| temperature | **1.1** |
+| top_p | 0.95 |
+| min_p | **0.10** |
+| top_k | 0 (disabled) |
+| repetition/DRY/XTC | **off** |
+
+*Source:* dvalin canonical (Mistral-Small RP prose) **A/B-validated by infra-ops** on the live serve. *Changes:* temp 1.0 → 1.1, min_p 0.03 → 0.10. **min_p 0.10 richened imagery vs 0.03** with no incoherence at temp 1.1. **repeat_penalty 1.05 was REJECTED** — in the A/B it injected a stray markdown title into a grief scene; rep-style penalties hurt Drummer/Magistral RP creativity (matches the model card and dvalin's own note). Alt prose model: `MS3.2-PaintedFantasy-v4.1-24B` (swap via the `char-rp-gguf` stack `.env`).
+
+### 4. char-rp-reasoning — QwQ-32B-ArliAI-RpR-v4 (llama.cpp, reasoning RP)
+
+| param | value |
+|---|---|
+| temperature | 1.0 |
+| top_p | **0.95** |
+| top_k | 40 |
+| min_p | 0.02 |
+| repetition/DRY/XTC | **off** |
+
+*Source:* ArliAI RpR v4 card — explicit **NO** rep/DRY/XTC penalties. *Change:* added explicit top_p 0.95. Reasoning is server-side (`--reasoning on`, budget-capped); the CoT surfaces in `reasoning_content` with clean prose in `content`. SillyTavern wiring (non-sampler): `include names = never`; exact `<think>`/`</think>` tokens.
+
+## Changing a default
+
+Edit the seat's `litellm_params` in `stacks/litellm/conf/config.yaml`, `scp` to
+`/opt/docker/conf/litellm/config.yaml` on ana-docker, `docker restart litellm`.
+(`gen` and `summarizer-large` must change together — same deployment.)