esh-pfi-infrastructure/docs/pfi/model-sampler-defaults.md

Canonical sampler defaults — PFI/VastBlue LiteLLM gateway seats

Applied: 2026-07-08 · Gateway: ana-docker:4000 · Config: stacks/litellm/conf/config.yaml → /opt/docker/conf/litellm/config.yaml

Canonical high-quality sampler defaults for the four model seats, derived by dvalin-smithy-dev (full rationale + sources: dvalin-smithy/hoard-drafts/pfi-gateway-sampler-defaults-20260708.md), triaged + A/B-validated by infra-ops, and wired into the gateway. These are the gateway defaults; callers may override per request.

Optimized for output / prose quality (not throughput or determinism).

Engine surfaces

• gen / gen-reasoning — vLLM 0.24 (OpenAI sampler surface). No native DRY/XTC → anti-repetition via presence_penalty. Thinking split via chat_template_kwargs.enable_thinking on distinct --served-model-names (avoids the shared-config-mutation footgun).
• char-rp / char-rp-reasoning — llama.cpp / llama-server (supports min_p, top_k, DRY, XTC, dynatemp). min_p + top_p do the tail work; top_k 0 disables top-k.

The four seats (applied values)

1. gen — Qwen3.6-35B-A3B heretic (vLLM, non-thinking)

Also governs summarizer-large (shares the same qwen3.6-27b-aeon @ :8015 deployment → kept identical).

param	value
temperature	0.7
top_p	0.80
top_k	20
presence_penalty	1.5
repetition/frequency	1.0 / 0.0
enable_thinking	false

Source: Qwen3.6 README instruct/non-thinking rec. Change: presence_penalty 1.0 → 1.5.

2. gen-reasoning — same model (vLLM, thinking)

param	value
temperature	1.0
top_p	0.95
top_k	20
presence_penalty	1.5
repetition/frequency	1.0 / 0.0
enable_thinking	true

Source: Qwen3.6 README general thinking profile (NOT the temp-0.6 coding sub-profile — the prior default was that coding profile by mistake). Changes: temperature 0.6 → 1.0, presence_penalty 1.0 → 1.5. Reasoning is verbose (~9k chars) → callers set generous max_tokens (catalog default 32768). Optional per-route coding override: temp 0.6 / presence 0.0.

3. char-rp — Magidonia-24B-v4.3 (llama.cpp, non-thinking prose RP)

param	value
temperature	1.1
top_p	0.95
min_p	0.10
top_k	0 (disabled)
repetition/DRY/XTC	off

Source: dvalin canonical (Mistral-Small RP prose) A/B-validated by infra-ops on the live serve. Changes: temp 1.0 → 1.1, min_p 0.03 → 0.10. min_p 0.10 richened imagery vs 0.03 with no incoherence at temp 1.1. repeat_penalty 1.05 was REJECTED — in the A/B it injected a stray markdown title into a grief scene; rep-style penalties hurt Drummer/Magistral RP creativity (matches the model card and dvalin's own note). Alt prose model: MS3.2-PaintedFantasy-v4.1-24B (swap via the char-rp-gguf stack .env).

4. char-rp-reasoning — Qwen3.5-27B-Deckard-PKD (llama.cpp, managed-reasoning RP)

param	value (request-level)
temperature	1.0
top_p	0.95
top_k	40
min_p	0.05
presence/repetition	off
DRY	0.8 server-side (base 1.75 / len 2, dry-after-temp) — not a request param
reasoning-budget	400 (server-side)

Source: dvalin-CONFIRMED canonical 2026-07-08 (thread 01KX1Y7P). Corrected 2026-07-09: this seat had lagged on QwQ-RpR-v4 — the A/B on 2026-07-08 replaced it with Deckard-PKD-Heretic i1-Q5_K_M (DavidAU, Qwen3.5-27B, :8018); the live gateway was always Deckard. Deckard won on brokkr's frozen scorer (0/30 loops, 0/30 refusals) over RpR-v4 (1/30 loop, forbids DRY) + Pantheon-27B (7/30 refusals). Reasoning ON server-side (--reasoning on, budget 400); CoT surfaces in reasoning_content, clean prose in content. Tuning ladder: flat prose→min_p 0.08, loops→DRY 0.9, over-damped→DRY 0.6/off. Do NOT import RpR/QwQ sampler rules (different family — QwQ hated DRY; Qwen3.5 benefits from it).

Changing a default

Edit the seat's litellm_params in stacks/litellm/conf/config.yaml, scp to /opt/docker/conf/litellm/config.yaml on ana-docker, docker restart litellm. (gen and summarizer-large must change together — same deployment.)