vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

litellm(gen/qwen3.5-122-a10b): presence_penalty=1.0 anti-repetition default

Browse Source 
The abliterated/NVFP4 Qwopus 122B "gen" model (+ qwen-large / summarizer-large
aliases) had no repetition control in its sampling defaults, causing degenerate
repetition loops. Add presence_penalty: 1.0 (Qwen-documented anti-repetition
lever, range 0-2) to all 7 qwen3.5-122-a10b gateway records. Overrideable
default; bake into the vLLM serving def once the value is validated.
This commit is contained in:
Vuong Hoang
2026-06-27 08:09:53 -07:00
parent 3239b0a613
commit 52d5f66216
1 changed files with 12 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/litellm/conf/config.yaml
+12
View File
@@ -59,6 +59,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: false
@@ -96,6 +97,11 @@ model_list:
# + --reasoning-parser qwen3. One upstream fanned out under qwen3.5-122-a10b[-reasoning]
# + aliases qwen-large[-reasoning] + gen[-reasoning]; -reasoning variants enable
# thinking. Keep api_base in sync.
# presence_penalty: 1.0 on ALL these qwen3.5-122-a10b records (+ summarizer-large
# above) — anti-repetition-loop damper for the abliterated/NVFP4 tendency (operator
# 2026-06-27). Gateway-tunable default (callers can override); bake the validated
# value into the vLLM serving def (stacks/qwen3.5-122b, --override-generation-config)
# once confirmed, to also cover direct (non-gateway) callers.
# ⚠️ Worldtree CHARACTER backend (was bound to mistral-small-4) is dark until
# repointed — operator-acknowledged. ---
- model_name: qwen3.5-122-a10b
@@ -103,6 +109,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: false
@@ -113,6 +120,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: true
@@ -123,6 +131,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: false
@@ -133,6 +142,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: true
@@ -143,6 +153,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: false
@@ -153,6 +164,7 @@ model_list:
model: hosted_vllm/qwen3.5-122-a10b
api_base: http://10.250.50.54:8013/v1
api_key: os.environ/VLLM_API_KEY
presence_penalty: 1.0
extra_body:
chat_template_kwargs:
enable_thinking: true