fix(erp-seat): tool_choice=none returned an empty turn — add --exclude-tools-when-tool-choice-none (vLLM kept the tools in the prompt, the model called one, parsing was off); 12-shape tool matrix green before/after

2026-09-08 23:01:26 -07:00
parent 32399d0da2
commit 7f6be8a56a
3 changed files with 18 additions and 0 deletions
@@ -35,6 +35,11 @@ services:
      - gemma4
      - --default-chat-template-kwargs
      - '{"enable_thinking": false}'
+      # tool_choice:"none" trap (measured 2026-09-08): without this flag vLLM still renders the
+      # tools into the prompt, the model emits a tool call anyway, and because tool parsing is
+      # off for tool_choice=none the reply comes back with content=null AND tool_calls=null —
+      # an empty turn. The flag drops the tools from the prompt so the model answers in prose.
+      - --exclude-tools-when-tool-choice-none
      - --chat-template
      - ${ERP_CHAT_TEMPLATE:-/tank/aimodels/erp-tune-v6-nvfp4a16/chat_template.jinja}
      - --max-model-len