feat(erp-tune): run 7 launched on pfi-gx10 (opening-split slot + mask union) — config, launcher, builder, runbook; erp-seat: pin vLLM nightly 311b3513 (forced tool_choice 1/9 -> 6/9, round trip clean), README on the gemma4-parser design limit

This commit is contained in:
vh
2026-09-08 23:31:09 -07:00
parent 3c54f59878
commit a34a72fc09
8 changed files with 251 additions and 2 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
# erp-seat — ana-ml2 GPU1. Real .env lives on the host at /opt/docker/compose/erp-seat/.env.
ERP_IMAGE=vllm/vllm-openai:v0.26.0
ERP_IMAGE=vllm/vllm-openai:nightly-311b3513af33bc29b4acb2fde2e9313e5e9966a0
ERP_MODEL=/tank/aimodels/erp-tune-v6-nvfp4a16
ERP_SERVED_NAME=erp-tune-v6-nvfp4a16
ERP_CHAT_TEMPLATE=/tank/aimodels/erp-tune-v6-nvfp4a16/chat_template.jinja
+12
View File
@@ -19,6 +19,18 @@ checkpoint so the GX10 is free to train the next run. First occupant: **run 6**
the model answers in prose (3/3). The rest of the matrix (auto / required / named / parallel /
nested schema / empty `tools: []` / streaming / tool-result round trip) was green before and
after. `stacks/gemma4-charrp` has the same exposure and does NOT carry the flag yet.
- **Forced tool_choice (named / `required`) is prompt-driven on EVERY Gemma-4 seat, not
grammar-enforced — by vLLM design.** `vllm/tool_parsers/gemma4_engine_tool_parser.py` sets
`supports_required_and_named = False` and its `adjust_request` deliberately skips the
structured-output JSON for required/named so the model can emit its native
`<|tool_call>call:…` syntax. A tune that weakened that syntax (this ERP tune) therefore
honours forced calls only sometimes. Measured 2026-09-08, 3 conversations × 3 turns, real
system prompt: v0.26.0 **1/9**; nightly `311b3513` (v0.27.2rc1, the `gen` seat's image) **6/9**
and the tool-result round trip stays clean 3/3 — so the seat runs the nightly. tts-dev measured
0/18 on v0.26.0 with `gen` 18/18 as the positive control, and **`response_format:
json_schema` (guided decoding) 18/18 on this seat** — that is the deterministic path for a
forced call; `tool_choice: auto` works normally. A parser plugin that re-enables guided JSON
would also need JSON extraction in the engine-parser path; not attempted.
- **GPU1 is shared** — check real usage (`nvidia-smi --query-compute-apps=pid,used_memory`) before
raising `ERP_GPU_MEM_UTIL`; the flag sizes KV, not CUDA context.
- **Rollback / next run:** point `ERP_MODEL` + `ERP_SERVED_NAME` at the next quant dir, keep the
+7 -1
View File
@@ -12,7 +12,7 @@ name: erp-seat
services:
vllm-erp-seat:
image: ${ERP_IMAGE:-vllm/vllm-openai:v0.26.0}
image: ${ERP_IMAGE:-vllm/vllm-openai:nightly-311b3513af33bc29b4acb2fde2e9313e5e9966a0}
container_name: ${ERP_CONTAINER:-vllm-erp-seat}
restart: unless-stopped
ipc: host
@@ -31,6 +31,12 @@ services:
- --tool-call-parser
- gemma4
- --enable-auto-tool-choice
# --reasoning-parser gemma4 + the enable_thinking pin are BOTH required (measured 2026-09-08):
# without them the post-tool-response turn leaks a literal "<|channel>thought\n<channel|>"
# prefix into content (3/3), exactly as stacks/gemma4-charrp/README.md warns. They do NOT
# fix forced (named) tool_choice — that failure is independent of the reasoning parser
# (measured 1/9 with it, 3/9 without): the model free-writes Gemma native call syntax
# instead of grammar-constrained JSON. See README for the forced-tool_choice status.
- --reasoning-parser
- gemma4
- --default-chat-template-kwargs