esh-pfi-infrastructure/stacks/erp-seat/README.md

erp-seat — ERP-tune seat on ana-ml2 (GPU1, :8021)

Serves the latest gated ERP LoRA merge as an NVFP4A16 (weight-only) compressed-tensors checkpoint so the GX10 is free to train the next run.

Current occupant: run 7 — erp-tune-v7-nvfp4a16 = merged-run07 (jenerallee78 ARA-abliterated Gemma-4-26B-A4B-it, index 33c59654…, + R47 SFT r7 = r6 plus the opening-split slot and its companion loss mask), quantized by services/erp-seat-quant/ on 2026-09-09. 49 GiB bf16 → 16 GiB NVFP4A16. Runbook docs/runbooks/gx10-run-07.md.

Previous: run 6 (erp-tune-v6-nvfp4a16, 2026-09-08). Its artifact is still on /tank/aimodels/ and the pre-swap host env is at /tmp/erp-seat-env.v6.bak on ana-ml2, so a rollback is an .env flip plus docker compose up -d.

• True name only. --served-model-name erp-tune-v7-nvfp4a16. Gateway aliases (trial) are set in LiteLLM on the operator's word, never here (no silent substitution — the bf16 arm on the GX10 and this NVFP4 arm are different artifacts).
• Recipe = stacks/gemma4-charrp (same arch + format, proven on this box): gemma4 tool and reasoning parsers, enable_thinking pinned false, the model's own stock template (ae53464b…, the one it trained through). Without the reasoning parser the post-tool turn leaks <|channel> markers; without the kwargs pin all prose lands in reasoning_content.
• tool_choice: "none" trap (measured 2026-09-08, fixed with --exclude-tools-when-tool-choice-none). Without the flag vLLM still renders the tools into the prompt, the model emits a tool call anyway, and because parsing is off for none the reply is content: null, tool_calls: null — an empty turn, 3/3 reproductions. With the flag the tools are dropped from the prompt and the model answers in prose (3/3). The rest of the matrix (auto / required / named / parallel / nested schema / empty tools: [] / streaming / tool-result round trip) was green before and after. stacks/gemma4-charrp has the same exposure and does NOT carry the flag yet.
• Forced tool_choice (named / required) is prompt-driven on EVERY Gemma-4 seat, not grammar-enforced — by vLLM design. vllm/tool_parsers/gemma4_engine_tool_parser.py sets supports_required_and_named = False and its adjust_request deliberately skips the structured-output JSON for required/named so the model can emit its native <|tool_call>call:… syntax. A tune that weakened that syntax (this ERP tune) therefore honours forced calls only sometimes. Measured 2026-09-08, 3 conversations × 3 turns, real system prompt: v0.26.0 1/9; nightly 311b3513 (v0.27.2rc1, the gen seat's image) 6/9 and the tool-result round trip stays clean 3/3 — so the seat runs the nightly. tts-dev measured 0/18 on v0.26.0 with gen 18/18 as the positive control, and response_format: json_schema (guided decoding) 18/18 on this seat — that is the deterministic path for a forced call; tool_choice: auto works normally. A parser plugin that re-enables guided JSON would also need JSON extraction in the engine-parser path; not attempted.
• GPU1 is shared — check real usage (nvidia-smi --query-compute-apps=pid,used_memory) before raising ERP_GPU_MEM_UTIL; the flag sizes KV, not CUDA context.
• Rollback / next run: point ERP_MODEL + ERP_SERVED_NAME at the next quant dir, keep the previous on disk. Deploy with scripts/deploy-stack.sh ana-ml2 erp-seat.