vh
911ff20356
- services/erp-seat-quant/quant_nvfp4a16_gemma4_moe.py: linearize_moe first (playbook §3.15), asserts the expert Linear count, routers/vision/audio/norms/lm_head ignored, W4A16 for RP long-session fidelity, post-steps restore processor configs + template and reset the tokenizer truncation cap (§3.14); --dry-run proves targets before GPU time - services/erp-seat-quant/run_quant_erp_v6.sh: detached container on GPU1 (vllm-llmcompressor) - stacks/erp-seat: serve recipe copied from gemma4-charrp, true served name only, port 8021
19 lines
1.3 KiB
Markdown
19 lines
1.3 KiB
Markdown
# erp-seat — ERP-tune seat on ana-ml2 (GPU1, `:8021`)
|
|
|
|
Serves the latest gated ERP LoRA merge as an **NVFP4A16** (weight-only) compressed-tensors
|
|
checkpoint so the GX10 is free to train the next run. First occupant: **run 6** —
|
|
`erp-tune-v6-nvfp4a16` = merged-run06 (jenerallee78 ARA-abliterated Gemma-4-26B-A4B-it, index
|
|
`33c59654…`, + R47 SFT r6) quantized by `services/erp-seat-quant/`.
|
|
|
|
- **True name only.** `--served-model-name erp-tune-v6-nvfp4a16`. Gateway aliases (`trial`) are
|
|
set in LiteLLM on the operator's word, never here (no silent substitution — the bf16 arm on the
|
|
GX10 and this NVFP4 arm are different artifacts).
|
|
- **Recipe** = `stacks/gemma4-charrp` (same arch + format, proven on this box): `gemma4` tool
|
|
and reasoning parsers, `enable_thinking` pinned false, the model's own stock template
|
|
(`ae53464b…`, the one it trained through). Without the reasoning parser the post-tool turn leaks
|
|
`<|channel>` markers; without the kwargs pin all prose lands in `reasoning_content`.
|
|
- **GPU1 is shared** — check real usage (`nvidia-smi --query-compute-apps=pid,used_memory`) before
|
|
raising `ERP_GPU_MEM_UTIL`; the flag sizes KV, not CUDA context.
|
|
- **Rollback / next run:** point `ERP_MODEL` + `ERP_SERVED_NAME` at the next quant dir, keep the
|
|
previous on disk. Deploy with `scripts/deploy-stack.sh ana-ml2 erp-seat`.
|