Files

T

vh 911ff20356 feat(erp-seat): NVFP4A16 quant pipeline for the Gemma-4 26B-A4B MoE ERP tune + ana-ml2 GPU1 serve stack

- services/erp-seat-quant/quant_nvfp4a16_gemma4_moe.py: linearize_moe first (playbook §3.15),
  asserts the expert Linear count, routers/vision/audio/norms/lm_head ignored, W4A16 for RP
  long-session fidelity, post-steps restore processor configs + template and reset the
  tokenizer truncation cap (§3.14); --dry-run proves targets before GPU time
- services/erp-seat-quant/run_quant_erp_v6.sh: detached container on GPU1 (vllm-llmcompressor)
- stacks/erp-seat: serve recipe copied from gemma4-charrp, true served name only, port 8021

2026-09-08 21:55:28 -07:00

.env.example

feat(erp-seat): NVFP4A16 quant pipeline for the Gemma-4 26B-A4B MoE ERP tune + ana-ml2 GPU1 serve stack

2026-09-08 21:55:28 -07:00

compose.yaml

feat(erp-seat): NVFP4A16 quant pipeline for the Gemma-4 26B-A4B MoE ERP tune + ana-ml2 GPU1 serve stack

2026-09-08 21:55:28 -07:00

README.md

feat(erp-seat): NVFP4A16 quant pipeline for the Gemma-4 26B-A4B MoE ERP tune + ana-ml2 GPU1 serve stack

2026-09-08 21:55:28 -07:00

README.md

erp-seat — ERP-tune seat on ana-ml2 (GPU1, `:8021`)

Serves the latest gated ERP LoRA merge as an NVFP4A16 (weight-only) compressed-tensors checkpoint so the GX10 is free to train the next run. First occupant: run 6 — erp-tune-v6-nvfp4a16 = merged-run06 (jenerallee78 ARA-abliterated Gemma-4-26B-A4B-it, index 33c59654…, + R47 SFT r6) quantized by services/erp-seat-quant/.

True name only. --served-model-name erp-tune-v6-nvfp4a16. Gateway aliases (trial) are set in LiteLLM on the operator's word, never here (no silent substitution — the bf16 arm on the GX10 and this NVFP4 arm are different artifacts).
Recipe = stacks/gemma4-charrp (same arch + format, proven on this box): gemma4 tool and reasoning parsers, enable_thinking pinned false, the model's own stock template (ae53464b…, the one it trained through). Without the reasoning parser the post-tool turn leaks <|channel> markers; without the kwargs pin all prose lands in reasoning_content.
GPU1 is shared — check real usage (nvidia-smi --query-compute-apps=pid,used_memory) before raising ERP_GPU_MEM_UTIL; the flag sizes KV, not CUDA context.
Rollback / next run: point ERP_MODEL + ERP_SERVED_NAME at the next quant dir, keep the previous on disk. Deploy with scripts/deploy-stack.sh ana-ml2 erp-seat.

README.md

erp-seat — ERP-tune seat on ana-ml2 (GPU1, :8021)

erp-seat — ERP-tune seat on ana-ml2 (GPU1, `:8021`)