Commit Graph
4 Commits
Author SHA1 Message Date
vh 6972e7ef7f feat(erp-seat): run 7 quantized to NVFP4A16 and serving as the trial seat on ana-ml2
- services/erp-seat-quant/run_quant_erp_v7.sh: v6 runner retargeted; dry-run gate
  passed identically (11,725 targets, 11,520 experts = 30x128x3, routers+vision BF16)
- 49 GiB bf16 relayed gx10 -> ana-ml2 (no key path either way; nh3-dev relays),
  checksums verified against source; quant 49 -> 16 GiB, all post-steps clean
- stacks/erp-seat: .env-driven swap to erp-tune-v7-nvfp4a16, served under its TRUE
  name; homepage labels + README updated, v6 rollback path recorded
- stacks/litellm: trial -> erp-tune-v7-nvfp4a16 (config-file alias; /model/update
  refuses a config model, so this is an edit + restart)
2026-09-09 15:30:15 -07:00
vh a34a72fc09 feat(erp-tune): run 7 launched on pfi-gx10 (opening-split slot + mask union) — config, launcher, builder, runbook; erp-seat: pin vLLM nightly 311b3513 (forced tool_choice 1/9 -> 6/9, round trip clean), README on the gemma4-parser design limit 2026-09-08 23:31:09 -07:00
vh 7f6be8a56a fix(erp-seat): tool_choice=none returned an empty turn — add --exclude-tools-when-tool-choice-none (vLLM kept the tools in the prompt, the model called one, parsing was off); 12-shape tool matrix green before/after 2026-09-08 23:01:26 -07:00
vh 911ff20356 feat(erp-seat): NVFP4A16 quant pipeline for the Gemma-4 26B-A4B MoE ERP tune + ana-ml2 GPU1 serve stack
- services/erp-seat-quant/quant_nvfp4a16_gemma4_moe.py: linearize_moe first (playbook §3.15),
  asserts the expert Linear count, routers/vision/audio/norms/lm_head ignored, W4A16 for RP
  long-session fidelity, post-steps restore processor configs + template and reset the
  tokenizer truncation cap (§3.14); --dry-run proves targets before GPU time
- services/erp-seat-quant/run_quant_erp_v6.sh: detached container on GPU1 (vllm-llmcompressor)
- stacks/erp-seat: serve recipe copied from gemma4-charrp, true served name only, port 8021
2026-09-08 21:55:28 -07:00