vh
6972e7ef7f
- services/erp-seat-quant/run_quant_erp_v7.sh: v6 runner retargeted; dry-run gate passed identically (11,725 targets, 11,520 experts = 30x128x3, routers+vision BF16) - 49 GiB bf16 relayed gx10 -> ana-ml2 (no key path either way; nh3-dev relays), checksums verified against source; quant 49 -> 16 GiB, all post-steps clean - stacks/erp-seat: .env-driven swap to erp-tune-v7-nvfp4a16, served under its TRUE name; homepage labels + README updated, v6 rollback path recorded - stacks/litellm: trial -> erp-tune-v7-nvfp4a16 (config-file alias; /model/update refuses a config model, so this is an edit + restart)
19 lines
1.0 KiB
Bash
Executable File
19 lines
1.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# NVFP4A16 quant of the ERP run-7 merged model on ana-ml2 GPU1 (co-resident with the GPU1 seats;
|
|
# CPU-resident load, per-layer onload). Detached container; watch with `docker logs -f erp-v7-quant`.
|
|
# NOTE: no PYTORCH_CUDA_ALLOC_CONF=expandable_segments (playbook §3.10).
|
|
set -euo pipefail
|
|
WORK=/tank/aimodels/erp-tune-v7-quant-work
|
|
SRC="${1:-/tank/aimodels/erp-tune-v7-bf16}"
|
|
OUT="${2:-/tank/aimodels/erp-tune-v7-nvfp4a16}"
|
|
MODE="${3:-full}" # full | dry-run
|
|
EXTRA=""; [ "$MODE" = "dry-run" ] && EXTRA="--dry-run"
|
|
NAME=erp-v7-quant; [ "$MODE" = "dry-run" ] && NAME=erp-v7-quant-dry
|
|
docker rm -f "$NAME" 2>/dev/null || true
|
|
docker run -d --name "$NAME" --gpus '"device=1"' --ipc host \
|
|
-v /tank/aimodels:/tank/aimodels \
|
|
--entrypoint python3 vllm-llmcompressor:latest \
|
|
"$WORK/quant_nvfp4a16_gemma4_moe.py" --model "$SRC" --out "$OUT" \
|
|
--num-samples "${NUM_SAMPLES:-256}" --seqlen "${SEQLEN:-8192}" $EXTRA
|
|
echo "launched $NAME: $(docker ps --filter name=$NAME --format '{{.Status}}')"
|