vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(erp-seat): run 7 quantized to NVFP4A16 and serving as the trial seat on ana-ml2

Browse Source 
- services/erp-seat-quant/run_quant_erp_v7.sh: v6 runner retargeted; dry-run gate
  passed identically (11,725 targets, 11,520 experts = 30x128x3, routers+vision BF16)
- 49 GiB bf16 relayed gx10 -> ana-ml2 (no key path either way; nh3-dev relays),
  checksums verified against source; quant 49 -> 16 GiB, all post-steps clean
- stacks/erp-seat: .env-driven swap to erp-tune-v7-nvfp4a16, served under its TRUE
  name; homepage labels + README updated, v6 rollback path recorded
- stacks/litellm: trial -> erp-tune-v7-nvfp4a16 (config-file alias; /model/update
  refuses a config model, so this is an edit + restart)
This commit is contained in:
vh
2026-09-09 15:30:15 -07:00
parent c335c38c19
commit 6972e7ef7f
5 changed files with 60 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/litellm/conf/config.yaml
+22 -15
View File
@@ -851,23 +851,30 @@ model_list:
# api_base: http://10.250.50.54:8099/v1
# api_key: os.environ/VLLM_API_KEY
# trial -> ERP/RP SFT RUN 6, NVFP4A16 (weight-only) quant of the merged LoRA, served on
# ana-ml2 GPU1 (:8021, vLLM v0.26.0, stacks/erp-seat -- a REAL compose stack with a restart
# policy, unlike the run-3c/run-5 hand-launched gx10 seats). Backing artifact
# /tank/aimodels/erp-tune-v6-nvfp4a16 (quant pipeline services/erp-seat-quant/).
# trial -> ERP/RP SFT RUN 7, NVFP4A16 (weight-only) quant of the merged LoRA, served on
# ana-ml2 GPU1 (:8021, vLLM nightly 311b3513, stacks/erp-seat -- a REAL compose stack with a
# restart policy, unlike the run-3c/run-5 hand-launched gx10 seats). Backing artifact
# /tank/aimodels/erp-tune-v7-nvfp4a16 (quant pipeline services/erp-seat-quant/).
#
# NAME: deliberately `trial`, NOT `erp-tune-v6-nvfp4a16`. Repointed 2026-09-08 (evening)
# from run 5 (bf16 on gx10) to run 6 (NVFP4 on ana-ml2) at the operator's request -- "alias
# erp-tune-v6-nvfp4 to trial, please. no gate." -- to hand-test prosody/feel in Open WebUI.
# NAME: deliberately `trial`, NOT `erp-tune-v7-nvfp4a16`. Repointed 2026-09-09 (afternoon)
# from run 6 to run 7 at the operator's request -- "quant the latest train to nvfp4 and serve
# on ana-ml2 as the trial seat". Same standing purpose: hand-testing prosody/feel in Open
# WebUI. (Repointed 2026-09-08 from run 5 to run 6 on the same standing instruction.)
#
# BASE: run 6 is the line's first GENUINELY abliterated base (jenerallee78 ARA @ 0631379a,
# index 33c59654); runs 3-5 were settled from bytes as stock. Run-5 recipe byte-held.
# RUN 7 = run 6 + ONE variable: an opening-split slot (293 rows) and its companion loss mask
# (224 entries, union with lossmask-r3). Base HELD from run 6 (jenerallee78 ARA @ 0631379a,
# index 33c59654). Trained 542/542 steps, train_loss 3.205, adapter 2026-09-09 13:23 PT.
# Runbook docs/runbooks/gx10-run-07.md.
#
# GATE: NONE on this artifact, by operator ruling. brokkr's run-6 gate runs against the bf16
# arm on gx10:8098 (erp-tune-v6), not this NVFP4 build; this seat has a smoke test and a
# 3-run decode probe (~200 tok/s single-stream) only. Refusal behaviour is expected to be
# LOWER than run 5's (abliterated base + k=5 base profile ~0% on 30/35 axes per brokkr) --
# this seat is for the operator's ear; treat it as unrated on every safety axis.
# GATE: NONE on this artifact, by operator ruling -- unchanged from run 6. brokkr's run-7
# gate runs against the BF16 arm on gx10:8098, not this NVFP4 build; this seat has a smoke
# test only. Refusal behaviour is expected to be LOWER than run 5's (abliterated base + k=5
# base profile ~0% on 30/35 axes per brokkr) -- this seat is for the operator's ear; treat
# it as unrated on every safety axis.
#
# ROLLBACK: /tank/aimodels/erp-tune-v6-nvfp4a16 is still on disk, and the previous host env
# is at /tmp/erp-seat-env.v6.bak on ana-ml2 -- flip ERP_MODEL/ERP_SERVED_NAME/ERP_CHAT_TEMPLATE
# in /opt/docker/compose/erp-seat/.env back to v6 and `docker compose up -d`.
#
# THIS GATEWAY IS THE SHARED-KEY SURFACE: `all-agents-local` reaches every model here,
# in every session and project. Removing this alias does not remove the operator's
@@ -877,7 +884,7 @@ model_list:
- model_name: trial
litellm_params:
model: hosted_vllm/erp-tune-v6-nvfp4a16
model: hosted_vllm/erp-tune-v7-nvfp4a16
api_base: http://10.250.50.54:8021/v1
api_key: os.environ/VLLM_API_KEY