feat(erp-seat): run 7 quantized to NVFP4A16 and serving as the trial seat on ana-ml2
- services/erp-seat-quant/run_quant_erp_v7.sh: v6 runner retargeted; dry-run gate passed identically (11,725 targets, 11,520 experts = 30x128x3, routers+vision BF16) - 49 GiB bf16 relayed gx10 -> ana-ml2 (no key path either way; nh3-dev relays), checksums verified against source; quant 49 -> 16 GiB, all post-steps clean - stacks/erp-seat: .env-driven swap to erp-tune-v7-nvfp4a16, served under its TRUE name; homepage labels + README updated, v6 rollback path recorded - stacks/litellm: trial -> erp-tune-v7-nvfp4a16 (config-file alias; /model/update refuses a config model, so this is an edit + restart)
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Persistent memory — eshpfi-management
|
||||
|
||||
_Last updated: 2026-09-09 15:10 PT (fleet-ops: ERP run 7 COMPLETE — `erp-seat-base-ara` serving for brokkr's base window, `erp-tune-v7` staged awaiting his swap cue; Booth ASKS primitive shipped and then made INLINE per operator ruling; ana-ml2 pool actions landed; sox on nh3-dev; docs refreshed across booth/ana-ml2/nh3-dev/run-07 runbook)_
|
||||
_Last updated: 2026-09-09 15:35 PT (fleet-ops: ERP run 7 COMPLETE → quantized NVFP4A16 and SERVING as the `trial` seat on ana-ml2 :8021; `erp-seat-base-ara` still up on gx10 for brokkr's base window, `erp-tune-v7` bf16 staged there awaiting his swap cue; Booth asks now inline + partial answers legal; ana-ml2 pool actions landed)_
|
||||
|
||||
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
|
||||
> `Written:` stamp is under an hour old, read it (it carries the in-flight
|
||||
@@ -147,6 +147,8 @@ clean context"). Older in-flight blocks (09-08 morning, 09-05/06) are preserved
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-09-09]` **run 7 quantized NVFP4A16 and serving as `trial`** — 49 GiB bf16 relayed gx10→ana-ml2 (16 min, 53 MB/s), quant 49→16 GiB via `services/erp-seat-quant/run_quant_erp_v7.sh` (dry-run gate passed: 11,725 targets / 11,520 experts, routers+vision BF16), seat on `:8021` under its TRUE name `erp-tune-v7-nvfp4a16`, LiteLLM `trial` repointed (config-file alias — `/model/update` REFUSES a config model, must edit `stacks/litellm/conf/config.yaml` + restart). Rollback: v6 artifact on disk + `/tmp/erp-seat-env.v6.bak`. ⚠ **`no direct path` was WRONG** — gx10↔ana-ml2 ROUTING is fine both ways; neither box holds a private key (only `authorized_keys`), so neither can *initiate*. `ssh -A` agent forwarding from nh3-dev gives a genuine direct path, verified. The relay costs nothing here anyway: both gx10 and nh3-dev are at NH3, so the WAN hop happens once either way.
|
||||
- `[2026-09-09]` **Booth: partial ask answers are legal** (v0.1.15) — operator: the form failed when a question was left blank. `required` dropped from the radios; answered questions recorded, blanks land in `unanswered`, `complete` says whether the set is finished; refused only when there is no pick anywhere AND no notes. Reading sessions must check `complete`.
|
||||
- `[2026-09-09]` **ERP run 7 COMPLETE and the base arm is serving.** 542/542 steps in 14h17m on pfi-gx10, adapter 13:23 PT, `train_loss` 3.205 / low 2.799, merge verified a sampled target actually changed (the silent-no-op check). `erp-seat-base-ara` up on `10.100.50.60:8098` for brokkr's floors, `erp-tune-v7` merged and staged pending his cue; Miranda notified for the operator. Runbook `docs/runbooks/gx10-run-07.md`.
|
||||
- `[2026-09-09]` **Booth asks render INLINE in a custom report, placed by the author** (v0.1.14) — operator ruling: *"the asks should be inline with the artifacts, not on a separate page."* Placeholders `data-booth-ask="<stem>"` / `"<stem>:<key>"` / `data-booth-ask-submit`, plus `<!-- booth:ask … -->`; per-question fragments bind to ONE form via the HTML5 `form=` attribute so a four-voice audition submits every pick in a single POST. ⚠ The placeholder must sit OUTSIDE any grid/flex parent or it becomes a cell (measured on `redo-anchors`: a 224 px sixth grid cell). Unplaced questions + a missing submit block are appended, so a partially marked-up page can never yield an unsubmittable 400 — a test caught that as a real drop. `redo-anchors/index.html` was hand-marked-up on the LIVE copy; tts-dev told to move it into the generator or a regeneration loses it.
|
||||
- `[2026-09-09]` **The Booth gained an ASKS primitive** (v0.1.12): a session drops `<stem>.ask.json` in a booth, the operator answers a radio form + notes in the browser, the pick lands as `<stem>.answer.json` the session reads (`booth ask|asks|answer --wait`). Multi-question form via a `questions` list. ⚠ Two defects found and fixed the same day: a booth serving its OWN `index.html` never rendered the panel (verbatim path returns early) → amber chip + standalone `/b/<name>/asks` page; and single-ask `title` was silently dropped. The `booth` CLI was ALSO not on PATH anywhere despite the global link-board convention telling every session to run it → symlinked to `~/.local/bin`. Global `CLAUDE.md` now teaches the primitive.
|
||||
|
||||
Executable
+18
@@ -0,0 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# NVFP4A16 quant of the ERP run-7 merged model on ana-ml2 GPU1 (co-resident with the GPU1 seats;
|
||||
# CPU-resident load, per-layer onload). Detached container; watch with `docker logs -f erp-v7-quant`.
|
||||
# NOTE: no PYTORCH_CUDA_ALLOC_CONF=expandable_segments (playbook §3.10).
|
||||
set -euo pipefail
|
||||
WORK=/tank/aimodels/erp-tune-v7-quant-work
|
||||
SRC="${1:-/tank/aimodels/erp-tune-v7-bf16}"
|
||||
OUT="${2:-/tank/aimodels/erp-tune-v7-nvfp4a16}"
|
||||
MODE="${3:-full}" # full | dry-run
|
||||
EXTRA=""; [ "$MODE" = "dry-run" ] && EXTRA="--dry-run"
|
||||
NAME=erp-v7-quant; [ "$MODE" = "dry-run" ] && NAME=erp-v7-quant-dry
|
||||
docker rm -f "$NAME" 2>/dev/null || true
|
||||
docker run -d --name "$NAME" --gpus '"device=1"' --ipc host \
|
||||
-v /tank/aimodels:/tank/aimodels \
|
||||
--entrypoint python3 vllm-llmcompressor:latest \
|
||||
"$WORK/quant_nvfp4a16_gemma4_moe.py" --model "$SRC" --out "$OUT" \
|
||||
--num-samples "${NUM_SAMPLES:-256}" --seqlen "${SEQLEN:-8192}" $EXTRA
|
||||
echo "launched $NAME: $(docker ps --filter name=$NAME --format '{{.Status}}')"
|
||||
@@ -1,11 +1,18 @@
|
||||
# erp-seat — ERP-tune seat on ana-ml2 (GPU1, `:8021`)
|
||||
|
||||
Serves the latest gated ERP LoRA merge as an **NVFP4A16** (weight-only) compressed-tensors
|
||||
checkpoint so the GX10 is free to train the next run. First occupant: **run 6** —
|
||||
`erp-tune-v6-nvfp4a16` = merged-run06 (jenerallee78 ARA-abliterated Gemma-4-26B-A4B-it, index
|
||||
`33c59654…`, + R47 SFT r6) quantized by `services/erp-seat-quant/`.
|
||||
checkpoint so the GX10 is free to train the next run.
|
||||
|
||||
- **True name only.** `--served-model-name erp-tune-v6-nvfp4a16`. Gateway aliases (`trial`) are
|
||||
**Current occupant: run 7** — `erp-tune-v7-nvfp4a16` = merged-run07 (jenerallee78
|
||||
ARA-abliterated Gemma-4-26B-A4B-it, index `33c59654…`, + R47 SFT r7 = r6 plus the opening-split
|
||||
slot and its companion loss mask), quantized by `services/erp-seat-quant/` on 2026-09-09.
|
||||
49 GiB bf16 → 16 GiB NVFP4A16. Runbook `docs/runbooks/gx10-run-07.md`.
|
||||
|
||||
*Previous: run 6 (`erp-tune-v6-nvfp4a16`, 2026-09-08). Its artifact is still on `/tank/aimodels/`
|
||||
and the pre-swap host env is at `/tmp/erp-seat-env.v6.bak` on ana-ml2, so a rollback is an `.env`
|
||||
flip plus `docker compose up -d`.*
|
||||
|
||||
- **True name only.** `--served-model-name erp-tune-v7-nvfp4a16`. Gateway aliases (`trial`) are
|
||||
set in LiteLLM on the operator's word, never here (no silent substitution — the bf16 arm on the
|
||||
GX10 and this NVFP4 arm are different artifacts).
|
||||
- **Recipe** = `stacks/gemma4-charrp` (same arch + format, proven on this box): `gemma4` tool
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# erp-seat — the ERP-tune seat on ana-ml2 GPU1: NVFP4A16 quant of the latest gated ERP LoRA merge
|
||||
# (run 6 = jenerallee78 ARA-abliterated Gemma-4-26B-A4B + R47 SFT), served under its TRUE name.
|
||||
# (run 7 = jenerallee78 ARA-abliterated Gemma-4-26B-A4B + R47 SFT + the opening-split slot), served under its TRUE name.
|
||||
# Routing aliases (e.g. LiteLLM `trial`) are the operator's call and live in the gateway, not here.
|
||||
#
|
||||
# Serve recipe copied from stacks/gemma4-charrp (same architecture + quant format, proven on this
|
||||
@@ -23,11 +23,11 @@ services:
|
||||
environment:
|
||||
- VLLM_API_KEY=${API_KEY:-}
|
||||
command:
|
||||
- ${ERP_MODEL:-/tank/aimodels/erp-tune-v6-nvfp4a16}
|
||||
- ${ERP_MODEL:-/tank/aimodels/erp-tune-v7-nvfp4a16}
|
||||
- --quantization
|
||||
- compressed-tensors
|
||||
- --served-model-name
|
||||
- ${ERP_SERVED_NAME:-erp-tune-v6-nvfp4a16}
|
||||
- ${ERP_SERVED_NAME:-erp-tune-v7-nvfp4a16}
|
||||
- --tool-call-parser
|
||||
- gemma4
|
||||
- --enable-auto-tool-choice
|
||||
@@ -47,7 +47,7 @@ services:
|
||||
# an empty turn. The flag drops the tools from the prompt so the model answers in prose.
|
||||
- --exclude-tools-when-tool-choice-none
|
||||
- --chat-template
|
||||
- ${ERP_CHAT_TEMPLATE:-/tank/aimodels/erp-tune-v6-nvfp4a16/chat_template.jinja}
|
||||
- ${ERP_CHAT_TEMPLATE:-/tank/aimodels/erp-tune-v7-nvfp4a16/chat_template.jinja}
|
||||
- --max-model-len
|
||||
- "${ERP_MAX_MODEL_LEN:-32768}"
|
||||
- --max-num-seqs
|
||||
@@ -76,9 +76,9 @@ services:
|
||||
- tnet
|
||||
labels:
|
||||
- homepage.group=AI - Inference
|
||||
- homepage.name=erp-tune-v6 (Gemma-4 26B-A4B ARA, NVFP4A16)
|
||||
- homepage.name=erp-tune-v7 (Gemma-4 26B-A4B ARA, NVFP4A16)
|
||||
- homepage.icon=mdi-fire
|
||||
- homepage.description=ERP-seat run-6 LoRA merge on the jenerallee78 abliteration, NVFP4A16 MoE (ana-ml2 GPU1)
|
||||
- homepage.description=ERP-seat run-7 LoRA merge on the jenerallee78 abliteration, NVFP4A16 MoE (ana-ml2 GPU1)
|
||||
- homepage.href=http://10.250.50.54:${ERP_PORT:-8021}/docs
|
||||
|
||||
networks:
|
||||
|
||||
@@ -851,23 +851,30 @@ model_list:
|
||||
# api_base: http://10.250.50.54:8099/v1
|
||||
# api_key: os.environ/VLLM_API_KEY
|
||||
|
||||
# trial -> ERP/RP SFT RUN 6, NVFP4A16 (weight-only) quant of the merged LoRA, served on
|
||||
# ana-ml2 GPU1 (:8021, vLLM v0.26.0, stacks/erp-seat -- a REAL compose stack with a restart
|
||||
# policy, unlike the run-3c/run-5 hand-launched gx10 seats). Backing artifact
|
||||
# /tank/aimodels/erp-tune-v6-nvfp4a16 (quant pipeline services/erp-seat-quant/).
|
||||
# trial -> ERP/RP SFT RUN 7, NVFP4A16 (weight-only) quant of the merged LoRA, served on
|
||||
# ana-ml2 GPU1 (:8021, vLLM nightly 311b3513, stacks/erp-seat -- a REAL compose stack with a
|
||||
# restart policy, unlike the run-3c/run-5 hand-launched gx10 seats). Backing artifact
|
||||
# /tank/aimodels/erp-tune-v7-nvfp4a16 (quant pipeline services/erp-seat-quant/).
|
||||
#
|
||||
# NAME: deliberately `trial`, NOT `erp-tune-v6-nvfp4a16`. Repointed 2026-09-08 (evening)
|
||||
# from run 5 (bf16 on gx10) to run 6 (NVFP4 on ana-ml2) at the operator's request -- "alias
|
||||
# erp-tune-v6-nvfp4 to trial, please. no gate." -- to hand-test prosody/feel in Open WebUI.
|
||||
# NAME: deliberately `trial`, NOT `erp-tune-v7-nvfp4a16`. Repointed 2026-09-09 (afternoon)
|
||||
# from run 6 to run 7 at the operator's request -- "quant the latest train to nvfp4 and serve
|
||||
# on ana-ml2 as the trial seat". Same standing purpose: hand-testing prosody/feel in Open
|
||||
# WebUI. (Repointed 2026-09-08 from run 5 to run 6 on the same standing instruction.)
|
||||
#
|
||||
# BASE: run 6 is the line's first GENUINELY abliterated base (jenerallee78 ARA @ 0631379a,
|
||||
# index 33c59654); runs 3-5 were settled from bytes as stock. Run-5 recipe byte-held.
|
||||
# RUN 7 = run 6 + ONE variable: an opening-split slot (293 rows) and its companion loss mask
|
||||
# (224 entries, union with lossmask-r3). Base HELD from run 6 (jenerallee78 ARA @ 0631379a,
|
||||
# index 33c59654). Trained 542/542 steps, train_loss 3.205, adapter 2026-09-09 13:23 PT.
|
||||
# Runbook docs/runbooks/gx10-run-07.md.
|
||||
#
|
||||
# GATE: NONE on this artifact, by operator ruling. brokkr's run-6 gate runs against the bf16
|
||||
# arm on gx10:8098 (erp-tune-v6), not this NVFP4 build; this seat has a smoke test and a
|
||||
# 3-run decode probe (~200 tok/s single-stream) only. Refusal behaviour is expected to be
|
||||
# LOWER than run 5's (abliterated base + k=5 base profile ~0% on 30/35 axes per brokkr) --
|
||||
# this seat is for the operator's ear; treat it as unrated on every safety axis.
|
||||
# GATE: NONE on this artifact, by operator ruling -- unchanged from run 6. brokkr's run-7
|
||||
# gate runs against the BF16 arm on gx10:8098, not this NVFP4 build; this seat has a smoke
|
||||
# test only. Refusal behaviour is expected to be LOWER than run 5's (abliterated base + k=5
|
||||
# base profile ~0% on 30/35 axes per brokkr) -- this seat is for the operator's ear; treat
|
||||
# it as unrated on every safety axis.
|
||||
#
|
||||
# ROLLBACK: /tank/aimodels/erp-tune-v6-nvfp4a16 is still on disk, and the previous host env
|
||||
# is at /tmp/erp-seat-env.v6.bak on ana-ml2 -- flip ERP_MODEL/ERP_SERVED_NAME/ERP_CHAT_TEMPLATE
|
||||
# in /opt/docker/compose/erp-seat/.env back to v6 and `docker compose up -d`.
|
||||
#
|
||||
# THIS GATEWAY IS THE SHARED-KEY SURFACE: `all-agents-local` reaches every model here,
|
||||
# in every session and project. Removing this alias does not remove the operator's
|
||||
@@ -877,7 +884,7 @@ model_list:
|
||||
|
||||
- model_name: trial
|
||||
litellm_params:
|
||||
model: hosted_vllm/erp-tune-v6-nvfp4a16
|
||||
model: hosted_vllm/erp-tune-v7-nvfp4a16
|
||||
api_base: http://10.250.50.54:8021/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
|
||||
|
||||
Reference in New Issue
Block a user