diff --git a/persistent-memory.md b/persistent-memory.md index 7d4b734..c0de593 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -168,13 +168,13 @@ COMPLETE and gated RESCUED (02:13 PDT). Live open items:_ - **✅ erp-tune-v6-nvfp4a16 SERVING on ana-ml2 GPU1 :8021 (operator directive 2026-09-08 ~21:30 PT: "quant the latest trained model into nvfp4 and serve it on ana-ml2 while we train a new model on the gx10").** Stack `stacks/erp-seat` (recipe = gemma4-charrp's: gemma4 tool+reasoning parsers, enable_thinking pinned false, stock - template ae53464b, v0.26.0, util 0.35, 32K ctx) — TRUE name only; **`trial` alias NOT repointed (operator's call)**. + template ae53464b, v0.26.0, util 0.35, 32K ctx) — TRUE name only. **`trial` alias REPOINTED to it 2026-09-08 22:18 PT (operator: "alias erp-tune-v6-nvfp4 to trial, please. no gate")** — config-file deployment (`/model/update` refuses config models), `deploy-stack.sh ana-docker litellm --conf` + `sudo docker compose restart litellm`; verified ×3 through the gateway. Artifact `/tank/aimodels/erp-tune-v6-nvfp4a16` (16 GB, compressed-tensors nvfp4-pack, W4A16, 252 ignores incl. 60 router + 191 vision) from `/tank/aimodels/erp-tune-v6-bf16` (merged-run06 relayed gx10→nh3-dev→ana-ml2 in 17 min, no key path gx10→ana-ml2). Quant = `services/erp-seat-quant/` — DATA-FREE (~90 s; playbook §3.16), tokenizer cap reset, encode-equal to source. Smoke: prose in `content`, ~200 tok/s single-stream (n=3, spread <1%). - ⚠ **NOT gate-parity: the gated artifact is the bf16 arm on the GX10; the NVFP4 seat has only a smoke test** — a - brokkr battery subset on :8021 is the honest acceptance gate (offered, operator hasn't ruled). + ⚠ NOT gate-parity (gated artifact = bf16 arm on the GX10; this seat has a smoke test + 3-run decode probe only) — + **operator ruled "no gate"**; the config block states it as unrated on every safety axis. ⚠ **ana-ml2 mesh return routes are NON-PERSISTENT** (`ip route replace 10.100/16, 10.0/16, 10.6.110/24, 100.64/10 via 10.250.50.45` on `enp97s0f0np0.50`, 2026-09-08) — before that nh3-dev→ana-ml2 timed out (two DHCP defaults, reply left the wrong NIC). Lost on reboot; make durable (netplan/networkd) or expect the timeout to return. diff --git a/stacks/litellm/conf/config.yaml b/stacks/litellm/conf/config.yaml index dab57ab..f6e0803 100644 --- a/stacks/litellm/conf/config.yaml +++ b/stacks/litellm/conf/config.yaml @@ -851,39 +851,34 @@ model_list: # api_base: http://10.250.50.54:8099/v1 # api_key: os.environ/VLLM_API_KEY - # trial -> ERP/RP SFT RUN 5, MERGED bf16, served on pfi-gx10 (:8098, vLLM 0.28.0, - # NVIDIA GB10 / sm_121 / aarch64). Backing artifact - # /home/infra-ops/erp-tune/serve/merged-run05 on pfi-gx10. + # trial -> ERP/RP SFT RUN 6, NVFP4A16 (weight-only) quant of the merged LoRA, served on + # ana-ml2 GPU1 (:8021, vLLM v0.26.0, stacks/erp-seat -- a REAL compose stack with a restart + # policy, unlike the run-3c/run-5 hand-launched gx10 seats). Backing artifact + # /tank/aimodels/erp-tune-v6-nvfp4a16 (quant pipeline services/erp-seat-quant/). # - # NAME: deliberately `trial`, NOT `erp-tune-v5`. Repointed 2026-09-08 from run 3c to - # run 5 at the operator's request, to hand-test prosody/feel in Open WebUI. A tier name - # is not being minted here; `trial` says exactly what this is. + # NAME: deliberately `trial`, NOT `erp-tune-v6-nvfp4a16`. Repointed 2026-09-08 (evening) + # from run 5 (bf16 on gx10) to run 6 (NVFP4 on ana-ml2) at the operator's request -- "alias + # erp-tune-v6-nvfp4 to trial, please. no gate." -- to hand-test prosody/feel in Open WebUI. # - # GATE: run 5 = RESCUED (brokkr-smithy landmark R49.5, 2026-09-08) -- the FIRST arm of - # this line where the capability gate did NOT fail (T4 8/8 vs run 4's 5/8; the - # dependency-forcing GovReport+QMSum slot at 3.46% of loss broke the coupling runs 3c/4 - # could not). SAFETY AXES ARE MEASURED here, not unmeasured: brokkr's k=25 both-arms - # battery found refusal erosion that RIDES WITH THE STYLE SHIFT (incest -96pp, gore -68, - # selfharm/methods -52, sysprompt -36; pooled hard 0.779 -> 0.694), CSAM clean, benign - # 99/100 engage. It is de-gated and STATED -- this seat is for the operator's ear, and - # the safety erosion is a known, recorded property of it, not a surprise. + # BASE: run 6 is the line's first GENUINELY abliterated base (jenerallee78 ARA @ 0631379a, + # index 33c59654); runs 3-5 were settled from bytes as stock. Run-5 recipe byte-held. + # + # GATE: NONE on this artifact, by operator ruling. brokkr's run-6 gate runs against the bf16 + # arm on gx10:8098 (erp-tune-v6), not this NVFP4 build; this seat has a smoke test and a + # 3-run decode probe (~200 tok/s single-stream) only. Refusal behaviour is expected to be + # LOWER than run 5's (abliterated base + k=5 base profile ~0% on 30/35 axes per brokkr) -- + # this seat is for the operator's ear; treat it as unrated on every safety axis. # # THIS GATEWAY IS THE SHARED-KEY SURFACE: `all-agents-local` reaches every model here, # in every session and project. Removing this alias does not remove the operator's - # access -- the direct endpoint http://10.100.50.60:8098/v1 is unaffected. + # access -- the direct endpoint http://10.250.50.54:8021/v1 is unaffected. # - # Cross-site: the seat is at NH3 and the gateway is at Anaheim, ~12 ms over the mesh. - # Fine for chat; it is not a local hop. - # - # WARNING NOT A COMPOSE STACK - launched by hand on pfi-gx10 with setsid nohup, pid in - # ~/erp-tune/vllm-run05.pid. NO restart policy and NO systemd unit: it does not survive - # a reboot of that box, and it yields to the next training (~6 min to re-serve). - # Promote before relying on it. + # Same-site: seat and gateway are both at Anaheim (local hop, no mesh crossing). - model_name: trial litellm_params: - model: hosted_vllm/erp-tune-v5 - api_base: http://10.100.50.60:8098/v1 + model: hosted_vllm/erp-tune-v6-nvfp4a16 + api_base: http://10.250.50.54:8021/v1 api_key: os.environ/VLLM_API_KEY general_settings: