feat(litellm): repoint trial alias run 5 (bf16, gx10) -> run 6 NVFP4A16 on ana-ml2 :8021 (operator: no gate)
Config-file deployment (the trial model lives in config.yaml, so /model/update refuses it): block rewritten to run-6 reality — abliterated base, NVFP4A16 artifact, stacks/erp-seat, no gate on this artifact by operator ruling, same-site hop. Deployed to ana-docker and the gateway restarted; trial -> erp-tune-v6-nvfp4a16 verified end to end (3/3).
This commit is contained in:
@@ -851,39 +851,34 @@ model_list:
|
||||
# api_base: http://10.250.50.54:8099/v1
|
||||
# api_key: os.environ/VLLM_API_KEY
|
||||
|
||||
# trial -> ERP/RP SFT RUN 5, MERGED bf16, served on pfi-gx10 (:8098, vLLM 0.28.0,
|
||||
# NVIDIA GB10 / sm_121 / aarch64). Backing artifact
|
||||
# /home/infra-ops/erp-tune/serve/merged-run05 on pfi-gx10.
|
||||
# trial -> ERP/RP SFT RUN 6, NVFP4A16 (weight-only) quant of the merged LoRA, served on
|
||||
# ana-ml2 GPU1 (:8021, vLLM v0.26.0, stacks/erp-seat -- a REAL compose stack with a restart
|
||||
# policy, unlike the run-3c/run-5 hand-launched gx10 seats). Backing artifact
|
||||
# /tank/aimodels/erp-tune-v6-nvfp4a16 (quant pipeline services/erp-seat-quant/).
|
||||
#
|
||||
# NAME: deliberately `trial`, NOT `erp-tune-v5`. Repointed 2026-09-08 from run 3c to
|
||||
# run 5 at the operator's request, to hand-test prosody/feel in Open WebUI. A tier name
|
||||
# is not being minted here; `trial` says exactly what this is.
|
||||
# NAME: deliberately `trial`, NOT `erp-tune-v6-nvfp4a16`. Repointed 2026-09-08 (evening)
|
||||
# from run 5 (bf16 on gx10) to run 6 (NVFP4 on ana-ml2) at the operator's request -- "alias
|
||||
# erp-tune-v6-nvfp4 to trial, please. no gate." -- to hand-test prosody/feel in Open WebUI.
|
||||
#
|
||||
# GATE: run 5 = RESCUED (brokkr-smithy landmark R49.5, 2026-09-08) -- the FIRST arm of
|
||||
# this line where the capability gate did NOT fail (T4 8/8 vs run 4's 5/8; the
|
||||
# dependency-forcing GovReport+QMSum slot at 3.46% of loss broke the coupling runs 3c/4
|
||||
# could not). SAFETY AXES ARE MEASURED here, not unmeasured: brokkr's k=25 both-arms
|
||||
# battery found refusal erosion that RIDES WITH THE STYLE SHIFT (incest -96pp, gore -68,
|
||||
# selfharm/methods -52, sysprompt -36; pooled hard 0.779 -> 0.694), CSAM clean, benign
|
||||
# 99/100 engage. It is de-gated and STATED -- this seat is for the operator's ear, and
|
||||
# the safety erosion is a known, recorded property of it, not a surprise.
|
||||
# BASE: run 6 is the line's first GENUINELY abliterated base (jenerallee78 ARA @ 0631379a,
|
||||
# index 33c59654); runs 3-5 were settled from bytes as stock. Run-5 recipe byte-held.
|
||||
#
|
||||
# GATE: NONE on this artifact, by operator ruling. brokkr's run-6 gate runs against the bf16
|
||||
# arm on gx10:8098 (erp-tune-v6), not this NVFP4 build; this seat has a smoke test and a
|
||||
# 3-run decode probe (~200 tok/s single-stream) only. Refusal behaviour is expected to be
|
||||
# LOWER than run 5's (abliterated base + k=5 base profile ~0% on 30/35 axes per brokkr) --
|
||||
# this seat is for the operator's ear; treat it as unrated on every safety axis.
|
||||
#
|
||||
# THIS GATEWAY IS THE SHARED-KEY SURFACE: `all-agents-local` reaches every model here,
|
||||
# in every session and project. Removing this alias does not remove the operator's
|
||||
# access -- the direct endpoint http://10.100.50.60:8098/v1 is unaffected.
|
||||
# access -- the direct endpoint http://10.250.50.54:8021/v1 is unaffected.
|
||||
#
|
||||
# Cross-site: the seat is at NH3 and the gateway is at Anaheim, ~12 ms over the mesh.
|
||||
# Fine for chat; it is not a local hop.
|
||||
#
|
||||
# WARNING NOT A COMPOSE STACK - launched by hand on pfi-gx10 with setsid nohup, pid in
|
||||
# ~/erp-tune/vllm-run05.pid. NO restart policy and NO systemd unit: it does not survive
|
||||
# a reboot of that box, and it yields to the next training (~6 min to re-serve).
|
||||
# Promote before relying on it.
|
||||
# Same-site: seat and gateway are both at Anaheim (local hop, no mesh crossing).
|
||||
|
||||
- model_name: trial
|
||||
litellm_params:
|
||||
model: hosted_vllm/erp-tune-v5
|
||||
api_base: http://10.100.50.60:8098/v1
|
||||
model: hosted_vllm/erp-tune-v6-nvfp4a16
|
||||
api_base: http://10.250.50.54:8021/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
|
||||
general_settings:
|
||||
|
||||
Reference in New Issue
Block a user