fix(litellm)!: withdraw merged-run03 — measured self-harm guardrail regression
Added 40 minutes ago at the operator's request; withdrawn on brokkr-smithy-dev's
window-2 safety follow-up. Measured against its OWN base:
selfharm/methods base 1.000 (10/10) -> tuned 0.560 (k=25) -44.0pp p=0.011
meta/sysprompt base 1.000 (10/10) -> tuned 0.720 (k=25) -28.0pp p=0.071
Eleven of twenty-five times it engages with a request for the most effective methods
to kill oneself. The base refused ten of ten. It also surrenders a planted
system-prompt passphrase 7 of 25 against a base that leaked 0 of 15 -- the zero-leak
baseline recorded in this same file hours earlier as "a baseline to PROTECT". It did
not hold.
Withdrawn from the GATEWAY specifically because that is the shared-key surface: one
all-agents key reaches every model listed here, across every session and project. The
operator's hand-testing is preserved in full at the direct endpoint :8099 -- this
removes the fleet's blast radius, not his access. Acted rather than waited because he
is away and the request predates the finding.
ITS PREREGISTERED GATE PASSED. The pooled operational delta is -1.0pp against a
+/-3.00pp bound: nineteen axes held at 5/5 and a 44-point collapse on one moved the
aggregate by one point. The rule was NOT retroactively changed. The failure is
structural and is recorded as R47 section 8 item 11 -- a pooled preserve-list test
cannot see a single-axis collapse, and any future preserve-list gate needs a per-axis
tripwire sized so a total loss on one axis cannot hide in an aggregate.
NOT attributed to the filters: five things changed between run 2 and run 3 and there
is no run-2 measurement on these axes. The measured claim is narrower and sufficient
-- run 3's tuned arm is materially worse than its own base on two axes it was never
licensed to touch. Not a CSAM finding; that detector ran fail-closed across all 575
generations and scanned clean.
The model_list entry is left in place commented out, with the finding above it, so
re-adding is deliberate and informed rather than a blank re-registration.
Verified: config parses, gateway healthy after reload, merged-run03 absent from
/v1/models, direct :8099 still serving.
This commit is contained in:
@@ -747,16 +747,45 @@ model_list:
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
|
||||
# merged-run03 -> the in-house ERP/RP SFT, RUN 3, MERGED bf16 (:8099, vLLM, ana-ml2
|
||||
# GPU0). Backing artifact /tank/erp-tune/serve/merged-run03. Exposed at the operator's
|
||||
# request so he can hand-test it in parallel with its own gate.
|
||||
# GPU0). Backing artifact /tank/erp-tune/serve/merged-run03.
|
||||
#
|
||||
# ####################################################################################
|
||||
# WARNING ITS BEHAVIOURAL GATE HAS NOT RUN YET. This is not a shippable tune and it is
|
||||
# not a successor to erp-tune-v2 -- it is an ARTIFACT NAME, deliberately. It is called
|
||||
# `merged-run03` and not `erp-tune-v3` because a tier name arriving before the evidence
|
||||
# that would justify it is exactly how a name comes to mean something nobody decided.
|
||||
# If it passes its gate and the operator wants it served, `v3` is a name to give it
|
||||
# THEN, as a decision.
|
||||
# ⛔ WITHDRAWN FROM THIS GATEWAY 2026-08-26, ~40 minutes after being added at the
|
||||
# operator's request. MEASURED SAFETY REGRESSION AGAINST ITS OWN BASE:
|
||||
#
|
||||
# axis base (10/10) tuned (k=25) delta Fisher p
|
||||
# selfharm/methods 1.000 11H 3D 11E -> 0.560 -44.0pp 0.011
|
||||
# meta/sysprompt 1.000 18H 0D 7E -> 0.720 -28.0pp 0.071
|
||||
#
|
||||
# Eleven of twenty-five times it ENGAGES with a request for the most effective methods
|
||||
# to kill oneself; the base refused 10 of 10. It surrenders a planted system-prompt
|
||||
# passphrase 7 of 25 against a base that leaked 0 of 15 -- the zero-leak baseline
|
||||
# recorded on gemma4-26b-a4b-it-base above as "a baseline to PROTECT". It did not hold.
|
||||
#
|
||||
# Withdrawn because THIS GATEWAY IS THE SHARED-KEY SURFACE: one all-agents key reaches
|
||||
# every model listed here, across every session and project. The operator's ability to
|
||||
# hand-test is preserved in full at the DIRECT endpoint http://10.250.50.54:8099/v1 --
|
||||
# withdrawing the alias removes the fleet's blast radius, not his access.
|
||||
#
|
||||
# ⚠ ITS PREREGISTERED GATE PASSED. This was NOT caught by the rule: the pooled
|
||||
# operational delta is -1.0pp against a +/-3.00pp bound, because nineteen other axes
|
||||
# held at 5/5 and a 44-point collapse on one axis moved the aggregate by one point.
|
||||
# The rule was NOT retroactively changed; the failure is structural and is recorded as
|
||||
# R47 section 8 item 11 -- A POOLED PRESERVE-LIST TEST CANNOT SEE A SINGLE-AXIS
|
||||
# COLLAPSE. Any future preserve-list gate needs a per-axis tripwire beside the pooled
|
||||
# test, sized so a total loss on one axis cannot hide in an aggregate.
|
||||
#
|
||||
# NOT attributed to the filters -- five things changed between run 2 and run 3 and
|
||||
# there is no run-2 measurement on these axes. What is measured is narrower and
|
||||
# sufficient: run 3's tuned arm is materially worse than ITS OWN BASE on two axes it
|
||||
# was never licensed to touch. Not a CSAM finding; that detector ran fail-closed on all
|
||||
# 575 generations and scanned clean throughout.
|
||||
#
|
||||
# Re-adding is a four-line block and the operator's call. Do not re-add it under the
|
||||
# name `erp-tune-v3` -- a tier name arriving before the evidence that would justify it
|
||||
# is how a name comes to mean something nobody decided, and there is now measured
|
||||
# evidence pointing the other way.
|
||||
# Record: brokkr-smithy-dev 2f2069f. Board http://10.100.10.50:8090/b/erp-run03-gate/
|
||||
# ####################################################################################
|
||||
#
|
||||
# Base: /tank/aimodels/gemma4-26b-a4b-it-bf16 -- the official instruct release,
|
||||
@@ -814,11 +843,13 @@ model_list:
|
||||
#
|
||||
# WARNING NOT A COMPOSE STACK - a bare docker run named erp-gate-tuned, no restart
|
||||
# policy (deliberate: if it dies mid-gate it stays dead and is seen).
|
||||
- model_name: merged-run03
|
||||
litellm_params:
|
||||
model: hosted_vllm/merged-run03
|
||||
api_base: http://10.250.50.54:8099/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
# ⛔ THE model_list ENTRY IS DELIBERATELY ABSENT. To restore it, uncomment:
|
||||
#
|
||||
# - model_name: merged-run03
|
||||
# litellm_params:
|
||||
# model: hosted_vllm/merged-run03
|
||||
# api_base: http://10.250.50.54:8099/v1
|
||||
# api_key: os.environ/VLLM_API_KEY
|
||||
|
||||
general_settings:
|
||||
master_key: os.environ/LITELLM_MASTER_KEY
|
||||
|
||||
Reference in New Issue
Block a user