From 1e4d827c5de08e5049e992d1433472b6c4cfbbdc Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Wed, 26 Aug 2026 06:21:09 -0700 Subject: [PATCH] memory: erp-tune-v2 registered in the LiteLLM gateway at operator request Operator asked for it so he can evaluate the failed tune by hand, overriding my not-in-the-gateway recommendation. His call. erp-tune-v1 was DELETED from the config in the same reload rather than repointed, so the name now 400s cleanly instead of 500ing against a stopped backend. Deleting rather than repointing is the point: repointing would resolve a name a consumer already knows to different weights, silently. The config entry carries the failed-gate table, the long-form truncation (9.9%) and degeneracy (4.9%) rates, and the rp-length caveat in-line -- so someone reading the gateway config learns what they are calling without having to find the althing thread. Fleet verified healthy after the restart. --- persistent-memory.md | 1 + stacks/litellm/conf/config.yaml | 67 +++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) diff --git a/persistent-memory.md b/persistent-memory.md index d40fe9d..0a25b66 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -115,6 +115,7 @@ _As of 2026-08-26 ~02:10 PDT โ€” **run 2 is trained, merged, coherence-gated and - **๐Ÿ”ด GATE VERDICT: FAIL โ€” recorded as FAIL, and the T6/T3 trade is the actual result.** brokkr's full write-up at `4973991`, `research/R47-premium-corpus-gate/run02-gate/RESULT-run02-gate.md`. 3,456 generations, zero API errors, digests verified identical so the run is not void. **Gate 1 T6-must-not-regress: 73.5 โ†’ 88.5, +15.0 โœ…** (run 1 FAILED this same axis at โˆ’3.5). **Gate 2 no-task-regresses->1-item: T3 โˆ’12.0, T4 โˆ’5.5 โŒ.** Gate 3 T5 control 100% โœ…. Gate 4 latency 0.11s โœ…. T3 reads 88 on both tuned passes โ€” not variance. **Neither run ships; together they price what the abliteration was costing, which is a question neither could answer alone.** Should-help axis worked: diversity **+0.196 (~15x floor)**, confound-tested against attrition (matched-k=8 keeps +0.19, attrition โ‰ˆ7% of the effect) and against length (story got LONGER 669โ†’727 while rp got SHORTER 137โ†’88, same gain both โ€” a length artifact would have opposite signs). Memorisation none. โ†’ `persistent-memory.d/2026-08-26-erp-run2-complete-and-served.md` - **โš  THE FINDING WITH LEGS BEYOND THIS TUNE: an output-stability regression VISIBLE ONLY ON LONG-FORM.** Truncated **0/384 โ†’ 38/384**, degenerate **0/384 โ†’ 19/384** โ€” and the reasoning battery saw **zero of it on either arm across four passes**, because its answers are short. **A gate composed only of short-answer tasks would have passed this cleanly.** Playbook ยง4.6.3. brokkr's block-1 rp VOID (degeneracy 12/95 past the preregistered 10% budget) was the guard catching it first, recorded as VOID not reinterpreted. - **โš  PIPPA'S 123-WORD PRODUCT CLIP IS IN THE MIX AND THE DPO PAIRS WOULD INHERIT IT.** Measured 08-26: pippa bot turns max **123 exactly**, 100% at-or-under, **0.00% in the 124-130 band** โ€” a wall, not a preference (every other root crosses its p99 smoothly). The mechanism is the exposure asymmetry: **pippa is 70.3% of bot TURNS but 37.5% of bot WORDS, and length is learned per turn.** Generalises โ€” a clipped root is over-represented in the length signal by exactly the ratio its clipping creates. โš  Counter-evidence: the tune landed near pippa's **median 67**, not its **cap 123**, so "learned central tendency" is better supported than "learned the limit". Closing test is the tuned arm's own rp length distribution vs pippa's and bluemoon's. Scripts `/tank/erp-tune/pippa_clip.py`, `clip_share.py`. โ†’ `docs/pfi/erp-dpo-stage-prep.md` +- **๐ŸŸข `erp-tune-v2` IS NOW IN THE LITELLM GATEWAY โ€” operator's explicit request, 2026-08-26**, so he can evaluate the failed tune by hand. Reachable at `10.250.50.70:4000` as `erp-tune-v2` on the shared all-agents key (HTTP 200 verified), direct at `:8098`. **`erp-tune-v1` DELETED from the config in the same reload** โ€” it now returns a clean **400** instead of the **500** it had been throwing since its backing container stopped. Config backed up at `/opt/docker/conf/litellm/config.yaml.bak-pre-erpv2-20260826T131938Z`; fleet verified healthy after the restart (summarizer 200, 32 models). โš  **The config entry carries the FAILED-gate warning, the long-form truncation/degeneracy rates and the rp-length caveat in-line**, so anyone reading the gateway config learns what they are calling without finding this thread. This supersedes my earlier not-in-the-gateway recommendation โ€” the operator overrode it and that is his call. - **๐ŸŸข SEATS: `erp-tune-v2` UP on `:8098`, base arm torn down, GPU0 has ~13 GB spare.** The tune stays served (operator asked; it is coherent โ€” the gate verdict is a research result, not a serving fault) and stays **OUT of the LiteLLM gateway**, which now reads as clearly right rather than merely cautious: **a FAILED tune must not be one alias resolution away from a consumer who has not read the thread.** - **โณ SUPERSEDED โ€” the mid-swap state.** Base arm ran 08:54:52Zโ†’09:03:43Z, **zero errors/truncations/empties/degenerates across 1,728 generations**. Swapped to `erp-tune-v2` on `:8098`; **image digest `sha256:4091d5593f77โ€ฆ` VERIFIED IDENTICAL across both arms** (brokkr's requirement โ€” a differing digest voids the run). brokkr is now running the tuned set with `--freeze-from` the base's marker list. **The floors the tuned deltas must clear:** reasoning core **0.5 pt**, diversity overall **0.0125** (rp 0.0153, story 0.0096), story attractor **0.0000**, memorisation signal 0.00%. โš  **The rp family froze ZERO markers**, so its attractor metric is structurally 0.0 on both arms and cannot discriminate โ€” rp is measured on the distance axis only; do not quote an rp attractor delta. โš  Base's slop attractor: **"Elias" in 92/96 bare-prompt stories** (published figure on the same family is 102/144 โ€” ours is an independent replication at a higher rate, not a novel finding). - **โณ SUPERSEDED โ€” the mid-swap state.** brokkr **withdrew** his both-arms-concurrent requirement (his diversity battery emits the frozen marker list to a *file*, so the arms were never a live dependency; the real requirement is all of ONE arm's passes on ONE instance before the swap). So: **sequential, no fleet seats displaced, operator not woken.** Live now: `erp-eval-base` on `:8099` (`erp-base-stock`, stock bf16 via symlinks into `/tank/aimodels`). **`erp-eval-v2` is DOWN and its container REMOVED**; weights are on disk so it is a cheap restart. Swap back with `/tank/erp-tune/serve-arm.sh tuned` when he reports the base set complete. โš  **Both arms go through `serve-arm.sh`**, which pins the vLLM image **by digest** (`sha256:251eba5cc7c1โ€ฆ`) โ€” a version change between arms is a base swap nobody would see โ€” and mounts `/tank/aimodels` for BOTH arms even though only base needs it, because a mount that differs between arms is a difference between arms. diff --git a/stacks/litellm/conf/config.yaml b/stacks/litellm/conf/config.yaml index fc96984..2cc7596 100644 --- a/stacks/litellm/conf/config.yaml +++ b/stacks/litellm/conf/config.yaml @@ -617,6 +617,73 @@ model_list: # 2026-08-21. Container removed, service deleted from stacks/vllm. The alias # is deleted rather than repointed so the name 404s cleanly. --- + # --- erp-tune-v1 RETIRED 2026-08-26. Its seat was stopped to free ana-ml2 GPU0 for + # run 2 and the alias is DELETED rather than repointed, so the name 404s cleanly. + # Repointing erp-tune-v1 at run 2's weights would resolve a name a consumer already + # knows to different weights, silently. Run 1's artifact is intact at + # /tank/erp-tune/serve/merged-final and can be re-served under its own name. --- + + # erp-tune-v2 -> the in-house ERP/RP SFT, run 2, MERGED bf16 (:8098, vLLM, ana-ml2 GPU0). + # Base: google/gemma-4-26B-A4B-it -- the OFFICIAL INSTRUCT release, NOT an abliteration. + # That is the one intended variable against run 1, which trained on an abliterated + # trainee. LoRA r64/a128 on 205 modules, 1 epoch over 20,982 records / 57.7M ctx tokens + # at max_seq_len 16384. Completed 2026-08-26 in 7:22:44, train_loss 2.839, lora_B gate + # 205/205 non-zero. Adds an impersonation loss-mask over 813 bot turns that wrote the + # USER's part (verified by a -221,712 loss-token delta against byte-identical context). + # + # #################################################################################### + # WARNING ITS BEHAVIOURAL GATE FAILED, 2026-08-26. NOT SHIPPABLE. Exposed here at the + # operator's explicit request so he can evaluate it by hand. + # + # gate 2 FAILED T3 constraint-following 100 -> 88 (-12.0 pt, ~1 pt floor, + # both tuned passes read 88 exactly, so it is not variance) + # T4 100 -> 94.5 (-5.5 pt) + # gate 1 PASSED T6 spatial 73.5 -> 88.5 (+15.0) -- run 1 FAILED this same axis + # at -3.5, so the base swap bought 15 points of spatial capability + # and cost 12 of constraint-following. That trade IS the result. + # also passed T5 control 100%, latency 0.11s median + # should-help diversity +0.196 (~15x floor), attractor hit -0.191, + # memorisation none on any root + # + # Full write-up: brokkr-smithy-dev commit 4973991, + # research/R47-premium-corpus-gate/run02-gate/RESULT-run02-gate.md + # #################################################################################### + # + # WARNING KNOWN OUTPUT-STABILITY REGRESSION ON LONG-FORM. If you drive it hard on long + # generations you WILL hit these, and they are the model, not the seat: + # truncated base 0/384 -> tuned 38/384 (9.9%) + # degenerate base 0/384 -> tuned 19/384 (4.9%) + # The reasoning battery saw ZERO of this on either arm across four passes, because its + # answers are short. Invisible to a short-answer gate. + # + # WARNING RP TURNS RUN ~36% SHORTER than the base (88.5 vs 137.1 words). PIPPA is 70.3% + # of the corpus's bot TURNS while being only 37.5% of its words, and its turns are + # hard-clipped at 123 words (a 2023 Character.AI product limit preserved in the + # dataset). Length is learned per turn, so that clip is over-represented in the length + # signal. Suspected cause, not demonstrated. + # + # WARNING bf16, NOT quantized -- deliberate, so the gate's tuned arm matched its bf16 + # base arm and tuning damage could not be confounded with quantization damage. + # + # WARNING 16K context, not 256K. The tune only ever saw sequences <= 16384 and the + # corpus p50 was 2,092 tokens. The base supports 262,144 and LoRA deltas are + # position-independent, but long-session behaviour was never trained. + # + # Serving flags are load-bearing: --reasoning-parser gemma4 PLUS + # --default-chat-template-kwargs enable_thinking=false. Without the second flag the + # parser defaults enable_thinking True and every plain RP response lands in + # reasoning_content with a null content field. + # + # WARNING NOT A COMPOSE STACK - a bare docker run named erp-eval-v2, launched by + # /tank/erp-tune/serve-arm.sh. restart:unless-stopped, so it survives a daemon restart + # but NOT a rebuild. Promote to /opt/docker/compose/ before relying on it. + + - model_name: erp-tune-v2 + litellm_params: + model: hosted_vllm/erp-tune-v2 + api_base: http://10.250.50.54:8098/v1 + api_key: os.environ/VLLM_API_KEY + general_settings: master_key: os.environ/LITELLM_MASTER_KEY database_url: os.environ/DATABASE_URL