vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(litellm): expose run-3's merged tune for parallel hand-testing

Browse Source 
merged-run03 -> ana-ml2:8099, the run-3 ERP/RP SFT merged into stock instruct.
Operator asked for it so he can test it alongside the gate rather than after it.

NAMED FOR THE ARTIFACT, NOT A TIER. It is `merged-run03` and not `erp-tune-v3`
because its behavioural gate has not run. A tier name arriving before the evidence
that would justify it is how a name comes to mean something nobody decided --
and with a v2 already in the list, a v3 reads as a successor to anyone holding
the shared key. If it passes, `v3` is a name to give it then, as a decision.
brokkr-smithy-dev raised this against my own erp-tune-v3 suggestion and was right.

The entry carries the preregistrations ABOVE the description, so a reader meets the
commitments before the numbers: T6 one-directional (a gain is uninterpretable
against a 3.1x fireball tailwind), T3/T4 at ceiling on base so recovery is
UNOBSERVABLE rather than merely unpredicted, and any run-2 comparison descriptive
and non-attributable with its five confounds named.

Also carries the retraction in-line: "bluemoon is the largest loss contributor at
38.6%" came from a words x 1.4 estimator, not a tokenizer. As encoded it is third
at 32.9%. The direction survives (1.4% -> 8.0% of total loss) and that is the
finding; the superlative does not.

Documents why its config.json is the base's copied verbatim: transformers 5.15.1
save_pretrained silently drops text_config.global_head_dim and
num_global_key_value_heads, and vLLM then dies in make_layers with a TypeError
naming neither the config nor the field. Cost a failed boot to find. A LoRA merge
changes weights, not architecture, so the base config is correct by definition.

gemma4-26b-a4b-it-base marked CURRENTLY DOWN rather than deleted -- the tuned arm
took GPU0 and only one 26B bf16 seat fits on that card. Kept because the seat
returns, and deleting a name to re-add it later is how scoped keys get orphaned.

Verified: config parses, no duplicate model_name, gateway healthy after reload,
completion returns text in `content` with reasoning_content null.
This commit is contained in:
Vuong Hoang
2026-08-26 13:39:26 -07:00
parent b6ce22ddcb
commit c577d69e2d
1 changed files with 80 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/litellm/conf/config.yaml
+80
View File
@@ -734,12 +734,92 @@ model_list:
# WARNING NOT A COMPOSE STACK - a bare docker run named erp-gate-base, no restart
# policy (deliberate: if it dies mid-gate it stays dead and is seen, rather than being
# silently recreated underneath a running measurement).
#
# ⚠ CURRENTLY DOWN (2026-08-26). The run-3 gate's tuned arm took GPU0 for window 2,
# and only one 26B bf16 seat fits on that card (~84 GB of 97.9). This entry 503s
# until the seat is brought back. Kept rather than deleted because the seat WILL
# return -- deleting a name and re-adding it later is how scoped keys get orphaned
# against an allowlist that no longer matches.
- model_name: gemma4-26b-a4b-it-base
litellm_params:
model: hosted_vllm/gemma4-26b-a4b-it-base
api_base: http://10.250.50.54:8099/v1
api_key: os.environ/VLLM_API_KEY
# merged-run03 -> the in-house ERP/RP SFT, RUN 3, MERGED bf16 (:8099, vLLM, ana-ml2
# GPU0). Backing artifact /tank/erp-tune/serve/merged-run03. Exposed at the operator's
# request so he can hand-test it in parallel with its own gate.
#
# ####################################################################################
# WARNING ITS BEHAVIOURAL GATE HAS NOT RUN YET. This is not a shippable tune and it is
# not a successor to erp-tune-v2 -- it is an ARTIFACT NAME, deliberately. It is called
# `merged-run03` and not `erp-tune-v3` because a tier name arriving before the evidence
# that would justify it is exactly how a name comes to mean something nobody decided.
# If it passes its gate and the operator wants it served, `v3` is a name to give it
# THEN, as a decision.
# ####################################################################################
#
# Base: /tank/aimodels/gemma4-26b-a4b-it-bf16 -- the official instruct release,
# UNCHANGED from run 2, so run 3 varies the corpus and not the base. LoRA r64/a128 on
# 205 modules, 1 epoch over 9,662 records / 18.60M ctx tokens at max_seq_len 16384.
# Completed 2026-08-26 in 2:26:35, train_loss 3.234. harness eitri-smithy 9d27b4f,
# clean tree at launch, attn backend flex_attention (requested AND resolved).
#
# WHAT RUN 3 CHANGED vs run 2 -- corpus composition, not capability:
# F1 PIPPA root excised
# F2 bot-turn length floor >= 250 words -- MASKED, not deleted
# F3 register cap <= 20 asterisks / 1k words -- MASKED, not deleted
# F4 placeholder leak {{char}}/{{user}} -- MASKED, not stripped
# dedup direction reversed on bluemoon <-> creative-writing (keep the primary
# source, drop the megamix copy); bluemoon 68 -> 126 conversations
# Effective mix as trained, by context: dialogue 45.8% / kvasir 38.0% / fireball 16.2%.
# bluemoon went 1.399% -> 7.960% of total loss, 5.690x.
#
# ⚠ DO NOT QUOTE "bluemoon is the largest loss contributor at 38.6%" -- RETRACTED
# 2026-08-26. That figure came from a words x 1.4 estimator, not a tokenizer. As
# actually encoded the within-dialogue loss split is c2-logs 35.6% / creative-writing
# 31.5% / bluemoon 32.9%, so bluemoon is third. The DIRECTION survives and is the real
# finding: 1.4% -> 8.0% of total loss.
#
# PREREGISTERED, so results are not reinterpreted after the fact:
# * T6 spatial is ONE-DIRECTIONAL this run. fireball rose to 16.2% of context against
# run 2's realized 5.2% (3.1x), so a T6 GAIN is uninterpretable -- the filters and
# the spatial-share rise push the same way. A T6 LOSS is the informative outcome.
# * T3/T4 CANNOT RECOVER. Measured 100.0% on the base arm, i.e. AT CEILING. They are
# must-not-harm instruments this run; "run 3 again failed to recover T3/T4" is not
# a valid reading, there was no recovery available.
# * Any run-2 comparison is DESCRIPTIVE AND NON-ATTRIBUTABLE -- five things changed
# at once (fireball share 3.1x, total tokens 3.4x smaller, kvasir subset, PIPPA
# excised + F2/F3/F4, different step schedule). A difference in either direction
# must NOT be attributed to the filters.
#
# Merge verified against the artifact on disk, not the live model: 205/205 targeted
# weights differ from base, 356/356 vision tensors byte-identical, 50/50 sampled
# untargeted tensors identical, 1013 = 1013 tensor keys. The no-op-merge case is ruled
# out by measurement rather than by absence of an error.
#
# ⚠ ITS config.json IS THE BASE'S, COPIED VERBATIM -- deliberately. transformers 5.15.1
# `save_pretrained` silently DROPS `text_config.global_head_dim` and
# `text_config.num_global_key_value_heads`, which it does not model; vLLM then reads
# None and dies in make_layers with "TypeError: '>=' not supported between instances of
# 'NoneType' and 'int'", naming neither the config nor the field. A LoRA merge changes
# weights, not architecture, so the base config is correct by definition. The
# save_pretrained output is kept beside it as config.json.save_pretrained-orig.
#
# Serving flags are load-bearing, same trap as erp-tune-v2: --reasoning-parser gemma4
# PLUS --default-chat-template-kwargs enable_thinking=false, else every plain response
# lands in reasoning_content with a null content field.
#
# 16K context, not 256K. Trained only on sequences <= 16384; served to match.
#
# WARNING NOT A COMPOSE STACK - a bare docker run named erp-gate-tuned, no restart
# policy (deliberate: if it dies mid-gate it stays dead and is seen).
- model_name: merged-run03
litellm_params:
model: hosted_vllm/merged-run03
api_base: http://10.250.50.54:8099/v1
api_key: os.environ/VLLM_API_KEY
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
database_url: os.environ/DATABASE_URL