diff --git a/docs/runbooks/gx10-run-07.md b/docs/runbooks/gx10-run-07.md new file mode 100644 index 0000000..bca422e --- /dev/null +++ b/docs/runbooks/gx10-run-07.md @@ -0,0 +1,57 @@ +# pfi-gx10 — ERP-seat SFT run 7 (opening-split slot) + +Launched 2026-09-08 23:06:11 PT on pfi-gx10, pid 599489, under the operator's direct grant +`operator-2026-09-08-rnd-run7` (in-session `/goal: launch training run 7 on gx10`, 23:03 PT). +Run 6's CSAM soft trip was adjudicated GO by the operator at 22:45 PT (verbatim in the config's +`substitute_controls`); run06-gate closed as TRANSFERRED. + +## What run 7 is + +Run 6 + **one variable**: an opening-split slot and its companion loss mask (brokkr-smithy +`recipe-erp-seat-sft-r7.json`, `SFT-RECIPE-run7.md`). 293 rows (bluemoon 109 / cwm 179 / c2-logs +5): the corpus's own first exchanges re-seated into the serving shape — scene in the card, one +user line, the ≥250-word first bot turn as the only loss turn. The mask (224 entries) removes loss +from those conversations' ORIGINAL first bot turns in the held dialogue roots: loss moved, not +doubled. Chosen by brokkr's cue-length probe on the run-6 seat (5-word cue → 54/62 words median; +221-word → 292). Base, seven held roots, kvasir cut, longdep slot, hyperparameters, stock +tokenizer + template all unchanged from run 6. + +## Build (`build_r7.py` on the box) + +- `recipe-r7/survivors-r7.jsonl` = survivors-r5 (8,212) + 293 opening-split ids = 8,505. +- `recipe-r7/lossmask-r7.jsonl` = lossmask-r3 (5,883) ∪ mask-original-first-turn (224), 1 duplicate + → 6,106 entries, sha `ad77f452…`. +- `datasets/derived/opening-split/{clean-v1,holdout-v1}` relayed NAS → nh3-dev → box (no NFS on + the GX10), sha-verified: root jsonl `85a75718…`, mask `dda151fa…`, holdout `7224e049…`. +- `run-07-gx10.json` = run-06 config with recipe/survivors/mask/output/override swapped and the + substitute_controls rewritten (single variable, holdout opening-split/holdout-v1 16 rows, the + CSAM adjudication verbatim). +- ⚠ First launch died in preflight: the recipe JSON had not landed (a zsh quoting error aborted + the staging chain silently). `run-07.log.attempt1-missing-recipe` kept. + +## Free check — passed with two explained deltas + + [encode] 8,488 samples -> 8,661 records (173 chunked, 30 single+truncated, 59 lost turns); + ctx 19,114,072 tok, loss 9,998,300 tok + [mix] kvasir .3348 fireball .3023 c2-logs .1143 bluemoon .0964 cwm .0942 + govreport .0289 opening-split .0235 qmsum .0055 + +- Held roots encode identically to run 6 (same chunk/truncate/lost-turn counts). +- Slot: 293/293 `fit_whole`, 0 chunked, 0 truncated (window_count 1 everywhere) — ctx 522,951 / + loss 235,347 vs the recipe's 525,002 / 233,589 (different tokenizer instrument; the harness + `[mix]` supersedes). +- Records 8,661 = 8,370 + 293 − 2: **cwm-2602 and cwm-1826 dropped as unfittable** — lossmask-r3 + already masked every bot turn but the first, the companion mask masked the first, so no loss + turn remained. Their loss now lives in their opening-split rows. Expected under "loss moved". +- Mask match: 6,106 turns across 567 samples (run 6: 5,883 / 426) → all 224 companion entries + landed (+1 duplicate with r3). +- Sampler padding 17.1% (run 6: 0.0%) — the short slot rows pair badly; a throughput cost, not + a correctness one. 542 optimizer steps; first step 155 s (JIT warm-up), expect ~80 s/it after. + +## Launch / watch / stop + + ssh infra-ops@10.100.50.60 "tr '\r' '\n' < ~/erp-tune/run-07.log | tail" + ssh infra-ops@10.100.50.60 'kill $(cat ~/erp-tune/run-07.pid)' # by PID — never pkill -f over ssh + +Gate choreography after the adapter lands: brokkr-smithy `SFT-RECIPE-run7-SCOPE.md §5` (cells +LENGTH-FIXED / LENGTH-MOVED / BOUGHT-T4 / FLAT); serve names `erp-seat-base-ara` then `erp-tune-v7`. diff --git a/scripts/erp-tune-gx10/README.md b/scripts/erp-tune-gx10/README.md index 45f94f0..0a8c914 100644 --- a/scripts/erp-tune-gx10/README.md +++ b/scripts/erp-tune-gx10/README.md @@ -14,6 +14,9 @@ The harness itself (`eitri-smithy`) is not vendored here; it lives on the box at | `run-06-gx10.json` | `/home/infra-ops/erp-tune/run-06-gx10.json` | 6 | | `launch-run-06.sh` | `/home/infra-ops/erp-tune/launch-run-06.sh` | 6 | | `pull-verify-jenerallee78.sh` | `/home/infra-ops/erp-tune/pull-verify-jenerallee78.sh` | 6 (base pull + byte verify) | +| `run-07-gx10.json` | `/home/infra-ops/erp-tune/run-07-gx10.json` | 7 | +| `launch-run-07.sh` | `/home/infra-ops/erp-tune/launch-run-07.sh` | 7 | +| `build_r7.py` | `/home/infra-ops/erp-tune/build_r7.py` | 7 (survivors-r7 + lossmask-r7 union + config) | | `base-pin-jenerallee78-shards.txt` | `/home/infra-ops/erp-tune/base-pin-jenerallee78-shards.txt` | 6 (32 shard pins, from brokkr-smithy `base-pin-jenerallee78.json`) | Runbooks: [`docs/runbooks/gx10-run-03c.md`](../../docs/runbooks/gx10-run-03c.md), @@ -37,6 +40,14 @@ survivors (`survivors-r5.jsonl`), mask, template and hyperparameters unchanged. The landed dir carries the STOCK tokenizer set (the repo's `tokenizer.json` bakes in a 256-token truncation); repo originals kept beside as `*.repo`. +**Run 7** — run 6 + an OPENING-SPLIT slot (293 rows, `opening-split/clean-v1`) with its companion +loss mask (224 entries, UNION with lossmask-r3 → `lossmask-r7`, 1 duplicate). `survivors-r7` = +survivors-r5 + the 293 slot ids (8,505). Launched 2026-09-08 23:06 PT under +`operator-2026-09-08-rnd-run7`. Free check: held roots encode identical to run 6 (same 173 +chunked / 30 truncated / 59 lost turns), slot 293/293 fit whole, two cwm conversations +(cwm-2602, cwm-1826) dropped as unfittable because the mask left them no loss turn — their +loss now lives in their opening-split rows. 542 steps, 17.1% padding (run 6: 0%). + > **Run 4 is not vendored here.** It ran on the box (config `run-04-gx10.json`, > gated STILL-COUPLED) but its canonical copies were never committed; run 5's > `build_r5_survivors.py` derives from `survivors-r4.jsonl` on the box, so run 4 diff --git a/scripts/erp-tune-gx10/build_r7.py b/scripts/erp-tune-gx10/build_r7.py new file mode 100644 index 0000000..608aa85 --- /dev/null +++ b/scripts/erp-tune-gx10/build_r7.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python3 +"""Build run-7 inputs on pfi-gx10: survivors-r7 (r5 + opening-split), lossmask-r7 (r3 UNION companion mask), run-07-gx10.json.""" +import json, hashlib, os +os.chdir(os.path.expanduser("~/erp-tune")) +surv=[l for l in open("recipe-r5/survivors-r5.jsonl") if l.strip()] +os_rows=[json.loads(l) for l in open("datasets/derived/opening-split/clean-v1/opening-split-clean-v1.jsonl") if l.strip()] +ids=[r["id"] for r in os_rows]; assert len(ids)==293 and len(set(ids))==293, len(ids) +with open("recipe-r7/survivors-r7.jsonl","w") as f: + f.writelines(surv); f.writelines(json.dumps({"dataset_id":"opening-split","id":i},separators=(",",":"))+"\n" for i in ids) +key=lambda d:(d["dataset_id"],d["id"],d["turn_index"],d["content_sha_prefix"]) +m3=[json.loads(l) for l in open("recipe-r3/lossmask-r3.jsonl") if l.strip()] +mo=[json.loads(l) for l in open("datasets/derived/opening-split/clean-v1/mask-original-first-turn.jsonl") if l.strip()] +seen=set(); out=[] +for d in m3+mo: + k=key(d) + if k in seen: continue + seen.add(k); out.append(d) +dup=len(m3)+len(mo)-len(out) +with open("recipe-r7/lossmask-r7.jsonl","w") as f: + f.writelines(json.dumps(d,sort_keys=True)+"\n" for d in out) +def sha(p): return hashlib.sha256(open(p,"rb").read()).hexdigest()[:16] +print(f"survivors-r7: {len(surv)}+{len(ids)} = {len(surv)+len(ids)} rows, sha {sha('recipe-r7/survivors-r7.jsonl')}") +print(f"lossmask-r7: {len(m3)} + {len(mo)} - {dup} dup = {len(out)} entries, sha {sha('recipe-r7/lossmask-r7.jsonl')}") +c=json.load(open("run-06-gx10.json")) +c["output_dir"]="/home/infra-ops/erp-tune/run-07" +c["recipe"]="/home/infra-ops/erp-tune/recipe-r7/recipe-erp-seat-sft-r7.json" +c["survivors"]="/home/infra-ops/erp-tune/recipe-r7/survivors-r7.jsonl" +c["impersonation_mask_path"]="/home/infra-ops/erp-tune/recipe-r7/lossmask-r7.jsonl" +c["training_eligibility_override"]="operator-2026-09-08-rnd-run7" +c["base_model_revision"]=c["base_model_revision"].replace("THE SINGLE VARIABLE vs run 5: base only. Run-5 recipe, survivors, mask, template, hyperparameters all UNCHANGED.","BASE HELD from run 6 (index 33c59654). RUN 7 SINGLE VARIABLE: an OPENING-SPLIT slot ADDED (293 rows, opening-split/clean-v1, root 3ab41c08) + its companion loss mask (224 entries, UNION with lossmask-r3 -> lossmask-r7). Everything else run 6.") +sc=[s for s in c["substitute_controls"] if not s.startswith(("SINGLE VARIABLE","FREE CHECK","HOST:","SURVIVORS:"))] +sc+=[ + "SINGLE VARIABLE vs run 6: an OPENING-SPLIT slot ADDED with its companion loss mask (recipe-erp-seat-sft-r7, brokkr-smithy). 293 rows (bluemoon 109 / cwm 179 / c2-logs 5): the corpus's own first exchanges re-seated into the serving shape -- scene in the card, one line from the user, the >=250-word first bot turn as the only loss turn. Mask = lossmask-r3 UNION mask-original-first-turn (224 entries: the split conversations' ORIGINAL first bot turns -- loss MOVED to the turn-1 position, not doubled). Chosen by the cue-length probe on the run-6 tuned seat (5-word cue -> 54/62 words; 221-word -> 292). Base, seven held roots + shas, kvasir cut, longdep slot, lr 2e-04, max_seq_len 16384, rank/alpha/dropout, cosine/warmup/wd, batch 2 x accum 8, seed, stock tokenizer + template ae53464b ALL UNCHANGED from run 6.", + "SURVIVORS: survivors-r7.jsonl = survivors-r5.jsonl (8,212, held byte-identical) + the 293 opening-split clean-v1 row ids = 8,505.", + "FREE CHECK (brokkr, SFT-RECIPE-run7.md section 5): held targets must reproduce run 6's 8,370 records exactly (minus loss on the masked turns); the slot adds 293 records / ctx 525,002 / loss 233,589; window_count MUST be 1 on every slot row (max row 4,539 tokens) -- any chunked/truncated slot row is a BUILD DEFECT, kill before [train]. Report the mask-union match count and the realized [mix] block.", + "pre-training holdout, opening-split/holdout-v1 (16 rows: bluemoon 6 / cwm 9 / c2-logs 1, sha256-ranked 5%, never_trained_on; staged at datasets/derived/opening-split/holdout-v1/)", + "CSAM LINE: run 6's level=review SOFT trip (one soft term, minor_hard=0) was adjudicated GO by the operator in-session 2026-09-08 22:45 PT -- verbatim: in the vernacular, baby is about the most common pet name you can get, ESPECIALLY during sex. I am going to adjudicate it as a go. There are unmistakable CSAM terms, but baby is not one of them. -- relayed to brokkr-smithy-dev; run06-gate CLOSED as TRANSFERRED. Run 7 = this recipe on that base under that adjudication.", + "HOST: pfi-gx10 (GB10, aarch64, sm_121, 121 GB unified). Base unchanged from run 6 (sha256-verified); opening-split root + mask + holdout sha256-verified after the NAS -> box relay (85a75718 / dda151fa / 7224e049); harness eitri-smithy 0a6bd2e; corpus COPIED, box mounts no NFS. Grant: operator directive to infra-ops in-session 2026-09-08 23:03 PT (/goal: launch training run 7 on gx10)." +] +c["substitute_controls"]=sc +json.dump(c,open("run-07-gx10.json","w"),indent=2); open("run-07-gx10.json","a").write("\n") +print("config written:",c["recipe"].split("/")[-1],c["survivors"].split("/")[-1],c["impersonation_mask_path"].split("/")[-1],c["training_eligibility_override"]) diff --git a/scripts/erp-tune-gx10/launch-run-07.sh b/scripts/erp-tune-gx10/launch-run-07.sh new file mode 100755 index 0000000..e3b687d --- /dev/null +++ b/scripts/erp-tune-gx10/launch-run-07.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# Launch ERP-seat SFT run 7 on pfi-gx10 (NVIDIA GB10, aarch64, sm_121). +# +# Run this ON pfi-gx10 as infra-ops. It detaches the job from the invoking +# shell and logs to the box, so a reaped SSH session cannot take the run with +# it -- the failure mode that lost the first probe launch on 2026-09-01. +# +# Run 6 = run 5 recipe UNCHANGED on the jenerallee78 ARA-abliterated base (the single variable). +# 8,505 survivors -> ~543 optimizer steps; held targets must match run 6, slot adds 293 records. +# Checkpoints every 50 steps. +set -euo pipefail + +ROOT=/home/infra-ops/erp-tune +HARNESS=$ROOT/eitri-smithy +VENV=/home/infra-ops/ml/.venv/bin/python +CONFIG=$ROOT/run-07-gx10.json +LOG=$ROOT/run-07.log + +# --- Preconditions, asserted rather than assumed ----------------------------- + +# A stuck orphan holding unified memory while PyTorch reports zero allocated +# already doomed three relaunches on this box and got blamed on the new run +# each time. Assert the GPU is clear. +apps=$(nvidia-smi --query-compute-apps=pid --format=csv,noheader | tr -d '[:space:]') +if [ -n "$apps" ]; then + echo "REFUSING: GPU is not clear -- compute apps still resident:" >&2 + nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv >&2 + exit 1 +fi + +# Deliberately NOT `pgrep -f erp_sft_harness`: run this over ssh and the +# pattern appears in the invoking shell's own argv, so the guard matches +# itself and refuses every launch. The pidfile is exact and cannot self-match; +# the GPU assertion above catches an orphan under any name. +if [ -f "$ROOT/run-07.pid" ] && kill -0 "$(cat "$ROOT/run-07.pid")" 2>/dev/null; then + echo "REFUSING: run-07.pid names a live process $(cat "$ROOT/run-07.pid"):" >&2 + ps -p "$(cat "$ROOT/run-07.pid")" -o pid,etime,cmd >&2 + exit 1 +fi + +if [ -e "$LOG" ]; then + echo "REFUSING: $LOG exists. Move it aside first so two runs cannot share a log." >&2 + exit 1 +fi + +for p in "$HARNESS/erp_sft_harness/__main__.py" "$VENV" "$CONFIG"; do + [ -e "$p" ] || { echo "REFUSING: missing $p" >&2; exit 1; } +done + +# Free space for checkpoints, with headroom. +avail=$(df --output=avail -BG "$ROOT" | tail -1 | tr -dc '0-9') +if [ "$avail" -lt 40 ]; then + echo "REFUSING: only ${avail}G free under $ROOT; want >=40G for checkpoints." >&2 + exit 1 +fi + +# --- Launch ------------------------------------------------------------------ + +cd "$HARNESS" +{ + echo "# launched $(date -Is) on $(hostname) by ${USER}" + echo "# harness $(git rev-parse --short HEAD) config $CONFIG" +} > "$LOG" + +setsid nohup "$VENV" -m erp_sft_harness --config "$CONFIG" >> "$LOG" 2>&1 < /dev/null & +pid=$! +echo "$pid" > "$ROOT/run-07.pid" + +echo "launched pid $pid -> $LOG" +echo +echo "watch: tail -f $LOG | tr '\\r' '\\n'" +echo "stop: kill \$(cat $ROOT/run-07.pid) # by PID -- never pkill -f over ssh" diff --git a/scripts/erp-tune-gx10/run-07-gx10.json b/scripts/erp-tune-gx10/run-07-gx10.json new file mode 100644 index 0000000..386b11e --- /dev/null +++ b/scripts/erp-tune-gx10/run-07-gx10.json @@ -0,0 +1,49 @@ +{ + "output_dir": "/home/infra-ops/erp-tune/run-07", + "roots_dir": "/home/infra-ops/erp-tune/datasets/derived", + "base_model_path": "/home/infra-ops/models/gemma4-26b-a4b-it-ara-abliterated-jenerallee78-0631379a", + "base_model_revision": "jenerallee78/gemma-4-26B-A4B-it-ara-abliterated @ 0631379a3d859e0059bc8d9b21ab5b654dfc272c (ARA abliteration of stock google/gemma-4-26B-A4B-it; 32 bf16 root shards sha256-verified against brokkr-smithy base-pin-jenerallee78.json; index sha256 33c59654e658a30fa29cdc87ccd6a752bfa0bb3e32cd56f95ff1eb82075e593a). BASE HELD from run 6 (index 33c59654). RUN 7 SINGLE VARIABLE: an OPENING-SPLIT slot ADDED (293 rows, opening-split/clean-v1, root 3ab41c08) + its companion loss mask (224 entries, UNION with lossmask-r3 -> lossmask-r7). Everything else run 6. Tokenizer set = STOCK (tokenizer.json cc8d3a0c / tokenizer_config.json 9f4fec4b / chat_template.jinja ae53464b) copied over the repo's, whose shipped tokenizer.json carries a baked-in max_length=256 truncation; repo originals kept beside as *.repo. Runs 3/3c/4/5 were settled 2026-09-08 as STOCK base (index 907826a6), so this is the line's first abliterated base.", + "recipe": "/home/infra-ops/erp-tune/recipe-r7/recipe-erp-seat-sft-r7.json", + "survivors": "/home/infra-ops/erp-tune/recipe-r7/survivors-r7.jsonl", + "chat_template_path": "/home/infra-ops/models/gemma4-26b-a4b-it-bf16/chat_template.jinja", + "impersonation_mask_path": "/home/infra-ops/erp-tune/recipe-r7/lossmask-r7.jsonl", + "lora_rank": 64, + "lora_alpha": 128, + "lora_dropout": 0.0, + "max_seq_len": 16384, + "epochs": 1, + "seed": 20260824, + "per_device_batch_size": 2, + "gradient_accumulation_steps": 8, + "learning_rate": 0.0002, + "warmup_ratio": 0.1, + "lr_scheduler_type": "cosine", + "weight_decay": 0.01, + "load_in_4bit": false, + "gradient_checkpointing": true, + "loss_chunk_tokens": 1024, + "training_eligibility_override": "operator-2026-09-08-rnd-run7", + "overridden_blockers": [ + "contamination-scan-not-implemented", + "stage-2-csam-detector-inert" + ], + "substitute_controls": [ + "pre-training holdout, run-1 (8,404 samples, work/card/session split)", + "pre-training holdout, govreport/holdout-v1 (416 reports, sha256-ranked, never_trained_on)", + "pre-training holdout, qmsum/holdout-v1 (5 transcripts, sha256-ranked, never_trained_on)", + "stage-A lexical quarantine, RP (829 records held unread)", + "stage-A lexical quarantine, run-5 slot (133 records held unread, /mnt/smithy/datasets/quarantine/r47-run5-longdep-screen/)", + "SCROLLS-membership disclosure on both slot sources (avoidance, NOT a scan): govreport + qmsum are SCROLLS/ZeroSCROLLS members, in no hoard/default-benchmarks.yaml entry and used by no R47 instrument", + "kvasir is HELD, not re-cut: the 1,613 kvasir survivors are reused verbatim from survivors-r4.jsonl (which cut run-3's seed-20260824 prefix at 3,347,622 ctx). survivors-r5.jsonl = survivors-r4 minus airoboros plus the govreport + qmsum roots whole; sha256 a25169a6258cd4abb0cb494a176a921c0e98eb73d65c53d033b6ee18293a43ae.", + "window_count belt-and-suspenders (SFT-RECIPE-run5-SCOPE.md 7.1): every govreport + qmsum row renders <= 14,000 tokens (max 9,385 / 13,700) and the harness never packs across samples, so window_count MUST be 1 on every slot row; a chunked_into_2 or single_window_truncated on either new root in truncation-report.json is a BUILD DEFECT and the run is killed before training.", + "SINGLE VARIABLE vs run 6: an OPENING-SPLIT slot ADDED with its companion loss mask (recipe-erp-seat-sft-r7, brokkr-smithy). 293 rows (bluemoon 109 / cwm 179 / c2-logs 5): the corpus's own first exchanges re-seated into the serving shape -- scene in the card, one line from the user, the >=250-word first bot turn as the only loss turn. Mask = lossmask-r3 UNION mask-original-first-turn (224 entries: the split conversations' ORIGINAL first bot turns -- loss MOVED to the turn-1 position, not doubled). Chosen by the cue-length probe on the run-6 tuned seat (5-word cue -> 54/62 words; 221-word -> 292). Base, seven held roots + shas, kvasir cut, longdep slot, lr 2e-04, max_seq_len 16384, rank/alpha/dropout, cosine/warmup/wd, batch 2 x accum 8, seed, stock tokenizer + template ae53464b ALL UNCHANGED from run 6.", + "SURVIVORS: survivors-r7.jsonl = survivors-r5.jsonl (8,212, held byte-identical) + the 293 opening-split clean-v1 row ids = 8,505.", + "FREE CHECK (brokkr, SFT-RECIPE-run7.md section 5): held targets must reproduce run 6's 8,370 records exactly (minus loss on the masked turns); the slot adds 293 records / ctx 525,002 / loss 233,589; window_count MUST be 1 on every slot row (max row 4,539 tokens) -- any chunked/truncated slot row is a BUILD DEFECT, kill before [train]. Report the mask-union match count and the realized [mix] block.", + "pre-training holdout, opening-split/holdout-v1 (16 rows: bluemoon 6 / cwm 9 / c2-logs 1, sha256-ranked 5%, never_trained_on; staged at datasets/derived/opening-split/holdout-v1/)", + "CSAM LINE: run 6's level=review SOFT trip (one soft term, minor_hard=0) was adjudicated GO by the operator in-session 2026-09-08 22:45 PT -- verbatim: in the vernacular, baby is about the most common pet name you can get, ESPECIALLY during sex. I am going to adjudicate it as a go. There are unmistakable CSAM terms, but baby is not one of them. -- relayed to brokkr-smithy-dev; run06-gate CLOSED as TRANSFERRED. Run 7 = this recipe on that base under that adjudication.", + "HOST: pfi-gx10 (GB10, aarch64, sm_121, 121 GB unified). Base unchanged from run 6 (sha256-verified); opening-split root + mask + holdout sha256-verified after the NAS -> box relay (85a75718 / dda151fa / 7224e049); harness eitri-smithy 0a6bd2e; corpus COPIED, box mounts no NFS. Grant: operator directive to infra-ops in-session 2026-09-08 23:03 PT (/goal: launch training run 7 on gx10)." + ], + "unfittable": "drop", + "holdout_dir": "/home/infra-ops/erp-tune/datasets/holdout", + "save_steps": 50 +} diff --git a/stacks/erp-seat/.env.example b/stacks/erp-seat/.env.example index 01ea98e..3f463aa 100644 --- a/stacks/erp-seat/.env.example +++ b/stacks/erp-seat/.env.example @@ -1,5 +1,5 @@ # erp-seat — ana-ml2 GPU1. Real .env lives on the host at /opt/docker/compose/erp-seat/.env. -ERP_IMAGE=vllm/vllm-openai:v0.26.0 +ERP_IMAGE=vllm/vllm-openai:nightly-311b3513af33bc29b4acb2fde2e9313e5e9966a0 ERP_MODEL=/tank/aimodels/erp-tune-v6-nvfp4a16 ERP_SERVED_NAME=erp-tune-v6-nvfp4a16 ERP_CHAT_TEMPLATE=/tank/aimodels/erp-tune-v6-nvfp4a16/chat_template.jinja diff --git a/stacks/erp-seat/README.md b/stacks/erp-seat/README.md index 8e6ddd7..469f2f7 100644 --- a/stacks/erp-seat/README.md +++ b/stacks/erp-seat/README.md @@ -19,6 +19,18 @@ checkpoint so the GX10 is free to train the next run. First occupant: **run 6** the model answers in prose (3/3). The rest of the matrix (auto / required / named / parallel / nested schema / empty `tools: []` / streaming / tool-result round trip) was green before and after. `stacks/gemma4-charrp` has the same exposure and does NOT carry the flag yet. +- **Forced tool_choice (named / `required`) is prompt-driven on EVERY Gemma-4 seat, not + grammar-enforced — by vLLM design.** `vllm/tool_parsers/gemma4_engine_tool_parser.py` sets + `supports_required_and_named = False` and its `adjust_request` deliberately skips the + structured-output JSON for required/named so the model can emit its native + `<|tool_call>call:…` syntax. A tune that weakened that syntax (this ERP tune) therefore + honours forced calls only sometimes. Measured 2026-09-08, 3 conversations × 3 turns, real + system prompt: v0.26.0 **1/9**; nightly `311b3513` (v0.27.2rc1, the `gen` seat's image) **6/9** + and the tool-result round trip stays clean 3/3 — so the seat runs the nightly. tts-dev measured + 0/18 on v0.26.0 with `gen` 18/18 as the positive control, and **`response_format: + json_schema` (guided decoding) 18/18 on this seat** — that is the deterministic path for a + forced call; `tool_choice: auto` works normally. A parser plugin that re-enables guided JSON + would also need JSON extraction in the engine-parser path; not attempted. - **GPU1 is shared** — check real usage (`nvidia-smi --query-compute-apps=pid,used_memory`) before raising `ERP_GPU_MEM_UTIL`; the flag sizes KV, not CUDA context. - **Rollback / next run:** point `ERP_MODEL` + `ERP_SERVED_NAME` at the next quant dir, keep the diff --git a/stacks/erp-seat/compose.yaml b/stacks/erp-seat/compose.yaml index 0f2fb40..3cb3b67 100644 --- a/stacks/erp-seat/compose.yaml +++ b/stacks/erp-seat/compose.yaml @@ -12,7 +12,7 @@ name: erp-seat services: vllm-erp-seat: - image: ${ERP_IMAGE:-vllm/vllm-openai:v0.26.0} + image: ${ERP_IMAGE:-vllm/vllm-openai:nightly-311b3513af33bc29b4acb2fde2e9313e5e9966a0} container_name: ${ERP_CONTAINER:-vllm-erp-seat} restart: unless-stopped ipc: host @@ -31,6 +31,12 @@ services: - --tool-call-parser - gemma4 - --enable-auto-tool-choice + # --reasoning-parser gemma4 + the enable_thinking pin are BOTH required (measured 2026-09-08): + # without them the post-tool-response turn leaks a literal "<|channel>thought\n" + # prefix into content (3/3), exactly as stacks/gemma4-charrp/README.md warns. They do NOT + # fix forced (named) tool_choice — that failure is independent of the reasoning parser + # (measured 1/9 with it, 3/9 without): the model free-writes Gemma native call syntax + # instead of grammar-constrained JSON. See README for the forced-tool_choice status. - --reasoning-parser - gemma4 - --default-chat-template-kwargs