memory: run 6 COMPLETE (train_loss 3.259), base seat erp-seat-base-ara serving for floors, processor_config.json gap on the abliterated repo
This commit is contained in:
+15
-13
@@ -144,19 +144,20 @@ COMPLETE and gated RESCUED (02:13 PDT). Live open items:_
|
||||
pulled was stock google, and the `-heretic` name rode along from intent. **Operator/Brokkr decision
|
||||
now evidenced (not a label guess): accept RESCUED-on-stock, or swap to a real abliteration (needs a
|
||||
bf16 source, not the GGUF) + re-run. Recipes should drop `-heretic` from the base name.**
|
||||
- **✅ RESOLVED → 🔥 RUN 6 TRAINING on pfi-gx10 (launched 2026-09-08T11:17:43Z, pid 4100375, `~/erp-tune/run-06.pid`,
|
||||
524 steps @ ~40–50 s/it → adapter ~18:30–19:00Z).** Operator picked the abliteration: jenerallee78's ARA
|
||||
abliteration of Gemma-4-26B-A4B-it @ `0631379a`, pulled to
|
||||
`/home/infra-ops/models/gemma4-26b-a4b-it-ara-abliterated-jenerallee78-0631379a` (named for the bytes), 32/32
|
||||
shards sha256+size == brokkr's pins, **base identity = index sha `33c59654…`**. Run-5 recipe byte-held (recipe-r6 =
|
||||
r5 targets, survivors-r5 verbatim); the free check passed EXACTLY (encode 8,370 rec / ctx 18,598,779 / loss
|
||||
9,935,076). Grant `operator-2026-09-08-rnd-run6` = the operator's direct in-session words ("unload the gx10 and
|
||||
commence training on the gx10. window is open now."). ⚠ The repo's `tokenizer.json` bakes in a 256-token
|
||||
truncation — STOCK tokenizer set installed over it, repo originals kept as `*.repo`. ⚠ `hf download` silently
|
||||
ignores `--include` when given multiple patterns after one flag (one `--include` per pattern). **erp-tune-v5
|
||||
seat STOPPED (GPU cleared for training) → LiteLLM `trial` alias is DARK until the next serve.** Gate choreography
|
||||
when the adapter lands (brokkr, cc channel, hands-off): serve base as **`erp-seat-base-ara`** (NOT
|
||||
`erp-seat-base` = stock), then merged arm as **`erp-tune-v6`**, run-5 flags. → `docs/runbooks/gx10-run-06.md`.
|
||||
- **✅ RUN 6 COMPLETE on pfi-gx10 (2026-09-08 16:06 PT, 524/524, 11 h 42 min) → 🔥 GATE IN PROGRESS: base seat
|
||||
`erp-seat-base-ara` SERVING on gx10:8098 (pid `vllm-base-run06gate.pid`, since 16:24 PT) for brokkr's floors;
|
||||
AWAITING HIS SWAP CUE → then stop it and serve `serve/merged-run06` as `erp-tune-v6` (same port/flags).**
|
||||
train_loss **3.259** (run 5: 3.235 — preregistered read wanted below; +0.024 is inside the ±0.1 per-step band).
|
||||
Adapter `run-06/adapter` (410 tensors); merged-run06 = adapter + jenerallee78 base (index 33c59654), stock
|
||||
template ae53464b. ⚠ **The abliterated repo ships NO `processor_config.json`** — vLLM's Gemma4 loader dies with
|
||||
"Can't load feature extractor" without it (first base launch failed exactly so). STOCK's copy (sha `32bdf45d…`)
|
||||
carried into both the base dir and merged-run06; processor plumbing, not weights. Base: jenerallee78 ARA
|
||||
abliteration @ `0631379a`, pulled to `/home/infra-ops/models/gemma4-26b-a4b-it-ara-abliterated-jenerallee78-0631379a`,
|
||||
32/32 shards verified vs brokkr's pins. Run-5 recipe byte-held (free check exact). Grant
|
||||
`operator-2026-09-08-rnd-run6`. ⚠ Repo `tokenizer.json` bakes a 256-token truncation → stock tokenizer set
|
||||
installed over it (`*.repo` kept). ⚠ `hf download` ignores `--include` with several patterns after one flag.
|
||||
**LiteLLM `trial` alias DARK until the gate ends** (then repoint 5→6 only on the operator's word).
|
||||
Miranda informed 16:07 PT. → `docs/runbooks/gx10-run-06.md`.
|
||||
- **📮 althing reachability on THIS bg seat = the cc-channel route, NOT the waiter.** `althing-listen`
|
||||
reaps within ~seconds-to-minutes on an idle background seat (harness kills detached bg tasks; hit it
|
||||
repeatedly this session). Fix (operator-directed 2026-09-08): `althing-route declare --handle infra-ops
|
||||
@@ -598,6 +599,7 @@ below is a live commitment or a known-open risk._
|
||||
- `[2026-09-08]` **R47 base settled from bytes = STOCK `google/gemma-4-26B-A4B-it`** — three-way sha match (local == HF etag == stock LFS oid; commit `4d7ae498` == stock HEAD); the `-heretic` label is a naming error, all runs trained from stock. Accept-vs-swap now evidenced. → `persistent-memory.d/2026-09-08-base-provenance-stock.md`
|
||||
- `[2026-09-08]` **yt-voice-clipper back UP** — dead since the 09-06 danted retirement (every job failed at yt-dlp, bot-gated on the Irvine datacenter IP). Fix: danted on **nh3-scale** (CT107) at `socks5h://100.64.0.1:1080`, fleet-ACL'd, residential egress 70.230.226.88 measured; `YTVC_PROXY` repointed, worker recreated, end-to-end job DONE with positive (proxied) + negative (direct = bot-gate) controls. Homepage card href/siteMonitor → `irv-ml1.nh3.internal:8000` (was dead wg0 IP). Then a SECOND fault: full downloads 403'd through the proxy (cookies irrelevant) = stale yt-dlp 2026.07.04 from a cached Dockerfile layer → `compose build --no-cache api` (2026.08.19), which dragged in a whisperx/nltk that needs `punkt_tab` → staged on the data volume + `NLTK_DATA` in the override. Operator's video x7kWJojf1MI → done, 8 clips. yt-voice-clipper-dev shipped both Dockerfile fixes + **CrisperWhisper 2.0 (v0.3.6, `b62849d`) — deployed and verified (12 clips, [UM]/[UH] tags)**. ⚠ The gitea push webhook had been targeting the dead wg0 IP since 09-06 (never fired) → repointed to `10.6.110.50:9008` with the HMAC secret re-applied; deploy script passes `YTDLP_REFRESH`. Script `scripts/setup-nh3-scale-socks-egress.sh`. → auto-memory `reference_nh3_egress_proxy`, `reference_ytvc_autodeploy`.
|
||||
- `[2026-09-08]` **ESH WAN static `128.177.138.182/30` (gw .181) is LIVE** — the Cityside /30 that was 'not provisioned' on 09-04 now carries traffic; egress verified from esh-docker-vm. CGNAT at ESH is over. Added to the crowdsec `esh` allowlist. All three follow-ups LANDED same day: FortiGate trusthost3 → the static (login from ESH verified), dormant esh-ana IPsec rebound to wan1/static, UDP 41641 forward → esh-scale now peers DIRECT (was DERP).
|
||||
- `[2026-09-08]` **ERP run 6 COMPLETE** — 524/524, train_loss 3.259 (run 5: 3.235). Merged; base seat `erp-seat-base-ara` serving on gx10:8098 for floors, awaiting brokkr's swap cue → `erp-tune-v6`. ⚠ abliterated repo lacks `processor_config.json` — stock's carried in (32bdf45d). Miranda informed.
|
||||
- `[2026-09-08]` **ERP run 6 LAUNCHED on pfi-gx10 on the jenerallee78 ARA-abliterated base** (index `33c59654…`, 32/32 shards byte-verified vs brokkr pins, stock tokenizer set installed over the repo's 256-token-truncating one, run-5 recipe byte-held, free check exact). Operator's direct grant `operator-2026-09-08-rnd-run6`; run-5 seat unloaded (`trial` dark). Gate names: `erp-seat-base-ara` / `erp-tune-v6`. → `docs/runbooks/gx10-run-06.md`, commit `3fec668`.
|
||||
- `[2026-09-08]` **Miranda = operator's chief of staff, may relay his directives** — added to user-level `~/.claude/CLAUDE.md` (dotfiles `7134a22`) as the named exception to the no-relayed-auth rule (unidentified peer relays still excluded); material-consequence calls she relays stay the operator's own.
|
||||
- `[2026-09-08]` **Fleet fixes shipped** — WhereTF Homepage card + DNS (`4506ef6`); ext-tts LiteLLM alias → `irv-ml1.nh3.internal` (DB `/model/update` + `extra_hosts`, `957c8f1`); the 09-06 irv-ml1 stale-IP trail repointed across 25 composes + services.yaml + ssh-target → DNS name (`e0d1c44`); Homepage `/api/services` outage fixed — ana-ml2 discovery via a socat proxy on ana-docker (`stacks/ana-ml2-proxy`, `913d2d2`, reversible).
|
||||
|
||||
Reference in New Issue
Block a user