From a19b7f62d943c1e4f2131fadc8bf64beab762a32 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 8 Sep 2026 13:45:13 -0700 Subject: [PATCH] =?UTF-8?q?memory:=20ESH=20static=20WAN=20follow-ups=20lan?= =?UTF-8?q?ded=20=E2=80=94=20FortiGate=20trusthost3=20=E2=86=92=20128.177.?= =?UTF-8?q?138.182=20(verified=20from=20ESH),=20esh-ana=20IPsec=20rebound?= =?UTF-8?q?=20to=20wan1,=20UDP=2041641=20forward=20=E2=86=92=20esh-scale?= =?UTF-8?q?=20peers=20direct?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-09-06-headscale-cutover.md | 2 +- persistent-memory.md | 16 ++++++++++------ 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/persistent-memory.d/2026-09-06-headscale-cutover.md b/persistent-memory.d/2026-09-06-headscale-cutover.md index bc1fc0d..02aa8f2 100644 --- a/persistent-memory.d/2026-09-06-headscale-cutover.md +++ b/persistent-memory.d/2026-09-06-headscale-cutover.md @@ -14,7 +14,7 @@ follow-ups in `docs/pfi/headscale-mesh-plan.md` § CUTOVER EXECUTED. Headlines: orchestrated). Routes PRE-STAGED + shadowed; DERP path 9ms ready. **Operator disables it in the UniFi UI**, then the mesh takes over. Told the operator "mesh is online" → he does it. - **FortiGate WAN SSH safety net (TEMPORARY):** wan1 allowaccess ping+ssh; admin infra-ops - trusthost2/3 = NH3 70.230.226.88 + ESH 23.164.40.160 (not 0.0.0.0). Reach it at + trusthost2/3 = NH3 70.230.226.88 + ESH **128.177.138.182** (static since 09-08; was CGNAT 23.164.40.160) (not 0.0.0.0). Reach it at `ssh infra-ops@38.120.12.42`. Config backed up flash `pre-wan-ssh-cutover-20260906`. Remove when the edge (being replaced by OPNsense/R420) is retired. - ⚠ **Method lesson:** tunnel + mesh static route for the same /16 on one gateway = asymmetric diff --git a/persistent-memory.md b/persistent-memory.md index c66f217..8732607 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -261,11 +261,15 @@ below is a live commitment or a known-open risk._ ESH is HISTORY; `100.104.0.1` still shows as the ISP's first hop, that is their access network, not NAT). IPv6 unchanged (`2607:73c0:402:1d00::/56`, hosts still egress as their own v6). Done 2026-09-08: `128.177.138.182` added to the crowdsec `esh` allowlist on ana-docker (the false-ban class is closed - for ESH). Still stale and OPEN: (a) FortiGate infra-ops `trusthost3` = old CGNAT `23.164.40.160` → - should be `128.177.138.182` or the WAN-SSH safety net does not work from ESH; (b) dormant `esh-ana` - IPsec object is bound to **wan2** (`192.168.200.111`) — rebind to wan1/`128.177.138.182` if it is ever - to serve as mesh failover; (c) esh-scale still reaches peers via DERP(lax) — a UDP/41641 port-forward - on the ESH UDM would let the mesh go direct now that the IP is static. `wan1-REVERT.json` is obsolete. + for ESH). **All three follow-ups LANDED 2026-09-08 ~20:45Z (operator: "land all 3"):** (a) FortiGate + infra-ops `trusthost3` 23.164.40.160 → `128.177.138.182/32` — VERIFIED by a real infra-ops login to + `38.120.12.42` from esh-docker-vm (`ana-gw #`); `execute backup config flash pre-trusthost3-esh-static-20260908` + ran ("Please wait...") but `execute revision list` errors on this box, so the backup is unconfirmed — + the before-state was a single line, recorded here. (b) dormant `esh-ana` IPsec rebound wan2/192.168.200.111 + → **wan1/`128.177.138.182`**, still `enabled=false`. (c) ESH UDM port-forward `esh-scale tailscale direct + (UDP 41641)` → 10.0.50.65:41641 (id `6aa0727a…`) — VERIFIED: esh-scale now peers **direct via + `128.177.138.182:41641`** (was DERP lax). Helpers: scratchpad `esh-udm-land.py` + `fg-trusthost3.sh`. + `wan1-REVERT.json` is obsolete. - ⚠ **esh-nas is effectively open to the whole ESH LAN** — twelve NFS exports rw to `10.0.0.0/8` with `sec=sys`, and every SMB share but `backup` guest-writable. @@ -592,7 +596,7 @@ below is a live commitment or a known-open risk._ - `[2026-09-08]` **ERP run 5 = RESCUED (landmark R49.5)** — first capability-gate pass in the ERP-seat line; the 3.46%-loss dependency-forcing slot (GovReport+QMSum) broke the coupling runs 3c/4 couldn't. Seat `erp-tune-v5` served on gx10:8098, `trial` alias repointed 3c→v5. → `persistent-memory.d/2026-09-08-run5-rescued.md` - `[2026-09-08]` **R47 base settled from bytes = STOCK `google/gemma-4-26B-A4B-it`** — three-way sha match (local == HF etag == stock LFS oid; commit `4d7ae498` == stock HEAD); the `-heretic` label is a naming error, all runs trained from stock. Accept-vs-swap now evidenced. → `persistent-memory.d/2026-09-08-base-provenance-stock.md` -- `[2026-09-08]` **ESH WAN static `128.177.138.182/30` (gw .181) is LIVE** — the Cityside /30 that was 'not provisioned' on 09-04 now carries traffic; egress verified from esh-docker-vm. CGNAT at ESH is over. Added to the crowdsec `esh` allowlist. Open follow-ups: FortiGate trusthost3, dormant esh-ana IPsec rebind wan2→wan1, tailscale direct-path port-forward. +- `[2026-09-08]` **ESH WAN static `128.177.138.182/30` (gw .181) is LIVE** — the Cityside /30 that was 'not provisioned' on 09-04 now carries traffic; egress verified from esh-docker-vm. CGNAT at ESH is over. Added to the crowdsec `esh` allowlist. All three follow-ups LANDED same day: FortiGate trusthost3 → the static (login from ESH verified), dormant esh-ana IPsec rebound to wan1/static, UDP 41641 forward → esh-scale now peers DIRECT (was DERP). - `[2026-09-08]` **ERP run 6 LAUNCHED on pfi-gx10 on the jenerallee78 ARA-abliterated base** (index `33c59654…`, 32/32 shards byte-verified vs brokkr pins, stock tokenizer set installed over the repo's 256-token-truncating one, run-5 recipe byte-held, free check exact). Operator's direct grant `operator-2026-09-08-rnd-run6`; run-5 seat unloaded (`trial` dark). Gate names: `erp-seat-base-ara` / `erp-tune-v6`. → `docs/runbooks/gx10-run-06.md`, commit `3fec668`. - `[2026-09-08]` **Miranda = operator's chief of staff, may relay his directives** — added to user-level `~/.claude/CLAUDE.md` (dotfiles `7134a22`) as the named exception to the no-relayed-auth rule (unidentified peer relays still excluded); material-consequence calls she relays stay the operator's own. - `[2026-09-08]` **Fleet fixes shipped** — WhereTF Homepage card + DNS (`4506ef6`); ext-tts LiteLLM alias → `irv-ml1.nh3.internal` (DB `/model/update` + `extra_hosts`, `957c8f1`); the 09-06 irv-ml1 stale-IP trail repointed across 25 composes + services.yaml + ssh-target → DNS name (`e0d1c44`); Homepage `/api/services` outage fixed — ana-ml2 discovery via a socat proxy on ana-docker (`stacks/ana-ml2-proxy`, `913d2d2`, reversible).