34 lines
2.5 KiB
Markdown
34 lines
2.5 KiB
Markdown
# 2026-09-06 — Headscale cutover COMPLETE: all three site-pairs on the mesh
|
|
|
|
**DONE.** Operator disabled Site Magic in the UI; NH3↔ESH re-homed to a DIRECT mesh path (8ms, no DERP). Full 6-direction matrix OPEN. Site Magic disabled, both IPsec tunnels dormant, headscale is the sole active site-to-site transport. ana-wg WG fallback untouched. Tunnels re-enablable for backup.
|
|
|
|
Operator goal (/goal): replace Site Magic + IPsec with headscale, tunnels dormant as backup;
|
|
"if paranoid, enable world-accessible SSH on the FortiGate first." Full detail + method +
|
|
follow-ups in `docs/pfi/headscale-mesh-plan.md` § CUTOVER EXECUTED. Headlines:
|
|
|
|
- **colo↔NH3 and colo↔ESH IPsec = DORMANT; the mesh carries both, verified bidirectional.**
|
|
NH3 UDM `pfi-nh3-ana` + ESH UDM `esh-ana` set enabled=false (API). Mesh /16 routes added on
|
|
both UDMs and the FortiGate (→ ana-scale 10.250.50.45 / nh3-scale 10.100.50.46 / esh-scale
|
|
10.0.50.65). Dependent flows OK over mesh: restic ESH→rest-server-ana, FortiGate mgmt.
|
|
- **NH3↔ESH Site Magic NOT cut by API** — `sdwan-mesh-tunnel` = `api.err.NoEdit` (cloud
|
|
orchestrated). Routes PRE-STAGED + shadowed; DERP path 9ms ready. **Operator disables it in
|
|
the UniFi UI**, then the mesh takes over. Told the operator "mesh is online" → he does it.
|
|
- **FortiGate WAN SSH safety net (TEMPORARY):** wan1 allowaccess ping+ssh; admin infra-ops
|
|
trusthost2/3 = NH3 70.230.226.88 + ESH 23.164.40.160 (not 0.0.0.0). Reach it at
|
|
`ssh infra-ops@38.120.12.42`. Config backed up flash `pre-wan-ssh-cutover-20260906`. Remove
|
|
when the edge (being replaced by OPNsense/R420) is retired.
|
|
- ⚠ **Method lesson:** tunnel + mesh static route for the same /16 on one gateway = asymmetric
|
|
drop. Disable the tunnel FIRST, then add the route. Broke colo once doing it tunnel-up; rolled
|
|
back. See [[incident_crowdsec_cgnat_false_ban]] (same day) and the plan doc.
|
|
- Dormancy = disabled+retained (flip UDM object back to enabled=true to restore); NO auto
|
|
failover wired. Bonus: exit nodes → free multi-location egress proxy (parked).
|
|
|
|
|
|
## Exit nodes (2026-09-06, operator-requested)
|
|
All three routers advertise+serve exit nodes (approved). Clients pick location:
|
|
`tailscale set --exit-node=nh3-scale|esh-scale|ana-scale`. NH3 = residential egress
|
|
(70.230.226.88) → replaces the nh3-dev SOCKS5 proxy. ⚠ Enabling exit nodes REQUIRED flipping
|
|
`--snat-subnet-routes=true` (reversed the source-preserving NoSNAT); cross-site traffic now
|
|
shows the router IP as source. A node advertising an exit node can't consume one — test from
|
|
the laptop/iPad, not the routers.
|