memory: snapshot — irv-ml1 remote cutover complete + subnet discovery; session follow-ups

This commit is contained in:
2026-09-06 01:05:14 -07:00
parent 02cf9d6cfb
commit ed3810aea2
+18 -9
View File
@@ -1,6 +1,6 @@
# Persistent memory — eshpfi-management
_Last updated: 2026-09-06 (infra: NASPool rebuild + headscale cutover)_
_Last updated: 2026-09-06 (infra: NASPool rebuild + headscale cutover incl. irv-ml1)_
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
> `Written:` stamp is under an hour old, read it (it carries the in-flight
@@ -108,9 +108,8 @@ no longer deployed sidecars here. See Recent decisions.)
(no NOPASSWD)** — stage model pulls to `/home`, not root-owned `/worktank`.
## Current state / in-flight
_Infra session 2026-09-06 (NASPool + headscale) — open follow-ups from this session,
none blocking; the ERP / althing / fiber items further down belong to other streams and
were not touched:_
_Infra session 2026-09-06 (NASPool rebuild + full headscale cutover incl. irv-ml1) — open
follow-ups; the ERP / althing / fiber items further down belong to other streams, untouched:_
- **NASPool parked copy still on ospool** — `ospool/naspool-evac` (1.65T) + `NASPool/*@evac`
snapshots. Destroy ONLY after the new raidz2 scrub is clean (it is, 0 errors 04:43Z) AND
@@ -120,11 +119,21 @@ were not touched:_
- **FortiGate WAN SSH is temporarily open** (`wan1` allowaccess ping+ssh; admin `infra-ops`
trusthost2/3 = 70.230.226.88 NH3 + 23.164.40.160 ESH). Safety net for the cutover — CLOSE it
when the edge is retired (OPNsense/R420). `ssh infra-ops@38.120.12.42`.
- **SOCKS proxy NOT retired** — operator asked, but yt-voice-clipper on irv-ml1 still uses
`YTVC_PROXY=socks5h://10.100.10.50:1080` (live, up 12d) and irv-ml1 isn't a mesh node. The
exit node is whole-host; the dante proxy does scoped per-yt-dlp egress → the two aren't
interchangeable. Keep the proxy, or first migrate irv-ml1 onto the mesh (tailscale's own
`--socks5-server` could replace it). Surfaced; operator to decide.
- **irv-ml1 FOLDED INTO THE MESH + cut over (done remotely, operator has NO Irvine access for
~5 days from 2026-09-06).** Node 100.64.0.6; wg0 DOWN and `wg-quick@wg0` DISABLED (not
reboot-restorable); full subnet router (accept-routes + advertises 10.6.110.0/24, gateway
routes added, fleet↔Irvine verified). Failover for the 5-day window = `wg0-watchdog.service`
(wg-quick up wg0 on ~5min mesh loss) + independent reverse SSH tunnel (`revtun-nh3.service`
→ nh3-dev via UDM fwd tcp/47822 src-restricted; reach it `ssh -i ~/.ssh/infra-ops_ed25519
-p 2201 infra-ops@127.0.0.1` on nh3-dev). Detail: docs/pfi/headscale-mesh-plan.md.
- **dante SOCKS proxy RETIRED** on nh3-dev (danted disabled, :1080 closed, config `.retired`).
**yt-voice-clipper is DOWN** until its SCOPED exit-node egress is wired (operator-accepted).
Follow-up: wire YTVC egress via tailscale `--socks5-server`+nh3 exit node or a per-container
netns — **NEVER set irv-ml1 `--exit-node` globally** (routes the reverse tunnel through the
mesh → kills the independent lifeline). Then bring YTVC back.
- **On-site (Irvine, ~5 days): decide** whether to keep or remove the reverse tunnel +
UDM forward `irv-revtun-ssh` + the revtun authorized_key on nh3-dev (small src-restricted WAN
surface), and whether to fully delete the wg0 config.
- **infra-ops now on all four PVE hypervisors** (pfi-pve/nh3-pve/esh-pve/esh-pve-nas) — PVE
ships without sudo, `apt install sudo` first or elway hangs on a password prompt.