From a95717e810b6c1ac4c5fddb417e5ef632fc492c5 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Thu, 3 Sep 2026 15:59:09 -0700 Subject: [PATCH] =?UTF-8?q?feat(gx10):=20rack=20networking=20=E2=80=94=20V?= =?UTF-8?q?LAN=2050=20via=20DHCP=20reservation,=20wired=20only?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit pfi-gx10 moved from desk Wi-Fi (10.100.10.226, VLAN 10) to the rack on 10.100.50.60 (nh3-servers, VLAN 50), reachable as pfi-gx10.nh3.internal. The address is a DHCP RESERVATION on the UDM, not a host static. Operator ruling during the move, and the better design: a host-side static works until the box moves and is then a stale netplan file on a machine whose address you no longer know. The pre-written playbook wrote a host static; it is kept for its safety ordering and annotated as retired. The port arrived on the native VLAN, not the server VLAN, so switch port 22 was repointed first. port_overrides is a whole-array PUT — two unrelated overrides on ports 21 and 23 were read, preserved and written back, with the original array backed up to a file before the change. Wi-Fi stayed up as the escape hatch until the wired path was proven from outside, and was downed last as its own step. The step worth keeping: while Wi-Fi was up, traffic to nh3-dev still preferred wlP9s9 because that interface sits directly on the userland subnet, so reachability proved nothing about the wired path. `ping -I enP7s7` across the VLAN boundary is what actually settled it before the hatch came down. --- dns/internal.yaml | 1 + docs/runbooks/gx10-rack-network.md | 65 ++++++++++++++++++++++++++++++ playbooks/gx10-rack-network.yaml | 20 +++++++++ servers/pfi-gx10/ssh-target | 2 +- 4 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 docs/runbooks/gx10-rack-network.md diff --git a/dns/internal.yaml b/dns/internal.yaml index bf76194..906223e 100644 --- a/dns/internal.yaml +++ b/dns/internal.yaml @@ -75,6 +75,7 @@ hosts: # ---- nh3: NH3 office ---- - {name: nh3-docker, site: nh3, v4: 10.100.50.40, note: general-purpose docker host + AdGuard} + - {name: pfi-gx10, site: nh3, v4: 10.100.50.60, note: GB10 Grace Blackwell, 121 GB unified — DHCP RESERVATION on the UDM, not a host static, so the box stays portable} - {name: nh3-dev, site: nh3, v4: 10.100.10.50, note: dev box, fleet sidecars, Claude sessions} - {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box} - {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+} diff --git a/docs/runbooks/gx10-rack-network.md b/docs/runbooks/gx10-rack-network.md new file mode 100644 index 0000000..d34cae4 --- /dev/null +++ b/docs/runbooks/gx10-rack-network.md @@ -0,0 +1,65 @@ +# pfi-gx10 — desk to rack, network + +Done 2026-09-03. `pfi-gx10.nh3.internal` → **10.100.50.60**, wired only. + +## The shape, and why it is this shape + +**The address lives on the switch/firewall side, not on the host.** Operator +ruling at the time of the move: a DHCP **reservation** on the UDM, with the host +left on DHCP. A host-side static works until the box moves, and then it is a +stale netplan file on a machine whose address you no longer know. A reservation +moves with the MAC. + + UniFi switch port 22 native network -> nh3-servers (VLAN 50) + UniFi client reservation -> 30:c5:99:3d:a7:45 = 10.100.50.60 + host unchanged: still DHCP + +`playbooks/gx10-rack-network.yaml` is kept for its safety ordering but its +host-static approach was **not** used, and it says so at the top. + +## ⚠ Order is the whole safety argument + +The lockout risk in remote network work is applying a config that does not work +and losing the only path back. There was never one path: + +1. **Verify the cable is really in** — `/sys/class/net/enP7s7/carrier` = 1. + Configuring a dead interface and calling it done is a green that costs a rack + visit to discover. +2. **Identify by MAC, not by interface name.** Names renumber across kernels; + `30:c5:99:3d:a7:45` does not. +3. **Move the switch port, then reserve, then renew** — Wi-Fi stays up + throughout as the escape hatch. +4. **Prove the new address from OUTSIDE** before removing anything. +5. ⚠ **Prove inter-VLAN routing before killing Wi-Fi.** This is the step that + is easy to skip and expensive to miss: while Wi-Fi was up, replies to + nh3-dev (10.100.10.50) still preferred `wlP9s9`, because that interface sits + directly on the userland subnet. So "I can reach it" proved nothing about + whether the WIRED path could. Forcing the interface first is what settles it: + + ping -c3 -I enP7s7 10.100.10.50 # VLAN 50 -> VLAN 10, 0% loss + ping -c2 -I enP7s7 1.1.1.1 # egress + +6. **Down Wi-Fi last, as its own change** — a server with two default routes + picks one and you find out later, in a way that looks like packet loss. + +## Editing UniFi port overrides + +`port_overrides` is a **whole-array PUT**. Read the existing array, filter out +the port you are changing, append your entry, and write the lot back — anything +you omit is deleted. Two unrelated overrides (ports 21 and 23) were on this +switch and were preserved deliberately; the pre-change array is worth backing +up to a file first. + + GET /proxy/network/api/s/default/stat/device # find switch by MAC + PUT /proxy/network/api/s/default/rest/device/ # {"port_overrides":[...]} + PUT /proxy/network/api/s/default/rest/user/ # the reservation + +Reservations live on the **client** object (`use_fixedip`, `fixed_ip`, +`network_id`), not on the network. + +## Wi-Fi + +`nmcli radio wifi off` — persists across reboot in +`/var/lib/NetworkManager/NetworkManager.state`, verified by reading it back +rather than assuming. `/etc/netplan` was backed up to +`/etc/netplan.bak-preWifiDown` first. diff --git a/playbooks/gx10-rack-network.yaml b/playbooks/gx10-rack-network.yaml index 3b05891..ee3c9ae 100644 --- a/playbooks/gx10-rack-network.yaml +++ b/playbooks/gx10-rack-network.yaml @@ -1,3 +1,23 @@ +# pfi-gx10 — RETIRED 2026-09-03. Kept for its reasoning, not for running. +# +# ⚠ THE STATIC-IP APPROACH IN THIS FILE WAS NOT USED. Operator ruling on the +# day of the rack move: put the address on the SWITCH/FIREWALL side as a DHCP +# RESERVATION and leave the host on DHCP, so moving the box later does not mean +# unpicking a stale netplan static on a machine you may not be able to reach. +# The host-static this playbook writes is exactly the "interesting adventure" +# that ruling avoids. +# +# What was actually done instead (see docs/runbooks/gx10-rack-network.md): +# 1. UniFi switch port 22 native network -> nh3-servers (VLAN 50) +# 2. UniFi client reservation: 30:c5:99:3d:a7:45 -> 10.100.50.60 +# 3. host DHCP renew; Wi-Fi left UP until the wired path was proven +# 4. dns/internal.yaml + dns-sync +# 5. nmcli radio wifi off, last, as its own step +# +# The safety ORDER below is still right and was followed: verify carrier, +# verify MAC not interface name, keep Wi-Fi up as the escape hatch, prove the +# new address from OUTSIDE, tear the hatch down separately and last. +# # pfi-gx10 — move from desk (Wi-Fi + DHCP, VLAN 10) to rack (wired static, # VLAN 50 nh3-servers). Run AFTER the box is racked and the switch port has # been placed on nh3-servers. diff --git a/servers/pfi-gx10/ssh-target b/servers/pfi-gx10/ssh-target index ccea938..735896b 100644 --- a/servers/pfi-gx10/ssh-target +++ b/servers/pfi-gx10/ssh-target @@ -1 +1 @@ -infra-ops@10.100.10.226 +infra-ops@10.100.50.60