feat(gx10): rack networking — VLAN 50 via DHCP reservation, wired only

pfi-gx10 moved from desk Wi-Fi (10.100.10.226, VLAN 10) to the rack on
10.100.50.60 (nh3-servers, VLAN 50), reachable as pfi-gx10.nh3.internal.

The address is a DHCP RESERVATION on the UDM, not a host static. Operator
ruling during the move, and the better design: a host-side static works
until the box moves and is then a stale netplan file on a machine whose
address you no longer know. The pre-written playbook wrote a host static;
it is kept for its safety ordering and annotated as retired.

The port arrived on the native VLAN, not the server VLAN, so switch port 22
was repointed first. port_overrides is a whole-array PUT — two unrelated
overrides on ports 21 and 23 were read, preserved and written back, with the
original array backed up to a file before the change.

Wi-Fi stayed up as the escape hatch until the wired path was proven from
outside, and was downed last as its own step. The step worth keeping: while
Wi-Fi was up, traffic to nh3-dev still preferred wlP9s9 because that
interface sits directly on the userland subnet, so reachability proved
nothing about the wired path. `ping -I enP7s7` across the VLAN boundary is
what actually settled it before the hatch came down.
This commit is contained in:
vh
2026-09-03 15:59:09 -07:00
parent 0f748ea54e
commit a95717e810
4 changed files with 87 additions and 1 deletions
+1
View File
@@ -75,6 +75,7 @@ hosts:
# ---- nh3: NH3 office ---- # ---- nh3: NH3 office ----
- {name: nh3-docker, site: nh3, v4: 10.100.50.40, note: general-purpose docker host + AdGuard} - {name: nh3-docker, site: nh3, v4: 10.100.50.40, note: general-purpose docker host + AdGuard}
- {name: pfi-gx10, site: nh3, v4: 10.100.50.60, note: GB10 Grace Blackwell, 121 GB unified — DHCP RESERVATION on the UDM, not a host static, so the box stays portable}
- {name: nh3-dev, site: nh3, v4: 10.100.10.50, note: dev box, fleet sidecars, Claude sessions} - {name: nh3-dev, site: nh3, v4: 10.100.10.50, note: dev box, fleet sidecars, Claude sessions}
- {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box} - {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box}
- {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+} - {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+}
+65
View File
@@ -0,0 +1,65 @@
# pfi-gx10 — desk to rack, network
Done 2026-09-03. `pfi-gx10.nh3.internal` → **10.100.50.60**, wired only.
## The shape, and why it is this shape
**The address lives on the switch/firewall side, not on the host.** Operator
ruling at the time of the move: a DHCP **reservation** on the UDM, with the host
left on DHCP. A host-side static works until the box moves, and then it is a
stale netplan file on a machine whose address you no longer know. A reservation
moves with the MAC.
UniFi switch port 22 native network -> nh3-servers (VLAN 50)
UniFi client reservation -> 30:c5:99:3d:a7:45 = 10.100.50.60
host unchanged: still DHCP
`playbooks/gx10-rack-network.yaml` is kept for its safety ordering but its
host-static approach was **not** used, and it says so at the top.
## ⚠ Order is the whole safety argument
The lockout risk in remote network work is applying a config that does not work
and losing the only path back. There was never one path:
1. **Verify the cable is really in** — `/sys/class/net/enP7s7/carrier` = 1.
Configuring a dead interface and calling it done is a green that costs a rack
visit to discover.
2. **Identify by MAC, not by interface name.** Names renumber across kernels;
`30:c5:99:3d:a7:45` does not.
3. **Move the switch port, then reserve, then renew** — Wi-Fi stays up
throughout as the escape hatch.
4. **Prove the new address from OUTSIDE** before removing anything.
5. ⚠ **Prove inter-VLAN routing before killing Wi-Fi.** This is the step that
is easy to skip and expensive to miss: while Wi-Fi was up, replies to
nh3-dev (10.100.10.50) still preferred `wlP9s9`, because that interface sits
directly on the userland subnet. So "I can reach it" proved nothing about
whether the WIRED path could. Forcing the interface first is what settles it:
ping -c3 -I enP7s7 10.100.10.50 # VLAN 50 -> VLAN 10, 0% loss
ping -c2 -I enP7s7 1.1.1.1 # egress
6. **Down Wi-Fi last, as its own change** — a server with two default routes
picks one and you find out later, in a way that looks like packet loss.
## Editing UniFi port overrides
`port_overrides` is a **whole-array PUT**. Read the existing array, filter out
the port you are changing, append your entry, and write the lot back — anything
you omit is deleted. Two unrelated overrides (ports 21 and 23) were on this
switch and were preserved deliberately; the pre-change array is worth backing
up to a file first.
GET /proxy/network/api/s/default/stat/device # find switch by MAC
PUT /proxy/network/api/s/default/rest/device/<id> # {"port_overrides":[...]}
PUT /proxy/network/api/s/default/rest/user/<id> # the reservation
Reservations live on the **client** object (`use_fixedip`, `fixed_ip`,
`network_id`), not on the network.
## Wi-Fi
`nmcli radio wifi off` — persists across reboot in
`/var/lib/NetworkManager/NetworkManager.state`, verified by reading it back
rather than assuming. `/etc/netplan` was backed up to
`/etc/netplan.bak-preWifiDown` first.
+20
View File
@@ -1,3 +1,23 @@
# pfi-gx10 — RETIRED 2026-09-03. Kept for its reasoning, not for running.
#
# ⚠ THE STATIC-IP APPROACH IN THIS FILE WAS NOT USED. Operator ruling on the
# day of the rack move: put the address on the SWITCH/FIREWALL side as a DHCP
# RESERVATION and leave the host on DHCP, so moving the box later does not mean
# unpicking a stale netplan static on a machine you may not be able to reach.
# The host-static this playbook writes is exactly the "interesting adventure"
# that ruling avoids.
#
# What was actually done instead (see docs/runbooks/gx10-rack-network.md):
# 1. UniFi switch port 22 native network -> nh3-servers (VLAN 50)
# 2. UniFi client reservation: 30:c5:99:3d:a7:45 -> 10.100.50.60
# 3. host DHCP renew; Wi-Fi left UP until the wired path was proven
# 4. dns/internal.yaml + dns-sync
# 5. nmcli radio wifi off, last, as its own step
#
# The safety ORDER below is still right and was followed: verify carrier,
# verify MAC not interface name, keep Wi-Fi up as the escape hatch, prove the
# new address from OUTSIDE, tear the hatch down separately and last.
#
# pfi-gx10 — move from desk (Wi-Fi + DHCP, VLAN 10) to rack (wired static, # pfi-gx10 — move from desk (Wi-Fi + DHCP, VLAN 10) to rack (wired static,
# VLAN 50 nh3-servers). Run AFTER the box is racked and the switch port has # VLAN 50 nh3-servers). Run AFTER the box is racked and the switch port has
# been placed on nh3-servers. # been placed on nh3-servers.
+1 -1
View File
@@ -1 +1 @@
infra-ops@10.100.10.226 infra-ops@10.100.50.60