From 2c248200b5f56288fc4ea4399ba190c535c94f21 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 25 Sep 2026 22:34:54 -0700 Subject: [PATCH] docs(nh3-pve): AMT cable is on UDM port 6, not nh3-sw1 (first pass read a stale port table) --- persistent-memory.md | 4 ++-- servers/nh3-pve/README.md | 23 +++++++++++++++-------- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/persistent-memory.md b/persistent-memory.md index 98752f7..59f3c6b 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -136,8 +136,8 @@ hermes-gateway, Booth, ttyd, zellij-web and dev-backup are running. UDM reservation). It had no IP because Linux had the port DOWN. `igc` powers off the PHY and AMT loses its link. Our own 09-25 removal of `enp88s0` from vmbr0 caused that. Fix: `auto enp88s0` (up, no IP, unbridged) plus IPv6 off via - sysctl.d, persistent. The cable is on an **nh3-sw1 trunk** (untagged - nh3-default), not the UDM. + sysctl.d, persistent. The cable is on **UDM port 6** (`forward: all`; + AMT is untagged, so nh3-default). **Open:** check that KVM + Opt-in None are set (needs the AMT admin password), then move the NanoKVM to the gx10. - Found and fixed: **lxc-pve 6.0.0-1 broke Docker in the CT** (runc 1.5 sysctl diff --git a/servers/nh3-pve/README.md b/servers/nh3-pve/README.md index 014910c..7eb3366 100644 --- a/servers/nh3-pve/README.md +++ b/servers/nh3-pve/README.md @@ -61,10 +61,15 @@ not power back on by itself. 1.2 with legacy renegotiation, so OpenSSL 3 clients need `Options = UnsafeLegacyRenegotiation`; browsers cope. 16992 (plain HTTP) is closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does not answer ping. - - **Port:** the I226-LM `enp88s0` (MAC `58:47:ca:76:96:0e`) is cabled to a - **trunk port on nh3-sw1**, not the UDM (seen from the host: STP from - `48:a9:8a:40:8a:51`, untagged nh3-default plus tagged VLANs 10/50/90/250). - AMT is untagged, so it lands on **nh3-default `10.100.0.0/24`**. + - **Port:** the I226-LM `enp88s0` (MAC `58:47:ca:76:96:0e`) is cabled to + **PFI-UDMSE port 6** (Prime; confirmed from the UDM, which lists the AMT MAC + on port 6 at 1 Gb). Port 6 is `forward: all`, meaning untagged nh3-default + plus every tagged VLAN, so AMT (untagged) lands on **nh3-default + `10.100.0.0/24`**. + ⚠ The first pass wrongly placed it on nh3-sw1, for two reasons. The UDM's port + table was read 15 s after link-up, before the UDM refreshed it (it polls, and + lags about a minute). And nh3-sw1's STP BPDUs, seen on the wire, are flooded + through the UDM's switch, so they do not locate the cable. - **Lease:** UDM DHCP reservation `nh3-pve-amt` → `10.100.0.151`. - ⚠ **Why it had no IP: Linux had the port DOWN.** When `igc` downs the I226-LM it powers off the PHY, and AMT's link goes with it (measured: "Link detected: @@ -81,8 +86,10 @@ not power back on by itself. - **Not yet verified:** KVM redirection is enabled with User Opt-in = None (this needs the AMT admin password), and AMT keeps its link with the host powered off (this depends on the MEBx power policy). - - Better home, not done: nh3-mgmt (VLAN 250), by setting that nh3-sw1 port's - native VLAN to 250. infra-ops has no nh3-sw1 access. + - Better home, not done (Prime's call): make UDM port 6 native nh3-mgmt + (VLAN 250) and drop the tagged VLANs, so AMT alone sits on the management + network. It would need a new reservation and a DNS change. The UDM API key + can do it. - MEBx menu: Ctrl+P at boot. - **Console OOB exists: a Sipeed NanoKVM** is attached (USB `3346:1009` on the host; web UI **`https://10.100.250.171`**, switch port 23, nh3-mgmt). It gives video and @@ -101,8 +108,8 @@ not power back on by itself. without network anyway: open the NanoKVM console, run `ip -br link`, and compare names against `/etc/network/interfaces` (backup in `/root/nic-pin-2026-09-25/`). - **`enp88s0` (the AMT port) is not a vmbr0 bridge port** (removed 2026-09-25). - STP is off, and that port is on the same nh3-sw1 trunk, so bridging it would - loop the site LAN. It is held admin-UP on its own instead (AMT bullet above). + STP is off, and UDM port 6 carries the same VLANs as the uplink, so bridging it + would loop the site LAN. It is held admin-UP on its own instead (AMT bullet above). - **GPU: RTX 2000E Ada at `01:00.0`** (`10de:28b0`), installed 2026-09-25. The NIC pins held: the X710 moved to bus 03 and every NIC kept its name. **NVIDIA 580.178.04** (open modules, DKMS) has been on the host since 2026-09-25 at 1527