From 0f032fac2c65a2b55514cb0b8370dfdb7140c03d Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 25 Sep 2026 22:27:44 -0700 Subject: [PATCH] =?UTF-8?q?fix(nh3-pve):=20AMT=20reachable=20=E2=80=94=20h?= =?UTF-8?q?old=20the=20vPro=20port=20up;=20DHCP=20reservation=20+=20DNS?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AMT had no IP because Linux had enp88s0 admin-down (it left vmbr0 on 09-25 and had no auto stanza); igc powers off the I226-LM PHY when down, so AMT had no link. With the port up (no IP, unbridged) AMT took a lease within minutes. - nh3-pve: `auto enp88s0` (manual, unbridged) and sysctl.d disable_ipv6 on it; backup /root/interfaces.bak-20260925-amt; runtime already matched, no reload. - UDM: reservation nh3-pve-amt -> 10.100.0.151 (untagged nh3-default; the cable is on an nh3-sw1 trunk, not the UDM). - DNS nh3-pve-amt.nh3.internal. AMT 16.1.25 answers TLS :16993 and :664. --- dns/internal.yaml | 1 + persistent-memory.md | 13 ++++++---- servers/nh3-pve/README.md | 51 ++++++++++++++++++++++++++------------- 3 files changed, 43 insertions(+), 22 deletions(-) diff --git a/dns/internal.yaml b/dns/internal.yaml index 1ebf883..5829bef 100644 --- a/dns/internal.yaml +++ b/dns/internal.yaml @@ -82,6 +82,7 @@ hosts: - {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box} - {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+} - {name: nh3-pve, site: nh3, v4: 10.100.250.60, note: Proxmox hypervisor} + - {name: nh3-pve-amt, site: nh3, v4: 10.100.0.151, note: Intel AMT (vPro) on nh3-pve's I226-LM — UDM DHCP reservation, untagged nh3-default; web/WS-Man TLS :16993, redirection :664} - {name: pbs-nh3, site: nh3, v4: 10.100.50.90, note: Proxmox Backup Server — DR mirror} - {name: nh3-gw, site: nh3, v4: 10.100.0.1, note: UniFi UDM Pro SE — gateway + controller} # Irvine is not its own DNS site: it is a single box (irv-ml1) reached over diff --git a/persistent-memory.md b/persistent-memory.md index cb21978..98752f7 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -132,11 +132,14 @@ hermes-gateway, Booth, ttyd, zellij-web and dev-backup are running. `servers/nh3-ml1/README.md` exists. - ⏳ **Prime's call: gateway routing.** Recommendation: load-share `qwen3-embedding` + `reranker` across esh-ml1 and nh3-ml1. -- ⚠ **AMT is cabled (1 Gb link) and reported enabled, but not on the network.** No - UDM lease for `…:96:0e`, and no 16992/16993 on the four NH3 /24s (the sweep was - positive-controlled). It is not on the USW Pro 24, so probably nh3-sw1. Asked - Prime for the MEBx network settings and which switch port. **The NanoKVM stays - on nh3-pve until AMT KVM is confirmed.** +- ✅ **AMT LIVE 2224: `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`, + UDM reservation). It had no IP because Linux had the port DOWN. `igc` powers + off the PHY and AMT loses its link. Our own 09-25 removal of `enp88s0` from + vmbr0 caused that. Fix: `auto enp88s0` (up, no IP, unbridged) plus IPv6 off via + sysctl.d, persistent. The cable is on an **nh3-sw1 trunk** (untagged + nh3-default), not the UDM. + **Open:** check that KVM + Opt-in None are set (needs the AMT admin password), + then move the NanoKVM to the gx10. - Found and fixed: **lxc-pve 6.0.0-1 broke Docker in the CT** (runc 1.5 sysctl reopen denied). Upgraded that one package to 6.0.0-2 (Proxmox fix #7006), and `gpu-lxc.yaml` now does this itself. diff --git a/servers/nh3-pve/README.md b/servers/nh3-pve/README.md index d3fec4d..014910c 100644 --- a/servers/nh3-pve/README.md +++ b/servers/nh3-pve/README.md @@ -56,19 +56,34 @@ not power back on by itself. vmbr0 members, so either cage works. **Never drop either port from the bridge** without checking which one has carrier (`ip -br link`). The bridge carries the I226-V's MAC `…:96:0d` because it is the first port listed. -- **AMT: cabled and enabled, but NOT reachable (2026-09-25 visit).** The I226-LM - (`enp88s0`, MAC `58:47:ca:76:96:0e`) now has a 1 Gb link (measured at 1552 by - bringing the port up unbridged for a few seconds). Prime reports AMT enabled in - MEBx. **Nothing answers on the network, though.** The UDM has no client or lease - for `…:96:0e`, and no host on `10.100.{0,10,50,250}.0/24` has 16992 or 16993 - open. The sweep was checked against known-open ports and does see them. The - cable is not on the USW Pro 24, whose up ports are 19, 22, 23 and 26, all - accounted for. So it is on another switch, most likely nh3-sw1 - (`10.100.250.2`, no infra-ops access). Likely causes: MEBx "Activate Network - Access" was not done, a static IP outside those subnets, or a switch port on a - VLAN that gets no DHCP. MEBx menu: Ctrl+P at boot → password, manageability on, - network (static or DHCP), KVM on, User Opt-in = None, activate network access. - **Keep the NanoKVM here until AMT KVM is confirmed.** +- **AMT: LIVE at `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`), + Intel AMT 16.1.25, since 2026-09-25 2224. It uses a self-signed cert and TLS + 1.2 with legacy renegotiation, so OpenSSL 3 clients need + `Options = UnsafeLegacyRenegotiation`; browsers cope. 16992 (plain HTTP) is + closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does not answer ping. + - **Port:** the I226-LM `enp88s0` (MAC `58:47:ca:76:96:0e`) is cabled to a + **trunk port on nh3-sw1**, not the UDM (seen from the host: STP from + `48:a9:8a:40:8a:51`, untagged nh3-default plus tagged VLANs 10/50/90/250). + AMT is untagged, so it lands on **nh3-default `10.100.0.0/24`**. + - **Lease:** UDM DHCP reservation `nh3-pve-amt` → `10.100.0.151`. + - ⚠ **Why it had no IP: Linux had the port DOWN.** When `igc` downs the I226-LM + it powers off the PHY, and AMT's link goes with it (measured: "Link detected: + no" while down, a 1 Gb link and a DHCP lease within minutes of `ip link set + up`). + - It was down because we pulled `enp88s0` out of vmbr0 on 2026-09-25 to + avoid a loop, which left it unconfigured at boot. Our own change caused it. + - **Fix, persistent:** `auto enp88s0` / `iface enp88s0 inet manual` in + `/etc/network/interfaces` (PVE's "Autostart"; it stays up with no address + and no bridge). `/etc/sysctl.d/90-amt-port.conf` turns IPv6 off on it, so the + host never SLAACs an address or default route onto nh3-default. Backup: + `/root/interfaces.bak-20260925-amt`. + - **Never down this port or remove the `auto` line**, or AMT goes dark. + - **Not yet verified:** KVM redirection is enabled with User Opt-in = None + (this needs the AMT admin password), and AMT keeps its link with the host + powered off (this depends on the MEBx power policy). + - Better home, not done: nh3-mgmt (VLAN 250), by setting that nh3-sw1 port's + native VLAN to 250. infra-ops has no nh3-sw1 access. + - MEBx menu: Ctrl+P at boot. - **Console OOB exists: a Sipeed NanoKVM** is attached (USB `3346:1009` on the host; web UI **`https://10.100.250.171`**, switch port 23, nh3-mgmt). It gives video and keyboard, so BIOS, MEBx and a host that booted without network are all reachable @@ -85,9 +100,9 @@ not power back on by itself. verified with `udevadm test-builtin net_setup_link`). If the host ever boots without network anyway: open the NanoKVM console, run `ip -br link`, and compare names against `/etc/network/interfaces` (backup in `/root/nic-pin-2026-09-25/`). -- **`enp88s0` (the AMT port) is no longer a vmbr0 bridge port** (file edited - 2026-09-25, effective next boot). STP is off, so bridging a second cabled uplink - into the same L2 would loop the site LAN. +- **`enp88s0` (the AMT port) is not a vmbr0 bridge port** (removed 2026-09-25). + STP is off, and that port is on the same nh3-sw1 trunk, so bridging it would + loop the site LAN. It is held admin-UP on its own instead (AMT bullet above). - **GPU: RTX 2000E Ada at `01:00.0`** (`10de:28b0`), installed 2026-09-25. The NIC pins held: the X710 moved to bus 03 and every NIC kept its name. **NVIDIA 580.178.04** (open modules, DKMS) has been on the host since 2026-09-25 at 1527 @@ -133,7 +148,9 @@ not power back on by itself. - **OOB plan status.** Prime ruled on 2026-09-25 via Miranda to HOLD console-blind until the site visit. Target: the NanoKVM moves to pfi-gx10, and this MS-01 uses its own AMT. The visit did IGFX, turned SB off and cabled plus enabled AMT. - **Still open:** AMT is not on the network (above), so the NanoKVM stays here. + AMT came up on the network at 2224 once the host stopped downing its port. + **Still open:** confirm AMT KVM shows the console. Only then move the NanoKVM to + the gx10. ## Refresh state