fix(nh3-pve): AMT reachable — hold the vPro port up; DHCP reservation + DNS

AMT had no IP because Linux had enp88s0 admin-down (it left vmbr0 on 09-25 and
had no auto stanza); igc powers off the I226-LM PHY when down, so AMT had no
link. With the port up (no IP, unbridged) AMT took a lease within minutes.

- nh3-pve: `auto enp88s0` (manual, unbridged) and sysctl.d disable_ipv6 on it;
  backup /root/interfaces.bak-20260925-amt; runtime already matched, no reload.
- UDM: reservation nh3-pve-amt -> 10.100.0.151 (untagged nh3-default; the
  cable is on an nh3-sw1 trunk, not the UDM).
- DNS nh3-pve-amt.nh3.internal. AMT 16.1.25 answers TLS :16993 and :664.
This commit is contained in:
vh
2026-09-25 22:27:44 -07:00
parent 6d901c0e8d
commit 0f032fac2c
3 changed files with 43 additions and 22 deletions
+1
View File
@@ -82,6 +82,7 @@ hosts:
- {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box} - {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box}
- {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+} - {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+}
- {name: nh3-pve, site: nh3, v4: 10.100.250.60, note: Proxmox hypervisor} - {name: nh3-pve, site: nh3, v4: 10.100.250.60, note: Proxmox hypervisor}
- {name: nh3-pve-amt, site: nh3, v4: 10.100.0.151, note: Intel AMT (vPro) on nh3-pve's I226-LM — UDM DHCP reservation, untagged nh3-default; web/WS-Man TLS :16993, redirection :664}
- {name: pbs-nh3, site: nh3, v4: 10.100.50.90, note: Proxmox Backup Server — DR mirror} - {name: pbs-nh3, site: nh3, v4: 10.100.50.90, note: Proxmox Backup Server — DR mirror}
- {name: nh3-gw, site: nh3, v4: 10.100.0.1, note: UniFi UDM Pro SE — gateway + controller} - {name: nh3-gw, site: nh3, v4: 10.100.0.1, note: UniFi UDM Pro SE — gateway + controller}
# Irvine is not its own DNS site: it is a single box (irv-ml1) reached over # Irvine is not its own DNS site: it is a single box (irv-ml1) reached over
+8 -5
View File
@@ -132,11 +132,14 @@ hermes-gateway, Booth, ttyd, zellij-web and dev-backup are running.
`servers/nh3-ml1/README.md` exists. `servers/nh3-ml1/README.md` exists.
- ⏳ **Prime's call: gateway routing.** Recommendation: load-share - ⏳ **Prime's call: gateway routing.** Recommendation: load-share
`qwen3-embedding` + `reranker` across esh-ml1 and nh3-ml1. `qwen3-embedding` + `reranker` across esh-ml1 and nh3-ml1.
- ⚠ **AMT is cabled (1 Gb link) and reported enabled, but not on the network.** No - ✅ **AMT LIVE 2224: `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`,
UDM lease for `…:96:0e`, and no 16992/16993 on the four NH3 /24s (the sweep was UDM reservation). It had no IP because Linux had the port DOWN. `igc` powers
positive-controlled). It is not on the USW Pro 24, so probably nh3-sw1. Asked off the PHY and AMT loses its link. Our own 09-25 removal of `enp88s0` from
Prime for the MEBx network settings and which switch port. **The NanoKVM stays vmbr0 caused that. Fix: `auto enp88s0` (up, no IP, unbridged) plus IPv6 off via
on nh3-pve until AMT KVM is confirmed.** sysctl.d, persistent. The cable is on an **nh3-sw1 trunk** (untagged
nh3-default), not the UDM.
**Open:** check that KVM + Opt-in None are set (needs the AMT admin password),
then move the NanoKVM to the gx10.
- Found and fixed: **lxc-pve 6.0.0-1 broke Docker in the CT** (runc 1.5 sysctl - Found and fixed: **lxc-pve 6.0.0-1 broke Docker in the CT** (runc 1.5 sysctl
reopen denied). Upgraded that one package to 6.0.0-2 (Proxmox fix #7006), and reopen denied). Upgraded that one package to 6.0.0-2 (Proxmox fix #7006), and
`gpu-lxc.yaml` now does this itself. `gpu-lxc.yaml` now does this itself.
+34 -17
View File
@@ -56,19 +56,34 @@ not power back on by itself.
vmbr0 members, so either cage works. **Never drop either port from the bridge** vmbr0 members, so either cage works. **Never drop either port from the bridge**
without checking which one has carrier (`ip -br link`). The bridge carries the without checking which one has carrier (`ip -br link`). The bridge carries the
I226-V's MAC `…:96:0d` because it is the first port listed. I226-V's MAC `…:96:0d` because it is the first port listed.
- **AMT: cabled and enabled, but NOT reachable (2026-09-25 visit).** The I226-LM - **AMT: LIVE at `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`),
(`enp88s0`, MAC `58:47:ca:76:96:0e`) now has a 1 Gb link (measured at 1552 by Intel AMT 16.1.25, since 2026-09-25 2224. It uses a self-signed cert and TLS
bringing the port up unbridged for a few seconds). Prime reports AMT enabled in 1.2 with legacy renegotiation, so OpenSSL 3 clients need
MEBx. **Nothing answers on the network, though.** The UDM has no client or lease `Options = UnsafeLegacyRenegotiation`; browsers cope. 16992 (plain HTTP) is
for `…:96:0e`, and no host on `10.100.{0,10,50,250}.0/24` has 16992 or 16993 closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does not answer ping.
open. The sweep was checked against known-open ports and does see them. The - **Port:** the I226-LM `enp88s0` (MAC `58:47:ca:76:96:0e`) is cabled to a
cable is not on the USW Pro 24, whose up ports are 19, 22, 23 and 26, all **trunk port on nh3-sw1**, not the UDM (seen from the host: STP from
accounted for. So it is on another switch, most likely nh3-sw1 `48:a9:8a:40:8a:51`, untagged nh3-default plus tagged VLANs 10/50/90/250).
(`10.100.250.2`, no infra-ops access). Likely causes: MEBx "Activate Network AMT is untagged, so it lands on **nh3-default `10.100.0.0/24`**.
Access" was not done, a static IP outside those subnets, or a switch port on a - **Lease:** UDM DHCP reservation `nh3-pve-amt` → `10.100.0.151`.
VLAN that gets no DHCP. MEBx menu: Ctrl+P at boot → password, manageability on, - ⚠ **Why it had no IP: Linux had the port DOWN.** When `igc` downs the I226-LM
network (static or DHCP), KVM on, User Opt-in = None, activate network access. it powers off the PHY, and AMT's link goes with it (measured: "Link detected:
**Keep the NanoKVM here until AMT KVM is confirmed.** no" while down, a 1 Gb link and a DHCP lease within minutes of `ip link set
up`).
- It was down because we pulled `enp88s0` out of vmbr0 on 2026-09-25 to
avoid a loop, which left it unconfigured at boot. Our own change caused it.
- **Fix, persistent:** `auto enp88s0` / `iface enp88s0 inet manual` in
`/etc/network/interfaces` (PVE's "Autostart"; it stays up with no address
and no bridge). `/etc/sysctl.d/90-amt-port.conf` turns IPv6 off on it, so the
host never SLAACs an address or default route onto nh3-default. Backup:
`/root/interfaces.bak-20260925-amt`.
- **Never down this port or remove the `auto` line**, or AMT goes dark.
- **Not yet verified:** KVM redirection is enabled with User Opt-in = None
(this needs the AMT admin password), and AMT keeps its link with the host
powered off (this depends on the MEBx power policy).
- Better home, not done: nh3-mgmt (VLAN 250), by setting that nh3-sw1 port's
native VLAN to 250. infra-ops has no nh3-sw1 access.
- MEBx menu: Ctrl+P at boot.
- **Console OOB exists: a Sipeed NanoKVM** is attached (USB `3346:1009` on the host; - **Console OOB exists: a Sipeed NanoKVM** is attached (USB `3346:1009` on the host;
web UI **`https://10.100.250.171`**, switch port 23, nh3-mgmt). It gives video and web UI **`https://10.100.250.171`**, switch port 23, nh3-mgmt). It gives video and
keyboard, so BIOS, MEBx and a host that booted without network are all reachable keyboard, so BIOS, MEBx and a host that booted without network are all reachable
@@ -85,9 +100,9 @@ not power back on by itself.
verified with `udevadm test-builtin net_setup_link`). If the host ever boots verified with `udevadm test-builtin net_setup_link`). If the host ever boots
without network anyway: open the NanoKVM console, run `ip -br link`, and compare without network anyway: open the NanoKVM console, run `ip -br link`, and compare
names against `/etc/network/interfaces` (backup in `/root/nic-pin-2026-09-25/`). names against `/etc/network/interfaces` (backup in `/root/nic-pin-2026-09-25/`).
- **`enp88s0` (the AMT port) is no longer a vmbr0 bridge port** (file edited - **`enp88s0` (the AMT port) is not a vmbr0 bridge port** (removed 2026-09-25).
2026-09-25, effective next boot). STP is off, so bridging a second cabled uplink STP is off, and that port is on the same nh3-sw1 trunk, so bridging it would
into the same L2 would loop the site LAN. loop the site LAN. It is held admin-UP on its own instead (AMT bullet above).
- **GPU: RTX 2000E Ada at `01:00.0`** (`10de:28b0`), installed 2026-09-25. The - **GPU: RTX 2000E Ada at `01:00.0`** (`10de:28b0`), installed 2026-09-25. The
NIC pins held: the X710 moved to bus 03 and every NIC kept its name. **NVIDIA NIC pins held: the X710 moved to bus 03 and every NIC kept its name. **NVIDIA
580.178.04** (open modules, DKMS) has been on the host since 2026-09-25 at 1527 580.178.04** (open modules, DKMS) has been on the host since 2026-09-25 at 1527
@@ -133,7 +148,9 @@ not power back on by itself.
- **OOB plan status.** Prime ruled on 2026-09-25 via Miranda to HOLD console-blind - **OOB plan status.** Prime ruled on 2026-09-25 via Miranda to HOLD console-blind
until the site visit. Target: the NanoKVM moves to pfi-gx10, and this MS-01 uses until the site visit. Target: the NanoKVM moves to pfi-gx10, and this MS-01 uses
its own AMT. The visit did IGFX, turned SB off and cabled plus enabled AMT. its own AMT. The visit did IGFX, turned SB off and cabled plus enabled AMT.
**Still open:** AMT is not on the network (above), so the NanoKVM stays here. AMT came up on the network at 2224 once the host stopped downing its port.
**Still open:** confirm AMT KVM shows the console. Only then move the NanoKVM to
the gx10.
## Refresh state ## Refresh state