diff --git a/dns/internal.yaml b/dns/internal.yaml index 5829bef..f3fe32c 100644 --- a/dns/internal.yaml +++ b/dns/internal.yaml @@ -82,7 +82,7 @@ hosts: - {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box} - {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+} - {name: nh3-pve, site: nh3, v4: 10.100.250.60, note: Proxmox hypervisor} - - {name: nh3-pve-amt, site: nh3, v4: 10.100.0.151, note: Intel AMT (vPro) on nh3-pve's I226-LM — UDM DHCP reservation, untagged nh3-default; web/WS-Man TLS :16993, redirection :664} + - {name: nh3-pve-amt, site: nh3, v4: 10.100.250.61, note: Intel AMT (vPro) on nh3-pve's I226-LM — UDM port 6 (native nh3-mgmt), DHCP reservation; web/WS-Man TLS :16993, redirection :664} - {name: pbs-nh3, site: nh3, v4: 10.100.50.90, note: Proxmox Backup Server — DR mirror} - {name: nh3-gw, site: nh3, v4: 10.100.0.1, note: UniFi UDM Pro SE — gateway + controller} # Irvine is not its own DNS site: it is a single box (irv-ml1) reached over diff --git a/persistent-memory.md b/persistent-memory.md index 59f3c6b..93fab4c 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -132,14 +132,18 @@ hermes-gateway, Booth, ttyd, zellij-web and dev-backup are running. `servers/nh3-ml1/README.md` exists. - ⏳ **Prime's call: gateway routing.** Recommendation: load-share `qwen3-embedding` + `reranker` across esh-ml1 and nh3-ml1. -- ✅ **AMT LIVE 2224: `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`, - UDM reservation). It had no IP because Linux had the port DOWN. `igc` powers - off the PHY and AMT loses its link. Our own 09-25 removal of `enp88s0` from - vmbr0 caused that. Fix: `auto enp88s0` (up, no IP, unbridged) plus IPv6 off via - sysctl.d, persistent. The cable is on **UDM port 6** (`forward: all`; - AMT is untagged, so nh3-default). - **Open:** check that KVM + Opt-in None are set (needs the AMT admin password), - then move the NanoKVM to the gx10. +- **AMT → nh3-mgmt at 2237 (Prime): `https://10.100.250.61:16993`** + (`nh3-pve-amt.nh3.internal`, Homepage *NH3-PVE-AMT*). UDM port 6 is native + mgmt with tags blocked, and the reservation is `.61`. + - ⏳ AMT is still sitting on its old `10.100.0.151` lease and is unreachable + until that lease rebinds or expires: ~1920 to ~2224 on 09-26. AMT does not + re-DHCP when its VLAN changes or its link drops. + - A background poll watches `.61`. + - Faster route: the AMT admin password (WS-Man). + - Earlier root cause: Linux had the port down (the `igc` PHY was off). Fixed + with `auto enp88s0` plus IPv6 off and `arp_ignore=8` via sysctl.d. + - **Open:** check KVM + Opt-in None; a dummy HDMI plug before the NanoKVM + moves. - Found and fixed: **lxc-pve 6.0.0-1 broke Docker in the CT** (runc 1.5 sysctl reopen denied). Upgraded that one package to 6.0.0-2 (Proxmox fix #7006), and `gpu-lxc.yaml` now does this itself. diff --git a/servers/nh3-pve/README.md b/servers/nh3-pve/README.md index 7eb3366..1ae55db 100644 --- a/servers/nh3-pve/README.md +++ b/servers/nh3-pve/README.md @@ -56,21 +56,39 @@ not power back on by itself. vmbr0 members, so either cage works. **Never drop either port from the bridge** without checking which one has carrier (`ip -br link`). The bridge carries the I226-V's MAC `…:96:0d` because it is the first port listed. -- **AMT: LIVE at `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`), - Intel AMT 16.1.25, since 2026-09-25 2224. It uses a self-signed cert and TLS - 1.2 with legacy renegotiation, so OpenSSL 3 clients need - `Options = UnsafeLegacyRenegotiation`; browsers cope. 16992 (plain HTTP) is - closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does not answer ping. +- **AMT: `https://10.100.250.61:16993`** (`nh3-pve-amt.nh3.internal`; Homepage + card *NH3-PVE-AMT* under Infra - NH3). Intel AMT 16.1.25. It came up at 2224 + on 2026-09-25 as `10.100.0.151` and was moved to nh3-mgmt at 2237 on Prime's + order. + - ⏳ **Until its old lease runs out, AMT still holds `10.100.0.151`**, and + nothing reaches it at either address. AMT does not re-DHCP when its VLAN + changes. + - A link drop, real or `ethtool -r`, makes it send only one ARP for its old + gateway `10.100.0.1`, which gets no answer. It sends no DHCP. + - The lease is 24 h from 2224 on 09-25. Renewal at 12 h goes unicast to + `10.100.0.1`, which is unreachable from mgmt. The rebind at ~21 h + (~1920 on 09-26) or expiry (~2224 on 09-26) should NAK it or start a new + DHCP, and the reservation then gives `.61`. + - The faster route needs the AMT admin password (WS-Man) or the MEBx menu. + - It uses a self-signed cert and TLS 1.2 with legacy renegotiation, so OpenSSL 3 + clients need `Options = UnsafeLegacyRenegotiation`; browsers cope. 16992 + (plain HTTP) is closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does + not answer ping. - **Port:** the I226-LM `enp88s0` (MAC `58:47:ca:76:96:0e`) is cabled to **PFI-UDMSE port 6** (Prime; confirmed from the UDM, which lists the AMT MAC - on port 6 at 1 Gb). Port 6 is `forward: all`, meaning untagged nh3-default - plus every tagged VLAN, so AMT (untagged) lands on **nh3-default - `10.100.0.0/24`**. + on port 6 at 1 Gb). Since 2237 on 2026-09-25 the port override is + **native nh3-mgmt, tagged VLANs blocked** (name "nh3-pve AMT"). It was + `forward: all`, with untagged nh3-default. + Reservation `nh3-pve-amt` → `10.100.250.61`. + - **ARP guard:** port 6 now puts `enp88s0` on the same untagged L2 as vmbr0 + (`10.100.250.60`). Without a guard, Linux would answer ARP for `.60` out of + `enp88s0` with MAC `…:0e`, which AMT shares, and the site would see nh3-pve at + two MACs. `/etc/sysctl.d/90-amt-port.conf` sets `arp_ignore=8` and + `arp_announce=2` on `enp88s0`, next to the IPv6 disable. ⚠ The first pass wrongly placed it on nh3-sw1, for two reasons. The UDM's port table was read 15 s after link-up, before the UDM refreshed it (it polls, and lags about a minute). And nh3-sw1's STP BPDUs, seen on the wire, are flooded through the UDM's switch, so they do not locate the cable. - - **Lease:** UDM DHCP reservation `nh3-pve-amt` → `10.100.0.151`. - ⚠ **Why it had no IP: Linux had the port DOWN.** When `igc` downs the I226-LM it powers off the PHY, and AMT's link goes with it (measured: "Link detected: no" while down, a 1 Gb link and a DHCP lease within minutes of `ip link set @@ -86,10 +104,6 @@ not power back on by itself. - **Not yet verified:** KVM redirection is enabled with User Opt-in = None (this needs the AMT admin password), and AMT keeps its link with the host powered off (this depends on the MEBx power policy). - - Better home, not done (Prime's call): make UDM port 6 native nh3-mgmt - (VLAN 250) and drop the tagged VLANs, so AMT alone sits on the management - network. It would need a new reservation and a DNS change. The UDM API key - can do it. - MEBx menu: Ctrl+P at boot. - **Console OOB exists: a Sipeed NanoKVM** is attached (USB `3346:1009` on the host; web UI **`https://10.100.250.171`**, switch port 23, nh3-mgmt). It gives video and diff --git a/stacks/homepage/conf/services.yaml b/stacks/homepage/conf/services.yaml index 5ee69ed..5d5628e 100644 --- a/stacks/homepage/conf/services.yaml +++ b/stacks/homepage/conf/services.yaml @@ -192,6 +192,14 @@ siteMonitor: https://10.100.250.60:8006/ icon: si-proxmox description: Proxmox hypervisor (nh3-pve) + # Intel AMT on nh3-pve's I226-LM (UDM port 6, nh3-mgmt, DHCP reservation). + # No siteMonitor/ping on purpose: AMT drops ICMP, and its TLS needs legacy + # renegotiation, which Homepage's fetch refuses, so a monitor reads DOWN forever. + # Remote screen needs MeshCommander, not this page (servers/nh3-pve/README.md). + - NH3-PVE-AMT: + href: https://10.100.250.61:16993/ + icon: si-intel + description: Intel AMT (vPro) — OOB for nh3-pve; login admin / MEBx password - NH3-VM-Docker: href: http://10.100.50.40:5001 icon: si-docker