vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(nh3-pve): move AMT to nh3-mgmt (UDM port 6 native VLAN 250) + Homepage link

Browse Source 
- PFI-UDMSE port 6 override: native nh3-mgmt, tagged VLANs blocked (was
  forward all / native default). Reservation nh3-pve-amt -> 10.100.250.61.
- nh3-pve: arp_ignore=8 / arp_announce=2 on enp88s0 (now on vmbr0's untagged
  L2) so the host never answers ARP for 10.100.250.60 with the AMT port's MAC.
- DNS nh3-pve-amt.nh3.internal -> 10.100.250.61.
- Homepage: NH3-PVE-AMT card under Infra - NH3 (no siteMonitor/ping: AMT drops
  ICMP and its legacy-renegotiation TLS fails Homepage's fetch).
- AMT keeps its old 10.100.0.151 lease until rebind/expiry (~1920-2224 PT
  2026-09-26); it does not re-DHCP on a VLAN change or link drop (measured).
This commit is contained in:
vh
2026-09-25 22:59:42 -07:00
parent 2c248200b5
commit b3b75c16f4
4 changed files with 48 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dns/internal.yaml
+1 -1
View File
@@ -82,7 +82,7 @@ hosts:
- {name: nh3-extdev, site: nh3, v4: 10.100.50.42, note: manager / external-dev box}
- {name: nh3-nas, site: nh3, v4: 10.100.50.50, note: Synology RS2418+}
- {name: nh3-pve, site: nh3, v4: 10.100.250.60, note: Proxmox hypervisor}
- {name: nh3-pve-amt, site: nh3, v4: 10.100.0.151, note: Intel AMT (vPro) on nh3-pve's I226-LM — UDM DHCP reservation, untagged nh3-default; web/WS-Man TLS :16993, redirection :664}
- {name: nh3-pve-amt, site: nh3, v4: 10.100.250.61, note: Intel AMT (vPro) on nh3-pve's I226-LM — UDM port 6 (native nh3-mgmt), DHCP reservation; web/WS-Man TLS :16993, redirection :664}
- {name: pbs-nh3, site: nh3, v4: 10.100.50.90, note: Proxmox Backup Server — DR mirror}
- {name: nh3-gw, site: nh3, v4: 10.100.0.1, note: UniFi UDM Pro SE — gateway + controller}
# Irvine is not its own DNS site: it is a single box (irv-ml1) reached over
persistent-memory.md
+12 -8
View File
@@ -132,14 +132,18 @@ hermes-gateway, Booth, ttyd, zellij-web and dev-backup are running.
`servers/nh3-ml1/README.md` exists.
- ⏳ **Prime's call: gateway routing.** Recommendation: load-share
`qwen3-embedding` + `reranker` across esh-ml1 and nh3-ml1.
- ✅ **AMT LIVE 2224: `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`,
UDM reservation). It had no IP because Linux had the port DOWN. `igc` powers
off the PHY and AMT loses its link. Our own 09-25 removal of `enp88s0` from
vmbr0 caused that. Fix: `auto enp88s0` (up, no IP, unbridged) plus IPv6 off via
sysctl.d, persistent. The cable is on **UDM port 6** (`forward: all`;
AMT is untagged, so nh3-default).
**Open:** check that KVM + Opt-in None are set (needs the AMT admin password),
then move the NanoKVM to the gx10.
- **AMT → nh3-mgmt at 2237 (Prime): `https://10.100.250.61:16993`**
(`nh3-pve-amt.nh3.internal`, Homepage *NH3-PVE-AMT*). UDM port 6 is native
mgmt with tags blocked, and the reservation is `.61`.
- ⏳ AMT is still sitting on its old `10.100.0.151` lease and is unreachable
until that lease rebinds or expires: ~1920 to ~2224 on 09-26. AMT does not
re-DHCP when its VLAN changes or its link drops.
- A background poll watches `.61`.
- Faster route: the AMT admin password (WS-Man).
- Earlier root cause: Linux had the port down (the `igc` PHY was off). Fixed
with `auto enp88s0` plus IPv6 off and `arp_ignore=8` via sysctl.d.
- **Open:** check KVM + Opt-in None; a dummy HDMI plug before the NanoKVM
moves.
- Found and fixed: **lxc-pve 6.0.0-1 broke Docker in the CT** (runc 1.5 sysctl
reopen denied). Upgraded that one package to 6.0.0-2 (Proxmox fix #7006), and
`gpu-lxc.yaml` now does this itself.
servers/nh3-pve/README.md
+27 -13
View File
@@ -56,21 +56,39 @@ not power back on by itself.
vmbr0 members, so either cage works. **Never drop either port from the bridge**
without checking which one has carrier (`ip -br link`). The bridge carries the
I226-V's MAC `…:96:0d` because it is the first port listed.
- **AMT: LIVE at `https://10.100.0.151:16993`** (`nh3-pve-amt.nh3.internal`),
Intel AMT 16.1.25, since 2026-09-25 2224. It uses a self-signed cert and TLS
1.2 with legacy renegotiation, so OpenSSL 3 clients need
`Options = UnsafeLegacyRenegotiation`; browsers cope. 16992 (plain HTTP) is
closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does not answer ping.
- **AMT: `https://10.100.250.61:16993`** (`nh3-pve-amt.nh3.internal`; Homepage
card *NH3-PVE-AMT* under Infra - NH3). Intel AMT 16.1.25. It came up at 2224
on 2026-09-25 as `10.100.0.151` and was moved to nh3-mgmt at 2237 on Prime's
order.
- ⏳ **Until its old lease runs out, AMT still holds `10.100.0.151`**, and
nothing reaches it at either address. AMT does not re-DHCP when its VLAN
changes.
- A link drop, real or `ethtool -r`, makes it send only one ARP for its old
gateway `10.100.0.1`, which gets no answer. It sends no DHCP.
- The lease is 24 h from 2224 on 09-25. Renewal at 12 h goes unicast to
`10.100.0.1`, which is unreachable from mgmt. The rebind at ~21 h
(~1920 on 09-26) or expiry (~2224 on 09-26) should NAK it or start a new
DHCP, and the reservation then gives `.61`.
- The faster route needs the AMT admin password (WS-Man) or the MEBx menu.
- It uses a self-signed cert and TLS 1.2 with legacy renegotiation, so OpenSSL 3
clients need `Options = UnsafeLegacyRenegotiation`; browsers cope. 16992
(plain HTTP) is closed. 664 (TLS redirection: SOL/IDER/KVM) is open. It does
not answer ping.
- **Port:** the I226-LM `enp88s0` (MAC `58:47:ca:76:96:0e`) is cabled to
**PFI-UDMSE port 6** (Prime; confirmed from the UDM, which lists the AMT MAC
on port 6 at 1 Gb). Port 6 is `forward: all`, meaning untagged nh3-default
plus every tagged VLAN, so AMT (untagged) lands on **nh3-default
`10.100.0.0/24`**.
on port 6 at 1 Gb). Since 2237 on 2026-09-25 the port override is
**native nh3-mgmt, tagged VLANs blocked** (name "nh3-pve AMT"). It was
`forward: all`, with untagged nh3-default.
Reservation `nh3-pve-amt` → `10.100.250.61`.
- **ARP guard:** port 6 now puts `enp88s0` on the same untagged L2 as vmbr0
(`10.100.250.60`). Without a guard, Linux would answer ARP for `.60` out of
`enp88s0` with MAC `…:0e`, which AMT shares, and the site would see nh3-pve at
two MACs. `/etc/sysctl.d/90-amt-port.conf` sets `arp_ignore=8` and
`arp_announce=2` on `enp88s0`, next to the IPv6 disable.
⚠ The first pass wrongly placed it on nh3-sw1, for two reasons. The UDM's port
table was read 15 s after link-up, before the UDM refreshed it (it polls, and
lags about a minute). And nh3-sw1's STP BPDUs, seen on the wire, are flooded
through the UDM's switch, so they do not locate the cable.
- **Lease:** UDM DHCP reservation `nh3-pve-amt` → `10.100.0.151`.
- ⚠ **Why it had no IP: Linux had the port DOWN.** When `igc` downs the I226-LM
it powers off the PHY, and AMT's link goes with it (measured: "Link detected:
no" while down, a 1 Gb link and a DHCP lease within minutes of `ip link set
@@ -86,10 +104,6 @@ not power back on by itself.
- **Not yet verified:** KVM redirection is enabled with User Opt-in = None
(this needs the AMT admin password), and AMT keeps its link with the host
powered off (this depends on the MEBx power policy).
- Better home, not done (Prime's call): make UDM port 6 native nh3-mgmt
(VLAN 250) and drop the tagged VLANs, so AMT alone sits on the management
network. It would need a new reservation and a DNS change. The UDM API key
can do it.
- MEBx menu: Ctrl+P at boot.
- **Console OOB exists: a Sipeed NanoKVM** is attached (USB `3346:1009` on the host;
web UI **`https://10.100.250.171`**, switch port 23, nh3-mgmt). It gives video and
stacks/homepage/conf/services.yaml
+8
View File
@@ -192,6 +192,14 @@
siteMonitor: https://10.100.250.60:8006/
icon: si-proxmox
description: Proxmox hypervisor (nh3-pve)
# Intel AMT on nh3-pve's I226-LM (UDM port 6, nh3-mgmt, DHCP reservation).
# No siteMonitor/ping on purpose: AMT drops ICMP, and its TLS needs legacy
# renegotiation, which Homepage's fetch refuses, so a monitor reads DOWN forever.
# Remote screen needs MeshCommander, not this page (servers/nh3-pve/README.md).
- NH3-PVE-AMT:
href: https://10.100.250.61:16993/
icon: si-intel
description: Intel AMT (vPro) — OOB for nh3-pve; login admin / MEBx password
- NH3-VM-Docker:
href: http://10.100.50.40:5001
icon: si-docker