Files

T

vh 2118449881 feat(nh3-pve): prepare for GPU install — pin NIC names by MAC, pull AMT port from vmbr0

nh3-pve and esh-pve are the same Minisforum MS-01 (BIOS AHWSA.1.17). With a
card in the x16 slot its root port takes bus 01 and every NIC moves down a
bus (measured on esh-pve), so predictable names change (enp2s0f0np0 ->
enp3s0f0np0 etc.) and vmbr0 would boot with no uplink. systemd .link files
now pin all NICs by MAC, baked into every initramfs and synced to the ESP;
udev confirms the files apply. The AMT-capable I226-LM (enp88s0) leaves
vmbr0's bridge-ports in the file (next boot), so cabling it for AMT cannot
loop the STP-less bridge.

Also: documented the NanoKVM (https://10.100.250.171) as nh3-pve's console
OOB and that AMT is not wired; nh3-dev's Beszel agent no longer binds NAS
shares (it died on the last NH3 cold start); post-boot checklist in
persistent-memory.

2026-09-25 10:56:17 -07:00

4.6 KiB

Raw Blame History

nh3-pve

Proxmox VE hypervisor for the NH3 site (nh3-vmhost.phasefinal.com).

Network

LAN IP: 10.100.250.60
FQDN: nh3-vmhost.phasefinal.com
Web UI: https://10.100.250.60:8006
SSH: root@nh3-pve (key auth)

Hardware

CPU: 13th Gen Intel Core i9-13900H
RAM: 62.5 GB
Kernel: 6.8.12-11-pve (Proxmox 8.x)
Storage: mostly networked — /mnt/pve/pfi-nh3-nas (42 TB) mounted from the Synology at 10.100.50.50:/volume1/VMStorage; ~27 TB used

What it runs

Guests as of 2026-09-24 (qm list / pct list is the live truth). Expected state after a power loss is the column that matters in a recovery: a guest listed as stays off is not a casualty.

ID	guest	kind	onboot	expected after power loss
100	nh3-docker (`10.100.50.40`)	VM	1	up
101	nh3-extdev (`10.100.50.42`)	VM	1	up
102	nh3-dev (`10.100.10.50`)	VM	1	up
104	nh3-laser	VM	—	stays off — on-demand (Prime, 2026-09-24)
105	pbs-nh3 (`10.100.50.90`)	VM	1	up (onboot added 2026-09-24; it had none and stayed down after the outage)
108	opnsense-lab	VM	0	stays off (lab)
103	nh3-wg	CT	1	up
106	nh3-headscale	CT	1	up
107	nh3-scale	CT	1	up (mesh subnet router + fleet egress proxy)

Power-loss recovery (2026-09-24 outage). Every guest boots at once, and nh3-nas is the slowest to serve NFS. NFS clients now mount nh3-nas shares on first access (playbooks/nh3-nfs-automount.yaml, applied to nh3-docker and nh3-dev), so a late NAS no longer leaves shares down. pbs-nh3's datastore line uses bg (background retry) instead. qm guest exec 105 works (the guest agent is on), which is the root path into pbs-nh3; infra-ops is not provisioned there. pfi-gx10 (10.100.50.60) is bare metal on this site, not a guest here, and did not power back on by itself.

Hardware identity, OOB, and the GPU-slot NIC-rename trap (2026-09-25)

Box: Minisforum MS-01 ("Venus Series", BIOS AHWSA.1.17): the same model and BIOS as esh-pve. i9-13900H, X710 2×SFP+, I226-V + I226-LM (vPro/AMT), one PCIe x16 slot (x8 electrical, low-profile, single-slot, slot power only).
Uplink: only enp2s0f0np0 (SFP+ 10G) has a cable. The bridge carries the I226-V's MAC …:96:0d because it is the first port listed.
AMT: NOT wired. The AMT-capable I226-LM (enp88s0, MAC 58:47:ca:76:96:0e) has no cable. The ME is present (/dev/mei0, "AMT SOL Redirection" 00:16.3), but MEBx provisioning status is unknown. To wire it: cable the LM port, then at boot press Ctrl+P → set the MEBx password, enable manageability, set network (static or DHCP), KVM on, User Opt-in = None, activate network access. MEBx can be driven remotely through the NanoKVM below.
Console OOB exists: a Sipeed NanoKVM is attached (USB 3346:1009 on the host; web UI https://10.100.250.171, switch port 23, nh3-mgmt). It gives video and keyboard, so BIOS, MEBx and a host that booted without network are all reachable remotely. Power control through it is unverified (the MS-01 has no standard ATX front-panel header). ⚠ It captures the iGPU's HDMI, and RTX-class cards output mini-DP only, so keep the BIOS primary display on the iGPU after a GPU install.
⚠ A GPU in the slot RENAMES every NIC. With a card present the slot's root port 00:01.0 takes PCI bus 01 and everything behind it moves down a bus (measured on esh-pve): X710 02→03, I226-V 57→58, I226-LM 58→59, Wi-Fi 59→5a. The names would become enp3s0f0np0, enp88s0, enp89s0, wlp90s0. vmbr0 would then boot with no uplink and take the whole NH3 site's VMs off the network. Pinned 2026-09-25 by MAC with /etc/systemd/network/10-pin-*.link, also baked into every initramfs and synced to the ESP (playbooks/nh3-pve-pin-nic-names.yaml, verified with udevadm test-builtin net_setup_link). If the host ever boots without network anyway: open the NanoKVM console, run ip -br link, and compare names against /etc/network/interfaces (backup in /root/nic-pin-2026-09-25/).
enp88s0 (the AMT port) is no longer a vmbr0 bridge port (file edited 2026-09-25, effective next boot). STP is off, so bridging a second cabled uplink into the same L2 would loop the site LAN.

Refresh state

scripts/refresh-server-info.sh nh3-pve

Same Proxmox-inspect caveat as pfi-pve: snapshots capture host-level detail only. VM/LXC/storage enumeration needs the Proxmox tooling manually.

Placement rule

Hypervisor. New NH-site workloads land here as VMs.

4.6 KiB Raw Blame History Unescape Escape