CPU: 13th Gen Intel Core i9-13900H
RAM: 62.5 GB
Kernel: 6.8.12-11-pve (Proxmox 8.x)
Storage: mostly networked — /mnt/pve/pfi-nh3-nas (42 TB) mounted from the Synology at 10.100.50.50:/volume1/VMStorage; ~27 TB used

What it runs

Guests as of 2026-09-24 (qm list / pct list is the live truth). Expected state after a power loss is the column that matters in a recovery: a guest listed as stays off is not a casualty.

ID	guest	kind	onboot	expected after power loss
100	nh3-docker (`10.100.50.40`)	VM	1	up
101	nh3-extdev (`10.100.50.42`)	VM	1	up
102	nh3-dev (`10.100.10.50`)	VM	1	up
104	nh3-laser	VM	—	stays off — on-demand (Prime, 2026-09-24)
105	pbs-nh3 (`10.100.50.90`)	VM	1	up (onboot added 2026-09-24; it had none and stayed down after the outage)
108	opnsense-lab	VM	0	stays off (lab)
103	nh3-wg	CT	1	up
106	nh3-headscale	CT	1	up
107	nh3-scale	CT	1	up (mesh subnet router + fleet egress proxy)

Power-loss recovery (2026-09-24 outage). Every guest boots at once, and nh3-nas is the slowest to serve NFS. NFS clients now mount nh3-nas shares on first access (playbooks/nh3-nfs-automount.yaml, applied to nh3-docker and nh3-dev), so a late NAS no longer leaves shares down. pbs-nh3's datastore line uses bg (background retry) instead. qm guest exec 105 works (the guest agent is on), which is the root path into pbs-nh3; infra-ops is not provisioned there. pfi-gx10 (10.100.50.60) is bare metal on this site, not a guest here, and did not power back on by itself.

Hardware identity, OOB, and the GPU-slot NIC-rename trap (2026-09-25)

Box: Minisforum MS-01 ("Venus Series", BIOS AHWSA.1.17): the same model and BIOS as esh-pve. i9-13900H, X710 2×SFP+, I226-V + I226-LM (vPro/AMT), one PCIe x16 slot (x8 electrical, low-profile, single-slot, slot power only).
Uplink: one SFP+ 10G port has a cable. Since the 2026-09-25 GPU install it is enp2s0f1np1 (MAC …:96:0c, PCI 03:00.1). Before that it was enp2s0f0np0 (…:96:0b), so the cable moved cages during the visit. The pin follows MAC, so this is a physical move and not a rename. Both SFP+ ports are vmbr0 members, so either cage works. Never drop either port from the bridge without checking which one has carrier (ip -br link). The bridge carries the I226-V's MAC …:96:0d because it is the first port listed.
AMT: NOT wired. The AMT-capable I226-LM (enp88s0, MAC 58:47:ca:76:96:0e) has no cable. The ME is present (/dev/mei0, "AMT SOL Redirection" 00:16.3), but MEBx provisioning status is unknown. To wire it: cable the LM port, then at boot press Ctrl+P → set the MEBx password, enable manageability, set network (static or DHCP), KVM on, User Opt-in = None, activate network access. MEBx can be driven remotely through the NanoKVM below.
Console OOB exists: a Sipeed NanoKVM is attached (USB 3346:1009 on the host; web UI https://10.100.250.171, switch port 23, nh3-mgmt). It gives video and keyboard, so BIOS, MEBx and a host that booted without network are all reachable remotely. Power control through it is unverified (the MS-01 has no standard ATX front-panel header). ⚠ It captures the iGPU's HDMI, and RTX-class cards output mini-DP only, so keep the BIOS primary display on the iGPU after a GPU install.
⚠ A GPU in the slot RENAMES every NIC. With a card present the slot's root port 00:01.0 takes PCI bus 01 and everything behind it moves down a bus (measured on esh-pve): X710 02→03, I226-V 57→58, I226-LM 58→59, Wi-Fi 59→5a. The names would become enp3s0f0np0, enp88s0, enp89s0, wlp90s0. vmbr0 would then boot with no uplink and take the whole NH3 site's VMs off the network. Pinned 2026-09-25 by MAC with /etc/systemd/network/10-pin-*.link, also baked into every initramfs and synced to the ESP (playbooks/nh3-pve-pin-nic-names.yaml, verified with udevadm test-builtin net_setup_link). If the host ever boots without network anyway: open the NanoKVM console, run ip -br link, and compare names against /etc/network/interfaces (backup in /root/nic-pin-2026-09-25/).
enp88s0 (the AMT port) is no longer a vmbr0 bridge port (file edited 2026-09-25, effective next boot). STP is off, so bridging a second cabled uplink into the same L2 would loop the site LAN.
GPU installed 2026-09-25: RTX 2000E Ada at 01:00.0 (10de:28b0). The pins held: the X710 moved to bus 03 and every NIC kept its name. No NVIDIA driver yet, so nouveau binds it (gsp ctor failed: -2 is expected without GSP firmware). ⚠ After the install the iGPU is gone from the PCI bus: 00:02.0 enumerated on the 2026-09-24 boot and is absent now, /dev/dri does not exist, and the RTX is boot_vga=1. The BIOS's Auto primary display picked the PCIe card and hid the iGPU, so the NanoKVM's iGPU-HDMI capture has no source. Fix: BIOS → Primary Display = IGFX (or enable iGPU Multi-Monitor). Reaching the BIOS now needs a display on the RTX's mini-DP, for example the NanoKVM through a mini-DP→HDMI adapter. Verify with lspci | grep 00:02.0 and boot_vga on 00:02.0.
OOB decision (Prime via Miranda, 2026-09-25): HOLD. nh3-pve stays console-blind until the next NH3 site visit. No BIOS change and no reboot until then. The accepted risk: a boot without network means a site trip. Target end state: the NanoKVM moves to pfi-gx10, and this MS-01 gets out-of-band access through its own vPro/AMT (cable the I226-LM enp88s0 and provision MEBx; see the AMT bullet above). ⚠ Do the BIOS fix on that same visit. AMT's KVM redirection captures only the Intel iGPU's framebuffer, so with the iGPU hidden as it is now, AMT KVM would be just as blind as the NanoKVM. Order: while the NanoKVM is still here (on the RTX through an adapter), set Primary Display = IGFX, provision MEBx, confirm AMT KVM shows the console, and only then move the NanoKVM to the gx10.

Refresh state

scripts/refresh-server-info.sh nh3-pve

Same Proxmox-inspect caveat as pfi-pve: snapshots capture host-level detail only. VM/LXC/storage enumeration needs the Proxmox tooling manually.

Placement rule

Hypervisor. New NH-site workloads land here as VMs.

README.md Unescape Escape

nh3-pve

Network

Hardware

What it runs

Hardware identity, OOB, and the GPU-slot NIC-rename trap (2026-09-25)

Refresh state

Placement rule

README.md