vh
1cbde502f4
After the 2026-09-24 NH3 power outage every NFS line from nh3-nas failed at boot: nh3-docker's /mnt/compose and /mnt/backup, and nh3-dev's /mnt/backup. The NAS is the slowest box to serve NFS, and the plain fstab mounts stayed down until remounted by hand. nh3-dev's /mnt/smithy already used x-systemd.automount, and the same boot shows it self-healing on its next access. playbooks/nh3-nfs-automount.yaml brings the other lines to that shape (_netdev,nofail,x-systemd.automount,x-systemd.mount-timeout=30). Hard-mount semantics are unchanged; only WHEN the mount happens moves. Applied and read back on nh3-docker and nh3-dev: every nh3-nas line carries automount, each automount unit is active, and each share is NFS-mounted after access. A second run on nh3-docker is a no-op. The fstab check judges only errors the rewrite ADDS, because nh3-dev's installer cdrom line has a pre-existing verify error. The first run caught that and aborted without writing. servers/nh3-pve/README.md now lists each guest's expected state after a power loss: nh3-laser is on-demand and stays off (Prime), pbs-nh3 now has onboot=1 (it had none and stayed down), and pfi-gx10 is bare metal that does not auto-power-on.
nh3-pve
Proxmox VE hypervisor for the NH3 site (nh3-vmhost.phasefinal.com).
Network
- LAN IP: 10.100.250.60
- FQDN:
nh3-vmhost.phasefinal.com - Web UI:
https://10.100.250.60:8006 - SSH:
root@nh3-pve(key auth)
Hardware
- CPU: 13th Gen Intel Core i9-13900H
- RAM: 62.5 GB
- Kernel:
6.8.12-11-pve(Proxmox 8.x) - Storage: mostly networked —
/mnt/pve/pfi-nh3-nas(42 TB) mounted from the Synology at10.100.50.50:/volume1/VMStorage; ~27 TB used
What it runs
Guests as of 2026-09-24 (qm list / pct list is the live truth). Expected state
after a power loss is the column that matters in a recovery: a guest listed as
stays off is not a casualty.
| ID | guest | kind | onboot | expected after power loss |
|---|---|---|---|---|
| 100 | nh3-docker (10.100.50.40) |
VM | 1 | up |
| 101 | nh3-extdev (10.100.50.42) |
VM | 1 | up |
| 102 | nh3-dev (10.100.10.50) |
VM | 1 | up |
| 104 | nh3-laser | VM | — | stays off — on-demand (Prime, 2026-09-24) |
| 105 | pbs-nh3 (10.100.50.90) |
VM | 1 | up (onboot added 2026-09-24; it had none and stayed down after the outage) |
| 108 | opnsense-lab | VM | 0 | stays off (lab) |
| 103 | nh3-wg | CT | 1 | up |
| 106 | nh3-headscale | CT | 1 | up |
| 107 | nh3-scale | CT | 1 | up (mesh subnet router + fleet egress proxy) |
Power-loss recovery (2026-09-24 outage). Every guest boots at once, and
nh3-nas is the slowest to serve NFS. NFS clients now mount nh3-nas shares on
first access (playbooks/nh3-nfs-automount.yaml, applied to nh3-docker and
nh3-dev), so a late NAS no longer leaves shares down. pbs-nh3's datastore line
uses bg (background retry) instead. qm guest exec 105 works (the guest agent is
on), which is the root path into pbs-nh3; infra-ops is not provisioned there.
pfi-gx10 (10.100.50.60) is bare metal on this site, not a guest here, and did
not power back on by itself.
Refresh state
scripts/refresh-server-info.sh nh3-pve
Same Proxmox-inspect caveat as pfi-pve: snapshots capture host-level detail only. VM/LXC/storage enumeration needs the Proxmox tooling manually.
Placement rule
Hypervisor. New NH-site workloads land here as VMs.