vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(nh3): NFS clients mount nh3-nas shares on first access; record guest power-loss expectations

Browse Source 
After the 2026-09-24 NH3 power outage every NFS line from nh3-nas failed at
boot: nh3-docker's /mnt/compose and /mnt/backup, and nh3-dev's /mnt/backup.
The NAS is the slowest box to serve NFS, and the plain fstab mounts stayed
down until remounted by hand. nh3-dev's /mnt/smithy already used
x-systemd.automount, and the same boot shows it self-healing on its next
access. playbooks/nh3-nfs-automount.yaml brings the other lines to that
shape (_netdev,nofail,x-systemd.automount,x-systemd.mount-timeout=30).
Hard-mount semantics are unchanged; only WHEN the mount happens moves.

Applied and read back on nh3-docker and nh3-dev: every nh3-nas line carries
automount, each automount unit is active, and each share is NFS-mounted after
access. A second run on nh3-docker is a no-op. The fstab check judges only
errors the rewrite ADDS, because nh3-dev's installer cdrom line has a
pre-existing verify error. The first run caught that and aborted without
writing.

servers/nh3-pve/README.md now lists each guest's expected state after a
power loss: nh3-laser is on-demand and stays off (Prime), pbs-nh3 now has
onboot=1 (it had none and stayed down), and pfi-gx10 is bare metal that
does not auto-power-on.
This commit is contained in:
vh
2026-09-24 13:16:18 -07:00
parent ad2b4d9df8
commit 1cbde502f4
2 changed files with 97 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
servers/nh3-pve/README.md
+24 -1
View File
@@ -18,7 +18,30 @@ Proxmox VE hypervisor for the NH3 site (`nh3-vmhost.phasefinal.com`).
## What it runs
`nh3-docker` (`10.100.50.40`) is a VM on this hypervisor. Other VMs (if any) aren't catalogued yet — run `qm list` on the host when you need the current inventory.
Guests as of 2026-09-24 (`qm list` / `pct list` is the live truth). **Expected state
after a power loss** is the column that matters in a recovery: a guest listed as
`stays off` is not a casualty.
| ID | guest | kind | onboot | expected after power loss |
|---|---|---|---|---|
| 100 | nh3-docker (`10.100.50.40`) | VM | 1 | up |
| 101 | nh3-extdev (`10.100.50.42`) | VM | 1 | up |
| 102 | nh3-dev (`10.100.10.50`) | VM | 1 | up |
| 104 | nh3-laser | VM | — | **stays off — on-demand** (Prime, 2026-09-24) |
| 105 | pbs-nh3 (`10.100.50.90`) | VM | 1 | up (onboot added 2026-09-24; it had none and stayed down after the outage) |
| 108 | opnsense-lab | VM | 0 | stays off (lab) |
| 103 | nh3-wg | CT | 1 | up |
| 106 | nh3-headscale | CT | 1 | up |
| 107 | nh3-scale | CT | 1 | up (mesh subnet router + fleet egress proxy) |
**Power-loss recovery (2026-09-24 outage).** Every guest boots at once, and
nh3-nas is the slowest to serve NFS. NFS clients now mount nh3-nas shares on
first access (`playbooks/nh3-nfs-automount.yaml`, applied to nh3-docker and
nh3-dev), so a late NAS no longer leaves shares down. pbs-nh3's datastore line
uses `bg` (background retry) instead. `qm guest exec 105` works (the guest agent is
on), which is the root path into pbs-nh3; infra-ops is not provisioned there.
**pfi-gx10 (`10.100.50.60`) is bare metal on this site, not a guest here**, and did
not power back on by itself.
## Refresh state