ops(pfi-pve): upgraded to PVE 9.2.21 / kernel 7.0.14-20; reboot test + post-check recorded; pre-reboot checks corrected for PVE 9

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — eshpfi-management
 
-_Last updated: 2026-10-04 ~0546 PT (albok-service 0.1.6 live after Prime's push go; esh-nas-pve + nh3-pve on PVE 9; QNAP LCD fixed, awaiting Prime's look; NH3 Mac mini asleep, fixed on site today.)_
+_Last updated: 2026-10-04 ~0720 PT (pfi-pve on PVE 9; albok-service 0.1.6 live after Prime's push go; esh-nas-pve + nh3-pve on PVE 9; QNAP LCD fixed, awaiting Prime's look; NH3 Mac mini asleep, fixed on site today.)_
 
 > **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
 > `Written:` stamp is under **8 hours** old, read it (it carries the in-flight
@@ -119,6 +119,7 @@ _As of 2026-10-04 ~0546 PT. The newest subsection is first; older subsections ca
 
 ### 2026-10-03/04: live now (as of 2026-10-04 ~0546 PT)
 
+- **pfi-pve (Anaheim): DONE, PVE 9.2.21 / kernel 7.0.14-20 since 0656 2026-10-04** (Prime: "drive it", "proceed"). Prep fixed a real boot trap: the box boots UEFI but had `grub-pc`, so the ESP GRUB was the 2023 install's; now `grub-efi-amd64`. NIC pins, microcode, a PVE 8 reboot test (passed), then trixie. All 14 guests + services verified 0715. ⚠ Each pfi-pve reboot = ~4 min of NH3/ESH↔ANA (ana-scale) and **~20 min of LiteLLM, which also silences Miranda**. Break-glass = FortiGate WAN SSH `infra-ops@38.120.12.42` (verified). ⚠ ifupdown2 3.3 `ifreload -a -s` exits 1 on PVE's own `bridge-fd 0` — cosmetic (same on nh3-pve/esh-nas-pve). **Open: `lvremove -y pve/root-pre-pve9` on/after 2026-10-05** (asked infra-hermes); no `zpool upgrade`. → `docs/runbooks/pfi-pve-pve9-upgrade-plan.md`
 - **nh3-pve: DONE. ON PVE 9.2.21 / kernel 7.0.14-20 (booted 1402).** Post-check: every guest is back (the stopped 104/108 are as before), ZFS healthy, DNS, post office, albok, svos/hermes, mesh and AMT all OK. ⚠ **nh3-ml1 (CT 109) is DOWN:** no `proxmox-headers-7.0` was installed, so the nvidia 580.178.04 DKMS module was never built for 7.0, `nvidia-persistenced` failed, and CT 109 has no /dev/nvidia*. **FIXED 1409 (Prime go, `playbooks/nh3-pve-pve9-postfix.yaml`):** installed `proxmox-default-headers` + headers-7.0.14-20, nvidia DKMS built for 7.0 (580.178.04 compiles fine), nh3-ml1 back (TEI :8001/:8013 healthy, embed dim 1024); intel-microcode installed (non-free-firmware added; takes effect next boot); `pve-enterprise.sources` off; pve8to9 now 0 FAIL / 1 WARN (dkms, benign). VM 102 snapshot `pre-rootgrow-20261002` DELETED (Prime go). No `pre-pve9` snapshot had been taken. **POST-CHECK DONE: nh3-pve clean.** ⚠ Lesson: on a PVE host with DKMS, check that `proxmox-default-headers` is installed BEFORE a major upgrade, or the new kernel boots with no module.
 - **nh3-pve-2** (MS-03, NH3) is live at 10.100.250.62 on nh3-mgmt:
   - X710 `nic3` on USW port 25;