vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
9b71f6c2f38a8efff990643583a628737d569c30
esh-pfi-infrastructure/docs/pfi/proxmox-inventory-2026-10-02.md
T

8.8 KiB
Raw Blame History

Fleet Proxmox host inventory — 2026-10-02 (snapshot)

Requested by Prime via Miranda for PVE 8→9 planning. A point-in-time read; re-read live before acting. ESH cluster detail: docs/runbooks/esh-pve-cluster-pve9-upgrade-plan.md.

FLEET PROXMOX HOST INVENTORY — all sites. Captured 2026-10-02 ~19:45 PT, live (ssh + pvesh + guest agent/pct exec), read-only.

SITE SUMMARY

Site Proxmox hosts PVE major (exact) Notes
ANA (Anaheim colo) pfi-pve — read live 8 (8.3.5) Dell PowerEdge R750xs, standalone. sfsrv-ana (SureFire client's PVE host) NOT read — see below
NH3 nh3-pve (node name nh3-vmhost) — read live 8 (8.4.1) MS-01, standalone. nh3-pve-2 (new MS-03) is mid-install with the 9.2 ISO, not on the network yet
FV (Fountain Valley) none n/a fv-ml1 is bare-metal Debian 13 (GPU host), not Proxmox
IRV (Irvine) none n/a irv-ml1 is bare-metal Debian 12 (GPU host), not Proxmox
ESH (home lab) pve + esh-nas-pve, 2-node cluster esh-pve-cluster 8 (8.4.20) See the earlier ESH block (10 guests). ⚠ its 175 T tank pool is DEGRADED

HOST: pfi-pve (ANA) — 10.250.250.31

  • Hardware: Dell PowerEdge R750xs, Intel Xeon Silver 4310 (48 threads), 188 GiB RAM. Uptime 3 weeks.
  • PVE 8.3.5 (needs 8.4.x before any 9 upgrade), kernel 6.8.12-8, Debian 12. Standalone (no cluster).
  • Root: ext4 on LVM (31G free). Storage: ospool (ZFS 10.9T, 20% used — most guest disks), NASPool (ZFS 32.7T, as dir 'naspool-vmstorage', 1.5% used), local-lvm (lvmthin 130G), pbs-ana. Both zpools ONLINE.
  • Backups: vzdump 03:00 daily, ALL except 100/109/114, snapshot mode -> pbs-ana (keep last 5 + 1 daily/weekly/monthly/yearly); separate 22:00 job for CT 109 only.
  • Guests: 14 guests (14 running), 117 vCPU / 106.508 GiB RAM allocated
    VMID Type Name Status vCPU RAM Disk OS (live) Role PBS
    100 VM pbs-ana running 4 8G scsi0 32G local-lvm(lvmthin) Debian GNU/Linux 12 (bookworm) PBS (pbs-ana) — backup target for ALL sites NONE
    101 VM PFI-ANA-DC running 12 24G scsi0 240G ospool(zfspool) Windows Server 2022 Datacenter Windows AD domain controller 8 backups, newest 2026-10-02
    102 VM PFI-ANA-Docker running 8 16G scsi0 250G ospool(zfspool) Debian GNU/Linux 12 (bookworm) ana-docker: LiteLLM gateway, traefik, gitea, vaultwarden, Beszel/Dozzle hubs, RustDesk 8 backups, newest 2026-10-02
    103 VM PFI-SlaveBot running 8 8G ide0 256G ospool(zfspool) Windows 10 IoT Enterprise LTSC 2021 Windows 10 IoT utility VM 8 backups, newest 2026-10-02
    104 VM PFI-Mongo running 8 8G scsi0 256G ospool(zfspool) Debian GNU/Linux 12 (bookworm) MongoDB 8 backups, newest 2026-10-02
    105 VM PFI-Postgres running 16 8G scsi0 80G ospool(zfspool) Debian GNU/Linux 12 (bookworm) shared Postgres (vaultwarden/gitea/paperless) 8 backups, newest 2026-10-02
    106 VM corviduo-dev running 8 8G scsi0 80G ospool(zfspool) Debian GNU/Linux 13 (trixie) Worldtree dev VM (demo/personal/pinned deployments) 8 backups, newest 2026-10-02
    107 VM PFI-Pteradactyl running 8 8G scsi0 256G ospool(zfspool) Debian GNU/Linux 12 (bookworm) Pterodactyl game panel 8 backups, newest 2026-10-02
    109 CT ana-nas running 4 2G rootfs 80G ospool(zfspool) Debian GNU/Linux 12 (bookworm) ana-nas: NFS/SMB storage server (storage SPOF) 28 backups, newest 2026-10-02
    110 VM PFI-ANA-Webhost running 16 4G scsi0 250G ospool(zfspool) Debian GNU/Linux 11 (bullseye) web host (Debian 11) 8 backups, newest 2026-10-02
    111 VM pfi-tacticalrmm running 16 8G scsi0 256G ospool(zfspool) Debian GNU/Linux 12 (bookworm) TacticalRMM + MeshCentral 8 backups, newest 2026-10-02
    112 CT ana-filebot running 4 2G rootfs 80G ospool(zfspool) Debian GNU/Linux 12 (bookworm) file-task automation 8 backups, newest 2026-10-02
    113 CT ana-wg running 4 2G rootfs 8G ospool(zfspool) Debian GNU/Linux 12 (bookworm) WireGuard 8 backups, newest 2026-10-02
    114 CT ana-scale running 1 0.5G rootfs 8G ospool(zfspool) Debian GNU/Linux 12 (bookworm) ANA headscale subnet router (mesh) 1 backup, newest 2026-09-06

HOST: nh3-pve (NH3) — 10.100.250.60 (Proxmox node name nh3-vmhost)

  • Hardware: Minisforum MS-01, i9-13900H (20 threads), 62 GiB RAM, RTX 2000E Ada (for CT 109). Uptime 1 week. Intel AMT phones home to MeshCentral.
  • PVE 8.4.1, kernel 6.8.12-43, Debian 12. Standalone (no cluster).
  • Root: ZFS rpool (952G, 667G free). Storage: local-zfs (912G, 27% used — all guest disks), pfi-nh3-nas (NFS, 42.9T, 77% used), pbs-ana.
  • Backups: vzdump 21:00 daily, ALL except 107/109, snapshot mode -> pbs-ana (same retention); pbs-nh3 (VM 105) syncs pbs-ana as the DR mirror.
  • Guests: 10 guests (8 running), 58 vCPU / 83 GiB RAM allocated
    VMID Type Name Status vCPU RAM Disk OS (live) Role PBS
    100 VM nh3-docker1 running 8 8G scsi0 128G local-zfs(zfspool) Debian GNU/Linux 12 (bookworm) nh3-docker: althing post office, SearXNG, AdGuard NH3, albok-service 8 backups, newest 2026-10-02
    101 VM nh3-extdev running 8 8G scsi0 256G local-zfs(zfspool) Debian GNU/Linux 13 (trixie) manager / external-dev box 8 backups, newest 2026-10-02
    102 VM nh3-dev running 16 28G scsi0 378G local-zfs(zfspool) Debian GNU/Linux 12 (bookworm) nh3-dev: live Claude sessions, svos/hermes (Miranda channel), Booth, fleet TLS caddy 8 backups, newest 2026-10-02
    103 CT nh3-wg running 4 2G rootfs 40G local-zfs(zfspool) Ubuntu 22.04.5 LTS nh3-wg (WireGuard) 8 backups, newest 2026-10-02
    104 VM nh3-laser stopped 8 8G ide0 120G local-zfs(zfspool) win11 Windows 11 (laser) — stopped 8 backups, newest 2026-10-02
    105 VM pbs-nh3 running 4 8G scsi0 32G local-zfs(zfspool) Debian GNU/Linux 12 (bookworm) PBS DR mirror (pbs-nh3) 8 backups, newest 2026-10-02
    106 CT nh3-headscale running 1 0.5G rootfs 8G local-zfs(zfspool) Debian GNU/Linux 12 (bookworm) headscale control server (mesh) 7 backups, newest 2026-10-02
    107 CT nh3-scale running 1 0.5G rootfs 8G local-zfs(zfspool) Debian GNU/Linux 12 (bookworm) NH3 headscale subnet router (mesh) NONE
    108 VM opnsense-lab stopped 2 4G scsi0 32G local-zfs(zfspool) other OPNsense lab — stopped 7 backups, newest 2026-10-02
    109 CT nh3-ml1 running 6 16G rootfs 80G local-zfs(zfspool) Debian GNU/Linux 12 (bookworm) GPU CT: TEI embed/rerank (twin of esh-ml1) NONE

HOST: nh3-pve-2 (NH3) — new Minisforum MS-03 (Core Ultra 9 Panther Lake, 64 GB). PVE 9.2 installed from the ISO but onto the wrong NVMe; reinstall planned on site 2026-10-03. Host not on the network yet; its AMT (21.0.6) is at 10.100.250.63 and phones home to MeshCentral. No guests.

HOST: sfsrv-ana (ANA) — 10.250.250.115, SureFire client's Proxmox host (client-owned Dell R630, PFI-managed). NOT READ: our standing rule is to coordinate before touching the SureFire hosts, so version, guests and backup posture are unknown here. Fleet docs record backup coverage there as 'status unknown'. A read-only pveversion + guest list needs Prime's OK.

BACKUP POSTURE ACROSS SITES (vzdump -> pbs-ana at Anaheim; pbs-nh3 mirrors it)

  • Every guest is backed up nightly EXCEPT: pfi-pve 100 (pbs-ana itself — expected), 114 ana-scale (excluded; one stale backup from 2026-09-06); nh3-pve 107 nh3-scale and 109 nh3-ml1 (excluded); ESH 108 esh-scale and 110 esh-ml1 (no job covers them).
  • Pattern: all three headscale subnet routers (ana-scale 114, nh3-scale 107, esh-scale 108) are excluded. Likely deliberate after a vzdump lock on esh-scale once blackholed ESH, but none of them has a current restore point. Both GPU/TEI containers (nh3-ml1 109, esh-ml1 110) are also uncovered.
  • pfi-pve CT 109 (ana-nas, the Anaheim storage SPOF) has its own 22:00 job (28 restore points). Note that an older standing rule (2026-04) said not to vzdump CT 109; the dedicated job postdates it.

PVE 9 READINESS, PER HOST

  • pfi-pve: 8.3.5 -> latest 8.4 first. Hosts pbs-ana (every site's backup target), the LiteLLM gateway (ana-docker), TacticalRMM, shared Postgres and the ana-nas storage SPOF: a reboot is a fleet-wide event. Standalone, so no quorum concerns.
  • nh3-pve: 8.4.1 -> latest 8.4 first. Hosts nh3-dev (all agent sessions, Miranda's svos/hermes channel), nh3-docker (althing post office, albok, AdGuard NH3), headscale + the NH3 mesh router, pbs-nh3: a reboot takes down the fleet bus and Miranda's channel. Its AMT now phones home (remote recovery exists).
  • nh3-pve-2: already 9 (fresh install).
  • ESH cluster: 8.4.20; plan delivered separately (pool repair first).
Reference in New Issue View Git Blame Copy Permalink