scripts: proxmox_inspect.sh for VM/LXC/storage/backup discovery
Read-only companion to server_inspect.sh. Runs over ssh via stdin the
same way:
ssh root@pfi-pve 'bash -s' < scripts/proxmox_inspect.sh \\
> servers/pfi-pve/proxmox-details.txt
Captures what the generic inspect misses on PVE nodes:
- Cluster + node resources (pvesh get /cluster/resources)
- VM list + per-vmid config highlights (qm list, qm config)
- LXC list + per-ctid config highlights (pct list, pct config)
- Storage pools (pvesm status, zpool, storage.cfg, NFS mounts)
- Backup jobs table parsed from /etc/pve/jobs.cfg (no jq needed)
- Per-guest backup coverage verdict: YES/NO with reason
- Most-recent actual backup per VMID on each local dump storage
- PBS targets, replication jobs, listening ports, subscription status
Coverage analysis surfaces the real gap: on pfi-pve, VMIDs 106, 107,
109, 112, 113 have no scheduled backup; VMID 106 (PFI-Tailscale) has
not been backed up since 2024-05-31. nh3-pve uses "all" job and covers
everything. esh-pve + esh-pve-nas share the tank-vmbu dump storage via
NFS; coverage splits cleanly across the two nodes.
First snapshots committed for each PVE host.
This commit is contained in:
@@ -0,0 +1,273 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: nh3-vmhost.phasefinal.com
|
||||
Date: 2026-04-20T22:12:44-07:00
|
||||
Uptime: up 5 weeks, 5 days, 15 hours, 1 minute
|
||||
PVE: pve-manager/8.4.1/2a5fa54a8503f96d (running kernel: 6.8.12-11-pve)
|
||||
Kernel: 6.8.12-11-pve
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU model: 13th Gen Intel(R) Core(TM) i9-13900H
|
||||
CPU cores: 20
|
||||
MemTotal: 62.5 GB
|
||||
MemAvail: 39.7 GB
|
||||
|
||||
===== CLUSTER =====
|
||||
|
||||
(single-node host (or pvecm returned no cluster info))
|
||||
|
||||
===== CLUSTER RESOURCES (pvesh) =====
|
||||
|
||||
┌────────────────────────────────┬─────────┬─────────────┬──────────────────────────────────┬────────┬────────────┬────────────┬────────────┬─────────┬───────┬──────┬────────┬────────────┬───────────┬────────────┬─────────────┬───────────┬───────────┬────────────┬────────────┬──────┬───────────┬─────────────┬──────┬──────────┬───────────┬──────┐
|
||||
│ id │ type │ cgroup-mode │ content │ cpu │ disk │ diskread │ diskwrite │ hastate │ level │ lock │ maxcpu │ maxdisk │ maxmem │ mem │ name │ netin │ netout │ node │ plugintype │ pool │ status │ storage │ tags │ template │ uptime │ vmid │
|
||||
╞════════════════════════════════╪═════════╪═════════════╪══════════════════════════════════╪════════╪════════════╪════════════╪════════════╪═════════╪═══════╪══════╪════════╪════════════╪═══════════╪════════════╪═════════════╪═══════════╪═══════════╪════════════╪════════════╪══════╪═══════════╪═════════════╪══════╪══════════╪═══════════╪══════╡
|
||||
│ lxc/103 │ lxc │ │ │ 0.00% │ 830.62 MiB │ 464.69 MiB │ 0.00 B │ │ │ │ 4 │ 40.00 GiB │ 2.00 GiB │ 56.72 MiB │ nh3-wg │ 1.58 GiB │ 5.93 MiB │ nh3-vmhost │ │ │ running │ │ │ 0 │ 5w 5d 15h │ 103 │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ node/nh3-vmhost │ node │ 2 │ │ 9.16% │ 6.16 GiB │ │ │ │ p │ │ 20 │ 855.73 GiB │ 62.53 GiB │ 22.82 GiB │ │ │ │ nh3-vmhost │ │ │ online │ │ │ │ 5w 5d 15h │ │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ qemu/100 │ qemu │ │ │ 0.56% │ 0.00 B │ 961.29 MiB │ 15.22 GiB │ │ │ │ 8 │ 128.00 GiB │ 8.00 GiB │ 3.98 GiB │ nh3-docker1 │ 3.46 GiB │ 4.95 GiB │ nh3-vmhost │ │ │ running │ │ │ 0 │ 5w 5d 15h │ 100 │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ qemu/101 │ qemu │ │ │ 0.02% │ 0.00 B │ 180.83 MiB │ 1.31 GiB │ │ │ │ 8 │ 256.00 GiB │ 2.00 GiB │ 750.68 MiB │ nh3-ansible │ 1.18 GiB │ 3.45 MiB │ nh3-vmhost │ │ │ running │ │ │ 0 │ 5w 5d 15h │ 101 │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ qemu/102 │ qemu │ │ │ 22.25% │ 0.00 B │ 41.57 GiB │ 169.22 GiB │ │ │ │ 8 │ 250.00 GiB │ 8.00 GiB │ 4.79 GiB │ nh3-dev │ 20.55 GiB │ 29.55 GiB │ nh3-vmhost │ │ │ running │ │ │ 0 │ 2w 3d 11h │ 102 │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ qemu/104 │ qemu │ │ │ 0.00% │ 0.00 B │ 0.00 B │ 0.00 B │ │ │ │ 8 │ 120.00 GiB │ 8.00 GiB │ 0.00 B │ nh3-laser │ 0.00 B │ 0.00 B │ nh3-vmhost │ │ │ stopped │ │ │ 0 │ 0s │ 104 │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ sdn/nh3-vmhost/localnetwork │ sdn │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ nh3-vmhost │ │ │ ok │ │ │ │ │ │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ storage/nh3-vmhost/local │ storage │ │ backup,iso,vztmpl │ │ 863.62 MiB │ │ │ │ │ │ │ 850.41 GiB │ │ │ │ │ │ nh3-vmhost │ dir │ │ available │ local │ │ │ │ │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ storage/nh3-vmhost/local-zfs │ storage │ │ images,rootdir │ │ 65.85 GiB │ │ │ │ │ │ │ 915.42 GiB │ │ │ │ │ │ nh3-vmhost │ zfspool │ │ available │ local-zfs │ │ │ │ │
|
||||
├────────────────────────────────┼─────────┼─────────────┼──────────────────────────────────┼────────┼────────────┼────────────┼────────────┼─────────┼───────┼──────┼────────┼────────────┼───────────┼────────────┼─────────────┼───────────┼───────────┼────────────┼────────────┼──────┼───────────┼─────────────┼──────┼──────────┼───────────┼──────┤
|
||||
│ storage/nh3-vmhost/pfi-nh3-nas │ storage │ │ images,vztmpl,rootdir,backup,iso │ │ 26.44 TiB │ │ │ │ │ │ │ 41.87 TiB │ │ │ │ │ │ nh3-vmhost │ nfs │ │ available │ pfi-nh3-nas │ │ │ │ │
|
||||
└────────────────────────────────┴─────────┴─────────────┴──────────────────────────────────┴────────┴────────────┴────────────┴────────────┴─────────┴───────┴──────┴────────┴────────────┴───────────┴────────────┴─────────────┴───────────┴───────────┴────────────┴────────────┴──────┴───────────┴─────────────┴──────┴──────────┴───────────┴──────┘
|
||||
|
||||
===== VMs (qm list) =====
|
||||
|
||||
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
|
||||
100 nh3-docker1 running 8192 128.00 1512
|
||||
101 nh3-ansible running 2048 256.00 1741
|
||||
102 nh3-dev running 8192 250.00 3912921
|
||||
104 nh3-laser stopped 8192 120.00 0
|
||||
|
||||
===== VM CONFIGS (qm config per id) =====
|
||||
|
||||
|
||||
----- VMID 100 -----
|
||||
agent: 1
|
||||
boot: order=scsi0;ide2;net0
|
||||
cores: 4
|
||||
ide2: none,media=cdrom
|
||||
memory: 8192
|
||||
name: nh3-docker1
|
||||
net0: virtio=BC:24:11:86:C7:F3,bridge=vmbr0,firewall=1,tag=50
|
||||
onboot: 1
|
||||
ostype: l26
|
||||
scsi0: local-zfs:vm-100-disk-0,iothread=1,size=128G
|
||||
|
||||
----- VMID 101 -----
|
||||
agent: 1
|
||||
boot: order=scsi0;ide2;net0
|
||||
cores: 4
|
||||
ide2: local:iso/debian-12.5.0-amd64-netinst.iso,media=cdrom,size=629M
|
||||
memory: 2048
|
||||
name: nh3-ansible
|
||||
net0: virtio=BC:24:11:27:A1:73,bridge=vmbr0,firewall=1,tag=50
|
||||
onboot: 1
|
||||
ostype: l26
|
||||
scsi0: local-zfs:vm-101-disk-0,iothread=1,size=256G
|
||||
|
||||
----- VMID 102 -----
|
||||
agent: 1
|
||||
boot: order=scsi0;ide2;net0
|
||||
cores: 4
|
||||
ide2: local:iso/debian-12.5.0-amd64-netinst.iso,media=cdrom,size=629M
|
||||
memory: 8192
|
||||
name: nh3-dev
|
||||
net0: virtio=BC:24:11:59:87:13,bridge=vmbr0,firewall=1,tag=10
|
||||
onboot: 1
|
||||
ostype: l26
|
||||
scsi0: local-zfs:vm-102-disk-0,iothread=1,size=250G
|
||||
|
||||
----- VMID 104 -----
|
||||
agent: 1,fstrim_cloned_disks=1
|
||||
boot: order=ide0;ide2;net0
|
||||
cores: 4
|
||||
ide0: local-zfs:vm-104-disk-1,size=120G
|
||||
ide2: pfi-nh3-nas:iso/virtio-win-0.1.285.iso,media=cdrom,size=771138K
|
||||
memory: 8192
|
||||
name: nh3-laser
|
||||
net0: virtio=BC:24:11:A0:14:DF,bridge=vmbr0,tag=50
|
||||
ostype: win11
|
||||
|
||||
===== LXC CONTAINERS (pct list) =====
|
||||
|
||||
VMID Status Lock Name
|
||||
103 running nh3-wg
|
||||
|
||||
===== LXC CONFIGS (pct config per id) =====
|
||||
|
||||
|
||||
----- CTID 103 -----
|
||||
cores: 4
|
||||
features: nesting=1
|
||||
hostname: nh3-wg
|
||||
memory: 2048
|
||||
net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:F5:90:F2,ip=dhcp,tag=50,type=veth
|
||||
onboot: 1
|
||||
ostype: ubuntu
|
||||
rootfs: local-zfs:subvol-103-disk-0,size=40G
|
||||
swap: 2048
|
||||
unprivileged: 1
|
||||
|
||||
===== STORAGE (pvesm status) =====
|
||||
|
||||
Name Type Status Total Used Available %
|
||||
local dir active 891722496 884352 890838144 0.10%
|
||||
local-zfs zfspool active 959889604 69051432 890838172 7.19%
|
||||
pfi-nh3-nas nfs active 44962267904 28385127296 16577140608 63.13%
|
||||
|
||||
===== DATASTORE CONFIG (/etc/pve/storage.cfg) =====
|
||||
|
||||
dir: local
|
||||
path /var/lib/vz
|
||||
content iso,backup,vztmpl
|
||||
|
||||
zfspool: local-zfs
|
||||
pool rpool/data
|
||||
content images,rootdir
|
||||
sparse 1
|
||||
|
||||
nfs: pfi-nh3-nas
|
||||
export /volume1/VMStorage
|
||||
path /mnt/pve/pfi-nh3-nas
|
||||
server 10.100.50.50
|
||||
content backup,vztmpl,rootdir,images,iso
|
||||
options vers=3
|
||||
prune-backups keep-all=1
|
||||
|
||||
|
||||
===== ZFS POOLS (zpool list -v) =====
|
||||
|
||||
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
|
||||
rpool 952G 72.9G 879G - - 6% 7% 1.00x ONLINE -
|
||||
nvme-eui.00000000000000000026b7382d4a82a5-part3 953G 72.9G 879G - - 6% 7.66% - ONLINE
|
||||
|
||||
===== FILESYSTEMS (df -h, local fs only) =====
|
||||
|
||||
Filesystem Type Size Used Avail Use% Mounted on
|
||||
rpool/ROOT/pve-1 zfs 856G 6.2G 850G 1% /
|
||||
efivarfs efivarfs 192K 127K 61K 68% /sys/firmware/efi/efivars
|
||||
rpool/var-lib-vz zfs 851G 864M 850G 1% /var/lib/vz
|
||||
rpool zfs 850G 128K 850G 1% /rpool
|
||||
rpool/ROOT zfs 850G 128K 850G 1% /rpool/ROOT
|
||||
rpool/data zfs 850G 128K 850G 1% /rpool/data
|
||||
rpool/data/subvol-103-disk-0 zfs 40G 831M 40G 3% /rpool/data/subvol-103-disk-0
|
||||
/dev/fuse fuse 128M 20K 128M 1% /etc/pve
|
||||
10.100.50.50:/volume1/VMStorage nfs 42T 27T 16T 64% /mnt/pve/pfi-nh3-nas
|
||||
|
||||
===== MOUNTED NFS / CIFS =====
|
||||
|
||||
10.100.50.50:/volume1/VMStorage /mnt/pve/pfi-nh3-nas nfs rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.100.50.50,mountvers=3,mountport=892,mountproto=udp,local_lock=none,addr=10.100.50.50 0 0
|
||||
|
||||
===== BACKUP JOBS (/etc/pve/jobs.cfg + /etc/vzdump.conf) =====
|
||||
|
||||
vzdump: backup-5d8f1221-8f71
|
||||
schedule 21:00
|
||||
all 1
|
||||
compress zstd
|
||||
enabled 1
|
||||
fleecing 0
|
||||
mailnotification always
|
||||
mode snapshot
|
||||
notes-template {{guestname}}
|
||||
prune-backups keep-daily=1,keep-last=5,keep-monthly=1,keep-weekly=1,keep-yearly=1
|
||||
storage pfi-nh3-nas
|
||||
|
||||
|
||||
----- vzdump defaults -----
|
||||
|
||||
===== BACKUP COVERAGE ANALYSIS =====
|
||||
|
||||
JOB-ID SCHEDULE ALL ENABLED VMID-LIST EXCLUDE STORAGE
|
||||
backup-5d8f1221-8f71 21:00 1 1 - - pfi-nh3-nas
|
||||
|
||||
----- per-guest coverage -----
|
||||
ID VERDICT REASON
|
||||
100 YES covered by 'all' job
|
||||
101 YES covered by 'all' job
|
||||
102 YES covered by 'all' job
|
||||
104 YES covered by 'all' job
|
||||
103 YES covered by 'all' job
|
||||
|
||||
===== RECENT BACKUPS ON LOCAL DUMP STORAGES =====
|
||||
|
||||
|
||||
----- /var/lib/vz/dump/ -----
|
||||
|
||||
----- /mnt/pve/pfi-nh3-nas/dump/ -----
|
||||
VMID 100 2026-04-20 21:01 vzdump-qemu-100-2026_04_20-21_00_02.vma.zst
|
||||
VMID 101 2026-04-20 21:02 vzdump-qemu-101-2026_04_20-21_01_07.vma.zst
|
||||
VMID 102 2026-04-20 21:09 vzdump-qemu-102-2026_04_20-21_02_29.vma.zst
|
||||
VMID 103 2026-04-20 21:09 vzdump-lxc-103-2026_04_20-21_09_23.tar.zst
|
||||
VMID 104 2026-04-20 21:16 vzdump-qemu-104-2026_04_20-21_09_40.vma.zst
|
||||
|
||||
===== PBS TARGETS =====
|
||||
|
||||
(no PBS storage configured on this node)
|
||||
|
||||
===== REPLICATION JOBS (pvesr status) =====
|
||||
|
||||
JobID Enabled Target LastSync NextSync Duration FailCount State
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
0.0.0.0:37105
|
||||
0.0.0.0:59691
|
||||
[::]:111
|
||||
[::1]:25
|
||||
127.0.0.1:25
|
||||
127.0.0.1:85
|
||||
[::]:22
|
||||
*:3128
|
||||
[::]:43035
|
||||
[::]:45287
|
||||
*:8006
|
||||
|
||||
===== SUBSCRIPTION / UPDATES =====
|
||||
|
||||
key: pve8p-1145141919
|
||||
productname: YajuuSenpai
|
||||
status: active
|
||||
|
||||
----- apt upgradable (top 20) -----
|
||||
base-files/oldstable 12.4+deb12u13 amd64 [upgradable from: 12.4+deb12u11]
|
||||
bash/oldstable 5.2.15-2+b10 amd64 [upgradable from: 5.2.15-2+b8]
|
||||
bind9-dnsutils/oldstable-security 1:9.18.47-1~deb12u1 amd64 [upgradable from: 1:9.18.33-1~deb12u2]
|
||||
bind9-host/oldstable-security 1:9.18.47-1~deb12u1 amd64 [upgradable from: 1:9.18.33-1~deb12u2]
|
||||
bind9-libs/oldstable-security 1:9.18.47-1~deb12u1 amd64 [upgradable from: 1:9.18.33-1~deb12u2]
|
||||
btrfs-progs/oldstable 6.2-1+deb12u2 amd64 [upgradable from: 6.2-1+deb12u1]
|
||||
busybox/oldstable 1:1.35.0-4+b7 amd64 [upgradable from: 1:1.35.0-4+b4]
|
||||
corosync/stable 3.1.10-pve2~bpo12+1 amd64 [upgradable from: 3.1.9-pve1]
|
||||
criu/oldstable 3.17.1-2+deb12u2 amd64 [upgradable from: 3.17.1-2+deb12u1]
|
||||
curl/oldstable 7.88.1-10+deb12u14 amd64 [upgradable from: 7.88.1-10+deb12u12]
|
||||
dirmngr/oldstable 2.2.40-1.1+deb12u2 amd64 [upgradable from: 2.2.40-1.1]
|
||||
distro-info-data/oldstable 0.58+deb12u6 all [upgradable from: 0.58+deb12u4]
|
||||
e2fsprogs/oldstable 1.47.0-2+b2 amd64 [upgradable from: 1.47.0-2]
|
||||
gnupg-l10n/oldstable 2.2.40-1.1+deb12u2 all [upgradable from: 2.2.40-1.1]
|
||||
gnupg-utils/oldstable 2.2.40-1.1+deb12u2 amd64 [upgradable from: 2.2.40-1.1]
|
||||
gnupg/oldstable 2.2.40-1.1+deb12u2 all [upgradable from: 2.2.40-1.1]
|
||||
gnutls-bin/oldstable-security 3.7.9-2+deb12u6 amd64 [upgradable from: 3.7.9-2+deb12u4]
|
||||
gpg-agent/oldstable 2.2.40-1.1+deb12u2 amd64 [upgradable from: 2.2.40-1.1]
|
||||
gpg-wks-client/oldstable 2.2.40-1.1+deb12u2 amd64 [upgradable from: 2.2.40-1.1]
|
||||
gpg-wks-server/oldstable 2.2.40-1.1+deb12u2 amd64 [upgradable from: 2.2.40-1.1]
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Review this output for backup gaps (VERDICT=NO in coverage section)
|
||||
and anomalies in storage / PBS / replication sections.
|
||||
Reference in New Issue
Block a user