Files
vh 60367b307f servers: add new host dirs, refresh fleet snapshots, orientation doc
Bundles the inventory expansion since 2026-04-22:

- New host dirs (READMEs + ssh-target where dir name doesn't resolve):
    ana-nas, ana-wg, esh-vm-db, nh3-nas, pbs-ana, pbs-nh3.
- New PFI VM snapshots (registered + key-installed 2026-04-23):
    ana-filebot, pfi-ana-webhost, pfi-postgres, pfi-pteradactyl,
    pfi-tacticalrmm, sf-ana-container, sfsrv-ana (system + proxmox).
- servers/irv-ml1: ONBOARDING.md (the first-time setup notes from when
  the host was brought into the fleet) + ssh-target (10.100.79.3 over
  the WG tunnel — name doesn't DNS-resolve from this workstation).
- servers/{ana-ml2,pfi-pve,sf-r630}/README.md: updates to capture BMC
  IPs, the iDRAC vs OS hostname distinction (sf-r630 hardware =
  sfsrv-ana OS), and the ana-ml2 Supermicro BMC (10.250.250.50,
  distinct from the Dell R750xs iDRAC).
- configs/homepage/docker.yaml: irv-ml1-docker provider added so
  homepage auto-discovers irv-ml1's stacks over the WG tunnel.
- docs/orientation.md: narrative fleet overview written for fresh
  Claude sessions — sites, backup architecture, governing principles,
  gotchas, where-to-look guide. Pointed at from CLAUDE.md.
2026-04-24 21:56:46 -07:00
..

pbs-ana

Proxmox Backup Server — fleet primary. VM on pfi-pve with an NFS-mounted datastore on the Ana NAS. All 5 hypervisors (pfi-pve, nh3-pve, esh-pve, esh-pve-nas, sfsrv-ana) back up here; one-way sync to pbs-nh3 at 06:00 daily.

Network

  • LAN IP: 10.250.50.90
  • FQDN: pbs-ana.phasefinal.com
  • PBS web UI: https://10.250.50.90:8007/
  • SSH: ssh pbs-ana (config alias → lkraven@10.250.50.90). Root ops via sudo from lkraven.
  • Homepage card: PBS-ANA in Infra - ANA group.

Hardware (VM)

  • Hypervisor: pfi-pve
  • vCPU: 4 (QEMU)
  • RAM: 7.8 GB
  • Root disk: /dev/sda1 31 GB ext4 — OS only (2 GB used)
  • Datastore: NFS from ana-nas at /mnt/pbs-datastore10.250.50.50:/mnt/backup/pbs-ana, 20 TB (currently 336 GB used)

NFS details (the ZFS-case-insensitivity gotcha)

The ZFS dataset backing /mnt/backup on ana-nas is casesensitivity=insensitive, which breaks NFSv4 writes (EACCES). Mount must be NFSv3 — that's what /etc/fstab already uses:

10.250.50.50:/mnt/backup/pbs-ana /mnt/pbs-datastore nfs \
  defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0

Don't upgrade this to vers=4 without fixing the dataset. See docs/runbooks/pbs-deployment.md Phase 0.5 for the full history.

What runs here

  • proxmox-backup-proxy — HTTPS API + web UI (port 8007)
  • proxmox-backup — backend service
  • Scheduled verify job: weekly Saturday 23:00 (ignore-verified=true, outdated-after=30 days)
  • Garbage collection: default weekly

Namespaces

Per-hypervisor namespaces to prevent VMID collisions across PVE hosts (VMIDs are NOT globally unique):

  • pfi-pve
  • nh3-pve
  • esh-pve
  • esh-pve-nas
  • sfsrv-ana

API token (for hypervisor clients)

  • root@pam!fleet-vzdump — stored in password manager; referenced by each hypervisor's /etc/pve/storage.cfg pbs-ana stanza.

Sync to DR

One-way pull: pbs-nh3 pulls from here at 06:00 daily. pbs-ana does not push; all coordination is on the DR side.

Dependencies / blast radius

  • Hard dependency on ana-nas (CT 109). Datastore NFS goes dead if ana-nas crashes. Fleet-wide vzdump + verify + sync all fail until NFS recovers. See memory/incident_ana_nas_spof.md.
  • Self-backup excluded from scheduled vzdump — backing up a PBS VM to itself is a circular dependency. Config-only backup could be captured via file-level restic if needed; not currently set up (low priority — the datastore is on NFS, OS is reinstallable in ~30 min).

Refresh

scripts/refresh-server-info.sh pbs-ana

Snapshot at system-details.txt. Full deployment history: docs/runbooks/pbs-deployment.md.