vh
60367b307f
Bundles the inventory expansion since 2026-04-22:
- New host dirs (READMEs + ssh-target where dir name doesn't resolve):
ana-nas, ana-wg, esh-vm-db, nh3-nas, pbs-ana, pbs-nh3.
- New PFI VM snapshots (registered + key-installed 2026-04-23):
ana-filebot, pfi-ana-webhost, pfi-postgres, pfi-pteradactyl,
pfi-tacticalrmm, sf-ana-container, sfsrv-ana (system + proxmox).
- servers/irv-ml1: ONBOARDING.md (the first-time setup notes from when
the host was brought into the fleet) + ssh-target (10.100.79.3 over
the WG tunnel — name doesn't DNS-resolve from this workstation).
- servers/{ana-ml2,pfi-pve,sf-r630}/README.md: updates to capture BMC
IPs, the iDRAC vs OS hostname distinction (sf-r630 hardware =
sfsrv-ana OS), and the ana-ml2 Supermicro BMC (10.250.250.50,
distinct from the Dell R750xs iDRAC).
- configs/homepage/docker.yaml: irv-ml1-docker provider added so
homepage auto-discovers irv-ml1's stacks over the WG tunnel.
- docs/orientation.md: narrative fleet overview written for fresh
Claude sessions — sites, backup architecture, governing principles,
gotchas, where-to-look guide. Pointed at from CLAUDE.md.
100 lines
3.3 KiB
Markdown
100 lines
3.3 KiB
Markdown
# pbs-nh3
|
|
|
|
Proxmox Backup Server — **DR mirror**. VM on nh3-pve with an
|
|
NFS-mounted datastore on the NH3 Synology. Pulls nightly from pbs-ana
|
|
(the primary) at 06:00.
|
|
|
|
## Network
|
|
|
|
- **LAN IP:** 10.100.50.90
|
|
- **FQDN:** `pbs-nh3.phasefinal.com`
|
|
- **PBS web UI:** <https://10.100.50.90:8007/>
|
|
- **SSH:** `ssh pbs-nh3` (config alias → `lkraven@10.100.50.90`).
|
|
Root ops via sudo from lkraven.
|
|
- **Homepage card:** *PBS-NH3* in `Infra - NH3` group.
|
|
|
|
## Hardware (VM)
|
|
|
|
- **Hypervisor:** nh3-pve
|
|
- **vCPU:** 4 (QEMU)
|
|
- **RAM:** 7.8 GB
|
|
- **Root disk:** `/dev/sda1` 31 GB ext4 — OS only (2 GB used)
|
|
- **Datastore:** **NFS from nh3-nas** at `/mnt/pbs-datastore` →
|
|
`10.100.50.50:/volume1/pbs`. Volume size 42 TB; 27 TB used (64%).
|
|
|
|
## NFS details (the Synology ACL gotcha)
|
|
|
|
Synology's `/volume1/pbs` share was initially created with "Advanced
|
|
Permissions" which imposes a `syno_acl` that denies writes to
|
|
non-admin UIDs (even the `backup` uid PBS runs as). Fix was to flatten
|
|
the share to pure POSIX mode 777 so the mount becomes "Linux mode"
|
|
(`synoacltool -get` confirms). `no_root_squash + no_all_squash` also
|
|
set on the NFS export.
|
|
|
|
Mount uses NFSv3 (same pattern as pbs-ana, for consistency more than
|
|
necessity — Synology's Btrfs doesn't have the ZFS-case-insensitivity
|
|
issue):
|
|
|
|
```
|
|
10.100.50.50:/volume1/pbs /mnt/pbs-datastore nfs \
|
|
defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0
|
|
```
|
|
|
|
Full history: `docs/runbooks/pbs-deployment.md` Phase 5.3.
|
|
|
|
## What runs here
|
|
|
|
- `proxmox-backup-proxy` — HTTPS API + web UI (port 8007)
|
|
- `proxmox-backup` — backend service
|
|
- Scheduled verify job: weekly Sunday 12:00 (after nightly sync
|
|
completes), `ignore-verified=true`, `outdated-after=30 days`
|
|
- Scheduled sync from pbs-ana: daily at 06:00 UTC
|
|
|
|
## Remote config
|
|
|
|
- Remote name: `pbs-ana`
|
|
- Remote host: `10.250.50.90:8007`
|
|
- Auth: `root@pam!fleet-vzdump` token (same token as clients use on
|
|
the primary — shared here for simplicity; could be split to a
|
|
sync-only read-only token)
|
|
- Cert fingerprint: in password manager under "pbs-ana fingerprint"
|
|
|
|
## Namespaces
|
|
|
|
Namespaces are auto-replicated from pbs-ana via the sync job — you do
|
|
NOT need to pre-create them on the mirror. Expected namespaces after
|
|
first sync: same 5 as pbs-ana (`pfi-pve`, `nh3-pve`, `esh-pve`,
|
|
`esh-pve-nas`, `sfsrv-ana`).
|
|
|
|
## Dependencies / blast radius
|
|
|
|
- **Hard dependency on nh3-nas** (Synology). Datastore goes dead if
|
|
the Synology crashes; sync fails, verify fails, restore ops fail.
|
|
But: ANA primary keeps operating fine, so the fleet's backup SLA
|
|
is preserved — only DR redundancy is temporarily lost.
|
|
- **Sync source**: pbs-ana going offline means no new snapshots pull
|
|
here, but existing ones remain restorable. Sync resumes
|
|
automatically once pbs-ana recovers.
|
|
|
|
## DR failover (hypothetical)
|
|
|
|
If pbs-ana is permanently lost and we needed to restore fleet VMs
|
|
from pbs-nh3:
|
|
|
|
1. Re-point each PVE host's `pbs-ana` storage entry at pbs-nh3
|
|
(`pve-datacenter.cfg` → change `server` to 10.100.50.90)
|
|
2. Fingerprint + token still work (same token)
|
|
3. Cross-WAN restore is slower than local but functional
|
|
|
|
Documented as future work — not tested. Run a trial restore of a
|
|
small CT periodically to validate.
|
|
|
|
## Refresh
|
|
|
|
```
|
|
scripts/refresh-server-info.sh pbs-nh3
|
|
```
|
|
|
|
Snapshot at `system-details.txt`. Full deployment history:
|
|
`docs/runbooks/pbs-deployment.md`.
|