# pbs-nh3

Proxmox Backup Server — **DR mirror**. VM on nh3-pve with an
NFS-mounted datastore on the NH3 Synology. Pulls nightly from pbs-ana
(the primary) at 06:00.

## Network

- **LAN IP:** 10.100.50.90
- **FQDN:** `pbs-nh3.phasefinal.com`
- **PBS web UI:** <https://10.100.50.90:8007/>
- **SSH:** `ssh pbs-nh3` (config alias → `lkraven@10.100.50.90`).
  Root ops via sudo from lkraven.
- **Homepage card:** *PBS-NH3* in `Infra - NH3` group.

## Hardware (VM)

- **Hypervisor:** nh3-pve
- **vCPU:** 4 (QEMU)
- **RAM:** 7.8 GB
- **Root disk:** `/dev/sda1` 31 GB ext4 — OS only (2 GB used)
- **Datastore:** **NFS from nh3-nas** at `/mnt/pbs-datastore` →
  `10.100.50.50:/volume1/pbs`. Volume size 42 TB; 27 TB used (64%).

## NFS details (the Synology ACL gotcha)

Synology's `/volume1/pbs` share was initially created with "Advanced
Permissions" which imposes a `syno_acl` that denies writes to
non-admin UIDs (even the `backup` uid PBS runs as). Fix was to flatten
the share to pure POSIX mode 777 so the mount becomes "Linux mode"
(`synoacltool -get` confirms). `no_root_squash + no_all_squash` also
set on the NFS export.

Mount uses NFSv3 (same pattern as pbs-ana, for consistency more than
necessity — Synology's Btrfs doesn't have the ZFS-case-insensitivity
issue):

```
10.100.50.50:/volume1/pbs /mnt/pbs-datastore nfs \
  defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0
```

Full history: `docs/runbooks/pbs-deployment.md` Phase 5.3.

## What runs here

- `proxmox-backup-proxy` — HTTPS API + web UI (port 8007)
- `proxmox-backup` — backend service
- Scheduled verify job: weekly Sunday 12:00 (after nightly sync
  completes), `ignore-verified=true`, `outdated-after=30 days`
- Scheduled sync from pbs-ana: daily at 06:00 UTC

## Remote config

- Remote name: `pbs-ana`
- Remote host: `10.250.50.90:8007`
- Auth: `root@pam!fleet-vzdump` token (same token as clients use on
  the primary — shared here for simplicity; could be split to a
  sync-only read-only token)
- Cert fingerprint: in password manager under "pbs-ana fingerprint"

## Namespaces

Namespaces are auto-replicated from pbs-ana via the sync job — you do
NOT need to pre-create them on the mirror. Expected namespaces after
first sync: same 5 as pbs-ana (`pfi-pve`, `nh3-pve`, `esh-pve`,
`esh-pve-nas`, `sfsrv-ana`).

## Dependencies / blast radius

- **Hard dependency on nh3-nas** (Synology). Datastore goes dead if
  the Synology crashes; sync fails, verify fails, restore ops fail.
  But: ANA primary keeps operating fine, so the fleet's backup SLA
  is preserved — only DR redundancy is temporarily lost.
- **Sync source**: pbs-ana going offline means no new snapshots pull
  here, but existing ones remain restorable. Sync resumes
  automatically once pbs-ana recovers.

## DR failover (hypothetical)

If pbs-ana is permanently lost and we needed to restore fleet VMs
from pbs-nh3:

1. Re-point each PVE host's `pbs-ana` storage entry at pbs-nh3
   (`pve-datacenter.cfg` → change `server` to 10.100.50.90)
2. Fingerprint + token still work (same token)
3. Cross-WAN restore is slower than local but functional

Documented as future work — not tested. Run a trial restore of a
small CT periodically to validate.

## Refresh

```
scripts/refresh-server-info.sh pbs-nh3
```

Snapshot at `system-details.txt`. Full deployment history:
`docs/runbooks/pbs-deployment.md`.