Files
esh-pfi-infrastructure/configs/beszel-agent/PRIORITY2.md
T
vh ddfa1a6e5b feat(beszel): commit the fleet agent rollout — configs, playbooks and runbooks
Beszel agents are installed and verified across the fleet but the artifacts that
produced them were never committed, so the deployment existed only on the hosts.

Adds the per-host agent environment files (PORT, NICS, EXTRA_FILESYSTEMS and the
hub's PUBLIC key), the systemd unit, the guest install script, the Synology
compose, and the elway playbooks for native, guest-stage, guest-install and
Synology paths. The two dated memory detail files covering the priority-1 and
priority-2 waves ship alongside, per the convention that memory lands with the
work it describes.

No credentials here. The KEY= value in every host env is the Beszel hub's public
ed25519 key, identical across all nine and public by design; the agent README
says so explicitly. The nh3-nas sudo password referenced in the runbook prose
lives in Vaultwarden and the helper scripts named there never contained it.

⚠ Overlapping VMIDs across hypervisors are a standing trap and are recorded in
the priority-2 notes: pfi-pve 105=postgres and 100=pbs-ana, nh3-pve 105=pbs-nh3.
⚠ PBS-NH3's export was ~75.5% used at capture; resource checks are not job
success monitoring and should not be read as such.
2026-09-11 22:09:18 -07:00

48 lines
2.4 KiB
Markdown

# Priority 2 — verified 2026-09-11
Four native unprivileged Beszel 0.18.7 agents installed, active, boot-enabled,
and reporting fresh hub samples at 2026-09-12 01:56Z:
| Host | Address | Filesystems | Root usage |
|---|---|---|---|
| pfi-postgres | 10.250.50.80 | root (local database storage) | 6.31% |
| esh-vm-db | 10.0.50.60 | root, /mnt/backup | 2.22% |
| pbs-ana | 10.250.50.90 | root, /mnt/pbs-datastore | 10.43% |
| pbs-nh3 | 10.100.50.90 | root, /mnt/pbs-datastore | 7.32% |
PBS-ANA datastore filesystem: 1426.62/21749.13 GiB (~6.6%).
PBS-NH3 datastore filesystem: 32365.07/42879.36 GiB (~75.5%). These measure
the exported filesystem's capacity, not bytes belonging exclusively to PBS.
NFS provides capacity, not local block-device I/O counters.
Each has Disk >85%/5m, CPU >95%/15m, Memory >90%/10m and offline/2m:
16 new rules read back successfully. Existing operator-user infra-ops bridge
route retained. No notification destination changes or induced alert test.
Fleet readback: 18 registered, 17 up; only known ana-ml2 outage remains.
PostgreSQL accepts connections on both DB hosts; MongoDB running on ESH;
both PBS API and proxy services running. No DB/PBS/VM restarts performed.
Resource monitoring does NOT verify backup/sync-job success or DB query health.
## Administrative path
esh-vm-db uses infra-ops NOPASSWD sudo with playbooks/beszel-native.yaml.
Other three allow documented lkraven SSH but require sudo password; no matching
vault item found. Existing Proxmox guest-agent admin execution was verified and
used instead, WITHOUT changing accounts, sudoers or SSH settings:
| VM | Hypervisor | VMID |
|---|---|---|
| pfi-postgres | pfi-pve (10.250.250.31) | 105 |
| pbs-ana | pfi-pve (10.250.250.31) | 100 |
| pbs-nh3 | nh3-pve (10.100.250.60) | 105 |
First run `playbooks/beszel-guest-stage.yaml` against the guest's SSH alias with
`env_file=configs/beszel-agent/hosts/<name>.env`; then run
`playbooks/beszel-guest-install.yaml` against infra-ops@hypervisor with `vmid`.
Always verify host/VMID mapping first (IDs overlap across hypervisors).
Uses the previously checksum-verified binary in /tmp/beszel-priority1/;
see README.md for download/checksum. Stage files contain public key only.
Install script preserves existing application services and only restarts Beszel.
Hub registration/alert creation is separate from these deployment playbooks.
Changes saved locally, not committed, per outstanding main-branch approval.