Files
esh-pfi-infrastructure/persistent-memory.d/2026-09-11-beszel-priority1.md
T
vh ddfa1a6e5b feat(beszel): commit the fleet agent rollout — configs, playbooks and runbooks
Beszel agents are installed and verified across the fleet but the artifacts that
produced them were never committed, so the deployment existed only on the hosts.

Adds the per-host agent environment files (PORT, NICS, EXTRA_FILESYSTEMS and the
hub's PUBLIC key), the systemd unit, the guest install script, the Synology
compose, and the elway playbooks for native, guest-stage, guest-install and
Synology paths. The two dated memory detail files covering the priority-1 and
priority-2 waves ship alongside, per the convention that memory lands with the
work it describes.

No credentials here. The KEY= value in every host env is the Beszel hub's public
ed25519 key, identical across all nine and public by design; the agent README
says so explicitly. The nh3-nas sudo password referenced in the runbook prose
lives in Vaultwarden and the helper scripts named there never contained it.

⚠ Overlapping VMIDs across hypervisors are a standing trap and are recorded in
the priority-2 notes: pfi-pve 105=postgres and 100=pbs-ana, nh3-pve 105=pbs-nh3.
⚠ PBS-NH3's export was ~75.5% used at capture; resource checks are not job
success monitoring and should not be read as such.
2026-09-11 22:09:18 -07:00

1.9 KiB

Beszel priority 1 — six deployed and verified

Operator requested two NAS plus four hypervisors. Native 0.18.7 agents enabled and verified on ana-nas, pfi-pve, nh3-pve, esh-pve, esh-pve-nas; dedicated unprivileged beszel user, no Docker installed on hypervisors. Existing hub key authentication and operator account. All five have Disk/CPU/Memory/Status alerts using existing infra-ops webhook: 20 new rules, 50 total from our wiring. Homepage 13/13 up; live samples verify extra filesystem capacity. Physical NICS filters avoid double-counting guest virtual networking. No VM workloads, storage configuration, existing agents or hub versions changed.

Canonical configs/beszel-agent/, playbooks/beszel-native.yaml. Agent archive checksum and exact mount configuration in README/env files. ZFS parents' df usage is not pool allocation; degradation alerts still not configured.

RESOLVED access: operator pointed to Vaultwarden nh3-nas/infra-ops-password; password SSH and sudo both succeeded. Earlier key-only failure did not mean privileged access was unavailable. No account/permission changes needed. Synology Docker agent installed 21:50Z, md0 root and volume1 detected; existing rest-server healthy/unrestarted. Registered gru841k5cuwb5z2 + 4 alerts, 54 total from our rollout. Canonical stacks/beszel/synology/compose.yaml and initial playbook beszel-synology.yaml. Vault-fed /tmp/nh3-nas-elway.py handles sudo and DSM scp -O without logging password. Configs saved but NOT committed: previous auto-review rejected main-branch commit; no bypass/retry attempted.

RESOLVED at 2026-09-12 01:51Z after operator reported Anaheim back except ana-ml2: Homepage 13/14 up, all six priority-1 hosts up, only ana-ml2 down. Synology live sample 01:50:32Z: CPU 1.11%, memory 7.05%, root 66.02%, volume1 32365.07/42879.36 GiB (~75.5%). Four alerts verified. Priority-1 rollout complete. No networking or ana-ml2 changes made; configs remain uncommitted.