Files
esh-pfi-infrastructure/configs/beszel-agent/README.md
T
vh ddfa1a6e5b feat(beszel): commit the fleet agent rollout — configs, playbooks and runbooks
Beszel agents are installed and verified across the fleet but the artifacts that
produced them were never committed, so the deployment existed only on the hosts.

Adds the per-host agent environment files (PORT, NICS, EXTRA_FILESYSTEMS and the
hub's PUBLIC key), the systemd unit, the guest install script, the Synology
compose, and the elway playbooks for native, guest-stage, guest-install and
Synology paths. The two dated memory detail files covering the priority-1 and
priority-2 waves ship alongside, per the convention that memory lands with the
work it describes.

No credentials here. The KEY= value in every host env is the Beszel hub's public
ed25519 key, identical across all nine and public by design; the agent README
says so explicitly. The nh3-nas sudo password referenced in the runbook prose
lives in Vaultwarden and the helper scripts named there never contained it.

⚠ Overlapping VMIDs across hypervisors are a standing trap and are recorded in
the priority-2 notes: pfi-pve 105=postgres and 100=pbs-ana, nh3-pve 105=pbs-nh3.
⚠ PBS-NH3's export was ~75.5% used at capture; resource checks are not job
success monitoring and should not be read as such.
2026-09-11 22:09:18 -07:00

3.5 KiB

Native Beszel agents — priority 1

Deployed 2026-09-11 to ana-nas (LXC), pfi-pve, nh3-pve, esh-pve and esh-pve-nas. No Docker was installed on hypervisors. Agents run as dedicated unprivileged beszel, with a systemd service enabled at boot. Hub SSH-polls port 45876 using its existing public key; no new private credential is needed. The committed key is PUBLIC. Do not commit API credentials or agent tokens.

Version pinned to 0.18.7 to match the existing hub. Download beszel-agent_linux_amd64.tar.gz from the official GitHub v0.18.7 release; verified SHA256: 4ae327aac5ad5a231845b0ef613066d555bbe52f7ecb2f28a53d07c04e689aff. Extract the executable to /tmp/beszel-priority1/beszel-agent, then run:

scripts/elway infra-ops@10.250.50.50 --playbook playbooks/beszel-native.yaml \
  --var env_file=configs/beszel-agent/hosts/ana-nas.env

Use the matching address and env file for each host; binary_file overrides the local executable path. Hub registration and four alert records are separate from this host deployment playbook and already exist for all five.

Host environments list observed mountpoints. Hypervisor NICS explicitly select physical Ethernet ports, avoiding bridge/tap/veth double counting; revisit if hardware changes. Root is automatically monitored. ZFS extra-filesystem labels can be backing-device names or dataset basenames rather than mountpoint names. A parent dataset's usage does NOT include every child's allocation. These are filesystem-capacity charts, not whole-pool capacity/degradation guarantees. LXC host-level CPU/kernel metrics may reflect the shared host; ana-nas is not an independent physical NAS.

Acceptance: all five services enabled and active; hub samples contain all configured extra filesystems; Homepage reports 13/13 systems up. Added four rules each: Disk >85%/5m, CPU >95%/15m, Memory >90%/10m, offline/2m. Existing operator user's infra-ops bridge route retained. No induced alert test this pass; that route was end-to-end tested on 2026-09-10.

Synology sixth host — installed and verified

Operator pointed out the existing Vaultwarden nh3-nas/infra-ops-password. Password-based SSH AND sudo as infra-ops both work. Key-only SSH rejection was not a need to bootstrap access; always check the vault before asking operator. No permissions, keys or account settings changed.

Deployed 2026-09-11 at 21:50Z via existing DSM Container Manager: /volume1/compose/beszel/compose.yaml, canonical stacks/beszel/synology/compose.yaml. Restart policy unless-stopped, 0.18.7. Read-only zoneinfo mount measures DSM root (md0); read-only stack-directory mount measures /volume1 (cachedev_0). Agent logs confirm both. Existing rest-server stayed healthy with four-week uptime. DSM root ~67%, volume1 ~76%.

Hub registration gru841k5cuwb5z2 and four standard alerts created successfully before a connectivity failure. Recovery verified at 2026-09-12 01:51Z: Homepage shows 13/14 up, only ana-ml2 down (operator aware). All six priority-1 hosts up. Synology sample at 01:50:32Z reports CPU 1.11%, memory 7.05%, DSM root 66.02%, volume1 32365.07/42879.36 GiB (~75.5%). All four alert rules read back correctly. No network or ana-ml2 changes attempted.

Initial-install playbook playbooks/beszel-synology.yaml refuses to overwrite an existing compose file. DSM requires legacy SCP (scp -O). This session used /tmp/nh3-nas-elway.py to supply vaulted sudo password in memory, legacy SCP, and /tmp/nh3-nas-askpass.py for SSH; neither script contains the password. The native Debian unit is NOT intended for DSM's systemd 219.