Commit Graph
2 Commits
Author SHA1 Message Date
vh e979ccb337 fix(backups): the freshness alarm had no wire — reconnect it and make it testable
The daily backup-freshness check has been unable to raise an alert since the
2026-08-28 althing v3 cutover. It called althing-cli, which v3 DELETED rather
than deprecated. The check itself never stopped working: it detected three
stale backups every morning and told nobody, and the only trace was a WARN
line inside a unit that was already reporting `failed` for the stale backups
themselves. Three weeks, silent.

Four changes, because swapping the binary alone would have left it dead:

  * althing-cli -> postbox.
  * Add ALTHING_POST_OFFICE to the systemd user unit AND to the installer that
    writes it. postbox has no default address by design and a user unit
    inherits nothing from the interactive shell, so the binary swap on its own
    would have failed with a different message. Fixing only the live unit
    would have been undone by the next installer run; the two are now verified
    to agree.
  * Recipient infra-ops -> infra-hermes. This runs AS infra-ops, so the old
    address mailed the alarm to itself — the mirror trap named in CLAUDE.md.
    Day-to-day checks are infra-hermes's half of the split; he escalates.
  * Split the exit codes. 1 now means "backups stale, someone was told";
    2 means "the alert path itself failed". A broken alarm is a worse fault
    than the thing it watches and must not be indistinguishable from it.

Adds --test-alert: a positive control that sends a real message through the
real path on demand. The wire was cut for three weeks precisely because
nothing ever exercised it in the healthy state, and an alarm whose success
path is never run is not known to work.

Verified: positive control delivered; missing-address and unreachable-post-
office both correctly exit 2; a real run through systemd delivered the alert
and exited 1.
2026-09-19 05:09:54 -07:00
vh 76b317ce3e feat(backups): freshness check + daily alert timer; record rest-server-ana recovery, fstab hardening, esh-pve-nas gap, worldtree admin-key provisioning 2026-06-20 16:13:13 -07:00