Files
esh-pfi-infrastructure/persistent-memory.d/2026-09-27-restic-repository-file-fleetwide.md
T

41 lines
2.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# `[2026-09-27]` restic: repository URL out of the systemd units on all 8 hosts, and vaulted (Prime)
`resticprofile schedule` copies `env-file` values into the generated units under /etc/systemd/system
(0644). So `RESTIC_REPOSITORY`, including the rest-server basic-auth password, was readable by any local
user on every env-file host (found on esh-docker-vm; `systemctl cat` works without sudo). Blast radius:
the password allows reading the encrypted blobs and appending to one repo, and nothing more (rest-server
is `--append-only`, and the passphrase is a separate file).
**Fix** (`6e203dc`, `30f2c97`): `playbooks/restic-repository-file.yaml` does four things:
- derives `/etc/restic/repository` (root 0400) from `restic.env`;
- uploads the profile switched to `repository-file`, guarded by the live profile's pre-change sha;
- runs `cat config` through the new profile;
- re-schedules, then verifies the units exist and contain no `rest:http`.
Applied to ana-docker, fv-ml1 (`configs/restic/ana-ml2`), esh-docker-vm, esh-vm-db, irv-ml1, nh3-dev,
nh3-docker, and vm-esh-nas once Prime had bootstrapped infra-ops there. esh-ml1 was built that way. An
independent check across all 8 found 0 leaking units. nh3-docker's scheduled unit ran a real backup
(`a29b889d`). Every host's URL and passphrase are vaulted as `<host>/etc/restic/{repository,password}`.
- `restic.env` is **kept** (root 0600): the per-host READMEs and the freshness probe source it. **A rotation
must update the vault, `restic.env` and `repository`.**
- The rotation itself stays Prime's (backups runbook, Known gaps). The move stops the ongoing exposure but
does not un-expose what was readable.
- Before the edit, esh-docker-vm's and irv-ml1's live profiles had drifted ahead of the repo, and were
pulled in first (`c698751`).
## In-flight state at retirement (moved here at the 2026-10-06 snapshot)
- Seven hosts were moved from `env-file` to `repository-file` (`playbooks/restic-repository-file.yaml`),
so the systemd units no longer carry the rest-server password. Secrets are vaulted as
`<host>/etc/restic/{repository,password}`. `restic.env` is KEPT for manual snippets, so
**a rotation must update the vault, `restic.env` and `repository`.**
- **vm-esh-nas done too** (Prime bootstrapped infra-ops there at uid 850, NOPASSWD). **All 8 hosts are clean
and vaulted.**
- The passwords were readable until today, so the rotation (Prime's) remains the real fix.
- 2026-09-27 0859 freshness check: all restic and PBS repos ✅. Those 0100 runs PRE-date the move
(done ~0140–0200).
- **VERIFIED 2026-09-28 (infra-hermes, read-only):** the first nightly on `repository-file` (0100 PT)
succeeded on all 8 hosts (unit Result=success AND a today-dated snapshot in each repo), and the
0800 freshness check was all-green, PBS included. nh3-dev also passed a content restore (`af5580c3`).