Files
esh-pfi-infrastructure/playbooks/esh-docker-vm-restic-drop-kuma-block.yaml
T
vh 6e8da46a28 fix(restic/esh-docker-vm): stop dumping paperless's Postgres from this host
Operator decision. paperless-ngx's database lives on esh-vm-db and is backed
up at the source by that host's fail-closed pg_dumpall. esh-docker-vm's
second copy had failed auth every night since 2026-04-24 behind a WARN. Its
`> paperless.pg_dump` redirect left a 0-byte file in every snapshot
(confirmed in snapshot 6ec9f74f), which looked like a dump but held nothing.

The block was the only consumer of /etc/restic/dbcreds.env, so the creds
loader is gone and the template dbcreds.env.example is deleted. The host
file was moved (not deleted) to /var/lib/restic/repair-20260923/.
Paperless's media volumes are still captured under /var/lib/docker/volumes.

Also fixes ownership. elway's sudo upload does scp-as-user then `sudo mv`,
so the hook deployed at 08:04 landed infra-ops:infra-ops even though root
executes it. Both esh-docker-vm playbooks now chown it back to root and
verify root:root 700.

Verified: the live hook hash matches canonical (e0d3ddcef1bddf43), and
the manual backup saved snapshot decfae71 with 3 staged dumps and no WARN
lines.
2026-09-23 09:18:03 -07:00

49 lines
1.7 KiB
YAML

# esh-docker-vm: install the pre-backup hook without the uptime-kuma block.
#
# Kuma moved to ana-docker on 2026-09-22. The old block's unguarded
# `docker ps | grep uptime.kuma` lookup then matched nothing, exited 1, and
# set -euo pipefail aborted the hook. resticprofile treats a failed run-before
# as fatal, so every nightly backup since 2026-09-22 01:00 was skipped.
#
# Rerunnable: the preserve step is `creates:`-guarded, and the upload only
# changes the file when the content differs.
steps:
- name: Preserve the pre-fix hook
sudo: true
shell: |
set -eu
install -d -m 0700 /var/lib/restic/repair-20260923
cp -p /etc/restic/pre-backup.sh /var/lib/restic/repair-20260923/pre-backup.sh
creates: /var/lib/restic/repair-20260923/pre-backup.sh
- name: Install the hook without the uptime-kuma block
sudo: true
upload:
src: configs/restic/esh-docker-vm/pre-backup.sh
dest: /etc/restic/pre-backup.sh
mode: '0700'
# elway's sudo upload is scp-as-user then `sudo mv`, so the file lands owned
# by the SSH user. This hook is executed by root: it must be root-owned.
- name: Make the hook root-owned
sudo: true
shell: chown root:root /etc/restic/pre-backup.sh
verify:
- name: Hook parses under bash
sudo: true
shell: bash -n /etc/restic/pre-backup.sh
- name: No uptime-kuma lookup left in the live hook
sudo: true
shell: "! grep -q 'UK_CONTAINER' /etc/restic/pre-backup.sh"
- name: Hook runs to completion (stage summary line reached)
sudo: true
shell: /etc/restic/pre-backup.sh 2>&1 | grep -q 'stage ready:'
- name: Hook is root:root 0700
sudo: true
shell: test "$(stat -c '%U:%G %a' /etc/restic/pre-backup.sh)" = "root:root 700"