Files
esh-pfi-infrastructure/playbooks/esh-ml1-restic.yaml
T
vh d8f59a15d9 feat(esh-ml1): restic backup of augaman's gallery; augaman v0.1.2
esh-ml1 is outside vzdump, so augaman's face gallery reaches backup only
through restic. New playbooks/esh-ml1-restic.yaml installs restic 0.14.0 (the
same Debian package as the other ESH hosts) and resticprofile 0.33.1 (pinned,
sha256-checked). It uploads configs/restic/esh-ml1/ and schedules a daily
0100 PT backup plus a Sunday 0500 PT check to rest-server-ana. The CT runs UTC,
so both schedules name the zone explicitly.

pre-backup.sh is fail-closed: it runs augaman's own backup CLI, and any failure,
including a stopped container, aborts the run. Tested with a stub docker that
exits 1: the run returned 1, and neither the snapshot count nor last-success
moved. The restore was verified at identity level against augaman-dev's
public-domain canary (snapshot fd3061a1: the restored copy's digest over
identities and samples matches the live gallery). That meets the operator gate
for real enrollments.

The repository URL is read through repository-file rather than restic.env.
resticprofile schedule copies env-file values into world-readable systemd
units, which publishes the rest-server password on the env-file hosts
(observed on esh-docker-vm). This is recorded in the backups runbook under
Known gaps, and the playbook verifies no generated unit contains the URL.

esh-ml1 is added to the freshness check's expected ana-side repos and to the
runbook tables.

augaman moves to v0.1.2 (dependency layer keyed on the lock without the
project; per-crop embedding). pytest -m gpu tests/vision passes 3/3 on the
card, and the canary survived the container recreate.
2026-09-27 00:06:33 -07:00

104 lines
4.4 KiB
YAML

# esh-ml1 restic: backs up augaman's face gallery (and the deployed compose dir)
# to rest-server-ana. Config + rationale: configs/restic/esh-ml1/.
#
# Run: scripts/elway esh-ml1 --playbook playbooks/esh-ml1-restic.yaml
#
# PRE-REQUISITE, NOT DONE HERE (secrets never live in this repo): seed the two
# root-only secret files from the vault, and the rest-server-ana htpasswd entry
# for user `esh-ml1`. The exact commands are in configs/restic/esh-ml1/README.md.
# The "Secrets are seeded" step below halts the run if they are missing.
#
# Idempotent: a second run should report ok/skipped everywhere except the
# re-uploads and the schedule refresh.
vars:
resticprofile_version: 0.33.1
resticprofile_sha256: 5fecd20de21811a750a4d61e841b2258fdf018bbfffad136c96078df27b452df
steps:
- name: Install restic (Debian 12 package, 0.14.0, same as esh-docker-vm and esh-vm-db)
sudo: true
shell: DEBIAN_FRONTEND=noninteractive apt-get install -y -q restic
when: "! command -v restic >/dev/null"
- name: Install resticprofile {{ resticprofile_version }} (pinned, sha256-checked)
sudo: true
shell: |
set -eu
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
tgz="resticprofile_{{ resticprofile_version }}_linux_amd64.tar.gz"
curl -fsSL -o "$tmp/$tgz" "https://github.com/creativeprojects/resticprofile/releases/download/v{{ resticprofile_version }}/$tgz"
echo "{{ resticprofile_sha256 }} $tmp/$tgz" | sha256sum -c -
tar -xzf "$tmp/$tgz" -C "$tmp" resticprofile
install -o root -g root -m 0755 "$tmp/resticprofile" /usr/local/bin/resticprofile
when: "! /usr/local/bin/resticprofile version 2>/dev/null | grep -q 'version {{ resticprofile_version }} '"
- name: Directories (stage is root-only; its augaman subdir belongs to the container user 10001)
sudo: true
shell: |
set -eu
install -d -o root -g root -m 0755 /etc/restic
install -d -o root -g root -m 0700 /var/lib/restic/stage
install -d -o 10001 -g 10001 -m 0700 /var/lib/restic/stage/augaman
when: "! sudo -n sh -c 'test \"$(stat -c %U:%a /var/lib/restic/stage)\" = root:700 && test \"$(stat -c %u:%a /var/lib/restic/stage/augaman)\" = 10001:700 && test -d /etc/restic'"
- name: Secrets are seeded (password + repository, root 0400)
sudo: true
shell: |
set -eu
for f in /etc/restic/password /etc/restic/repository; do
test -s "$f" || { echo "missing $f: seed it from the vault (configs/restic/esh-ml1/README.md)"; exit 1; }
test "$(stat -c %U:%a "$f")" = root:400 || { echo "$f must be root 0400"; exit 1; }
done
changed_when: "false"
- name: restic understands --repository-file
shell: restic --help | grep -q -- --repository-file
changed_when: "false"
- name: Upload profiles.yaml
sudo: true
upload:
src: configs/restic/esh-ml1/profiles.yaml
dest: /etc/restic/profiles.yaml
mode: "0644"
- name: Upload pre-backup.sh
sudo: true
upload:
src: configs/restic/esh-ml1/pre-backup.sh
dest: /etc/restic/pre-backup.sh
mode: "0755"
- name: Initialise the repository (once)
sudo: true
shell: resticprofile --no-ansi --config /etc/restic/profiles.yaml --name default init
when: "! sudo -n restic --repository-file /etc/restic/repository --password-file /etc/restic/password cat config >/dev/null 2>&1"
- name: Install the systemd timers
sudo: true
shell: resticprofile --no-ansi --config /etc/restic/profiles.yaml --name default schedule
verify:
- name: Backup and check timers are active
shell: systemctl is-active --quiet resticprofile-backup@profile-default.timer && systemctl is-active --quiet resticprofile-check@profile-default.timer
changed_when: "false"
# The units must EXIST before "no match" means anything: a grep over a path that
# is not there also "finds nothing". `rest:http` is the exact form the leak takes
# on the env-file hosts (seen in esh-docker-vm's unit, 2026-09-27).
- name: No repository URL (and so no rest-server password) in the generated units
shell: |
set -eu
d=/etc/systemd/system
for u in resticprofile-backup@profile-default.service resticprofile-check@profile-default.service; do
test -f "$d/$u" || { echo "missing unit $u"; exit 1; }
if grep -q 'rest:http' "$d/$u"; then echo "$u embeds the repository URL"; exit 1; fi
done
changed_when: "false"
- name: pre-backup.sh parses
shell: bash -n /etc/restic/pre-backup.sh
changed_when: "false"