feat(esh-ml1): restic backup of augaman's gallery; augaman v0.1.2
esh-ml1 is outside vzdump, so augaman's face gallery reaches backup only through restic. New playbooks/esh-ml1-restic.yaml installs restic 0.14.0 (the same Debian package as the other ESH hosts) and resticprofile 0.33.1 (pinned, sha256-checked). It uploads configs/restic/esh-ml1/ and schedules a daily 0100 PT backup plus a Sunday 0500 PT check to rest-server-ana. The CT runs UTC, so both schedules name the zone explicitly. pre-backup.sh is fail-closed: it runs augaman's own backup CLI, and any failure, including a stopped container, aborts the run. Tested with a stub docker that exits 1: the run returned 1, and neither the snapshot count nor last-success moved. The restore was verified at identity level against augaman-dev's public-domain canary (snapshot fd3061a1: the restored copy's digest over identities and samples matches the live gallery). That meets the operator gate for real enrollments. The repository URL is read through repository-file rather than restic.env. resticprofile schedule copies env-file values into world-readable systemd units, which publishes the rest-server password on the env-file hosts (observed on esh-docker-vm). This is recorded in the backups runbook under Known gaps, and the playbook verifies no generated unit contains the URL. esh-ml1 is added to the freshness check's expected ana-side repos and to the runbook tables. augaman moves to v0.1.2 (dependency layer keyed on the lock without the project; per-crop embedding). pytest -m gpu tests/vision passes 3/3 on the card, and the canary survived the container recreate.
This commit is contained in:
@@ -0,0 +1,71 @@
|
||||
# resticprofile config for esh-ml1.
|
||||
#
|
||||
# ESH GPU LXC (CT 110 on esh-pve). The host is OUTSIDE vzdump on purpose:
|
||||
# everything on it is rebuilt from playbooks + stacks, and models re-download.
|
||||
# The one exception is augaman's face gallery (biometric data, not
|
||||
# reconstructable), so this profile exists for that and little else.
|
||||
#
|
||||
# Writes cross-site to rest-server-ana (10.250.50.70:8000/esh-ml1/) because
|
||||
# the ESH site has no local rest-server (same as esh-docker-vm / esh-vm-db).
|
||||
#
|
||||
# ⚠ DELIBERATE DEVIATION from the other hosts: the repository URL (which
|
||||
# embeds the rest-server basic-auth password) is read from
|
||||
# /etc/restic/repository via `repository-file`, NOT from an env-file.
|
||||
# `resticprofile schedule` copies env-file values into the generated systemd
|
||||
# unit, and units under /etc/systemd/system are world-readable (0644), so
|
||||
# the env-file pattern publishes the credential to every local user. With
|
||||
# repository-file the unit carries only the path. See README.md.
|
||||
#
|
||||
# ⚠ The CT runs in UTC; the schedules carry an explicit zone so they fire at
|
||||
# the same wall-clock time as the rest of the fleet.
|
||||
|
||||
version: "1"
|
||||
|
||||
global:
|
||||
priority: low
|
||||
ionice: true
|
||||
ionice-class: 2
|
||||
ionice-level: 7
|
||||
min-memory: 100
|
||||
|
||||
default:
|
||||
repository-file: /etc/restic/repository
|
||||
password-file: /etc/restic/password
|
||||
initialize: false
|
||||
lock: /var/lock/restic-esh-ml1.lock
|
||||
|
||||
backup:
|
||||
verbose: 1
|
||||
run-before:
|
||||
# Fail-closed: if augaman's backup CLI fails (or the container is down),
|
||||
# the whole run fails, last-success is not written, and the freshness
|
||||
# check alerts. A stale gallery copy must never be snapshotted as current.
|
||||
- /etc/restic/pre-backup.sh
|
||||
run-after:
|
||||
- date +%s > /var/lib/restic/last-success
|
||||
source:
|
||||
- /var/lib/restic/stage # augaman/gallery.db, written by pre-backup.sh
|
||||
- /opt/docker/compose # the deployed compose files + .env (small)
|
||||
tag:
|
||||
- host:esh-ml1
|
||||
- site:esh
|
||||
- fleet:home-lab
|
||||
schedule: "*-*-* 01:00:00 America/Los_Angeles"
|
||||
schedule-permission: system
|
||||
schedule-log: /var/log/restic-backup.log
|
||||
|
||||
forget:
|
||||
keep-daily: 7
|
||||
keep-weekly: 4
|
||||
keep-monthly: 12
|
||||
keep-yearly: 3
|
||||
tag:
|
||||
- host:esh-ml1
|
||||
# No schedule: rest-server-ana is --append-only, so forget always fails
|
||||
# from the client. It runs by hand during the prune ceremony.
|
||||
|
||||
check:
|
||||
read-data-subset: 10%
|
||||
schedule: "Sun *-*-* 05:00:00 America/Los_Angeles"
|
||||
schedule-permission: system
|
||||
schedule-log: /var/log/restic-check.log
|
||||
Reference in New Issue
Block a user