vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(esh-ml1): restic backup of augaman's gallery; augaman v0.1.2

Browse Source 
esh-ml1 is outside vzdump, so augaman's face gallery reaches backup only
through restic. New playbooks/esh-ml1-restic.yaml installs restic 0.14.0 (the
same Debian package as the other ESH hosts) and resticprofile 0.33.1 (pinned,
sha256-checked). It uploads configs/restic/esh-ml1/ and schedules a daily
0100 PT backup plus a Sunday 0500 PT check to rest-server-ana. The CT runs UTC,
so both schedules name the zone explicitly.

pre-backup.sh is fail-closed: it runs augaman's own backup CLI, and any failure,
including a stopped container, aborts the run. Tested with a stub docker that
exits 1: the run returned 1, and neither the snapshot count nor last-success
moved. The restore was verified at identity level against augaman-dev's
public-domain canary (snapshot fd3061a1: the restored copy's digest over
identities and samples matches the live gallery). That meets the operator gate
for real enrollments.

The repository URL is read through repository-file rather than restic.env.
resticprofile schedule copies env-file values into world-readable systemd
units, which publishes the rest-server password on the env-file hosts
(observed on esh-docker-vm). This is recorded in the backups runbook under
Known gaps, and the playbook verifies no generated unit contains the URL.

esh-ml1 is added to the freshness check's expected ana-side repos and to the
runbook tables.

augaman moves to v0.1.2 (dependency layer keyed on the lock without the
project; per-crop embedding). pytest -m gpu tests/vision passes 3/3 on the
card, and the canary survived the container recreate.
This commit is contained in:
vh
2026-09-27 00:06:33 -07:00
parent c26f7c94e4
commit d8f59a15d9
10 changed files with 347 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.md
+7 -7
View File
@@ -144,13 +144,13 @@ _As of 2026-09-26 ~1620 PT._
### augaman: face recognition for Cicada
- **v0.1.1 DEPLOYED on esh-ml1:8040 2026-09-26 2347 PT** (`stacks/augaman`),
healthy on CUDA, in `nvidia-smi`. Built on-box from a `git archive` of the tag.
- **infra-ops owes the gallery backup NEXT:** esh-ml1 has no restic.
`BACKUP_DIR=/var/lib/restic/stage/augaman` exists; needs restic + timer running
the backup CLI, then one verified restore. **Fixtures only until then** (Prime).
- `pytest -m gpu` FAILS 3/3 on the batch-vs-single tolerance (≥6e-5 vs 1e-5);
placement passes. Reported to augaman-dev: the contract is theirs to rule on.
- **v0.1.2 LIVE on esh-ml1:8040** (2026-09-27; v0.1.1 first deployed 2026-09-26 2347 PT),
`stacks/augaman`, healthy on CUDA, in `nvidia-smi`. Built on-box from a `git archive`
of the tag. `pytest -m gpu tests/vision` 3/3 PASS on v0.1.2.
- **Gallery backup WIRED + RESTORE-VERIFIED 2026-09-27** (Prime OK'd off-site): restic
daily 0100 PT → rest-server-ana `esh-ml1/`, fail-closed hook, secrets vaulted under
`esh-ml1/etc/restic/`. Identity-level restore matched the live gallery (canary,
snapshot `fd3061a1`). **The backup gate for real enrollments is MET.**
### esh-ml1