43 lines
2.3 KiB
Markdown
43 lines
2.3 KiB
Markdown
# ana-docker database staging
|
|
|
|
`pre-backup.sh` runs as root through resticprofile's `run-before`. Deploy and
|
|
exercise it with `scripts/elway infra-ops@10.250.50.70 --playbook
|
|
playbooks/ana-docker-backup-repair.yaml` (allow about 12 minutes for Gitea).
|
|
|
|
Required dump failures now abort the backup. Dumps are prepared in a private
|
|
pending directory; previous staged files are replaced only after successful
|
|
generation and nonempty checks. Gitea additionally gets tar validation and a
|
|
private in-container scratch directory with exit cleanup. Ordinary failures
|
|
clean up scratch; SIGKILL or host crashes cannot run shell traps.
|
|
|
|
Regression checks: `python3 configs/restic/ana-docker/test_pre_backup.py`.
|
|
The stage/credential path overrides are for isolated tests; production defaults
|
|
remain `/var/lib/restic/stage` and `/etc/restic/dbcreds.env`.
|
|
|
|
## Repair verified 2026-09-10
|
|
|
|
Gitea dumps had failed since June 4 because five root-only historical app.ini
|
|
copies inside its config directory were unreadable to the git dump user.
|
|
Those copies retain root-only permissions under
|
|
`/opt/docker/backups/gitea-config-history/`, included in backups. Do not put
|
|
unreadable config history back inside Gitea's dump tree.
|
|
|
|
Vaultwarden's stale backup credentials were synchronized with its live database
|
|
connection; the root:600 host file is also saved as Vaultwarden item
|
|
`ana-docker/restic-dbcreds.env`. No secrets belong in this repository.
|
|
|
|
Fresh stage files were saved to Restic snapshot `2ec5a37c`. Both database dumps
|
|
were restored FROM that repository snapshot into disposable PostgreSQL 16 with
|
|
network disabled and tmpfs storage: Vaultwarden 7 users, Gitea 9 users and
|
|
93 repositories. The temporary database container was removed afterward.
|
|
This verifies database restore, not a complete application disaster-recovery drill.
|
|
|
|
Only after successful restores, 101 abandoned Gitea dump files (47.31 GiB logical
|
|
size) were deleted. Their inventory is root-only at
|
|
`/opt/docker/backups/gitea-config-history/removed-dump-manifest-20260910.json`.
|
|
Old failed-dump history itself was not retained; the fresh replacement is in
|
|
Restic. The verification marker is `/var/lib/restic/verified-repair-20260910`.
|
|
|
|
OpenWebUI's existing warning/fallback to volume capture when sqlite3 is absent
|
|
is unchanged; this repair does not claim a verified OpenWebUI database backup.
|