fix(restic/esh-docker-vm): stop dumping paperless's Postgres from this host
Operator decision. paperless-ngx's database lives on esh-vm-db and is backed up at the source by that host's fail-closed pg_dumpall. esh-docker-vm's second copy had failed auth every night since 2026-04-24 behind a WARN. Its `> paperless.pg_dump` redirect left a 0-byte file in every snapshot (confirmed in snapshot 6ec9f74f), which looked like a dump but held nothing. The block was the only consumer of /etc/restic/dbcreds.env, so the creds loader is gone and the template dbcreds.env.example is deleted. The host file was moved (not deleted) to /var/lib/restic/repair-20260923/. Paperless's media volumes are still captured under /var/lib/docker/volumes. Also fixes ownership. elway's sudo upload does scp-as-user then `sudo mv`, so the hook deployed at 08:04 landed infra-ops:infra-ops even though root executes it. Both esh-docker-vm playbooks now chown it back to root and verify root:root 700. Verified: the live hook hash matches canonical (e0d3ddcef1bddf43), and the manual backup saved snapshot decfae71 with 3 staged dumps and no WARN lines.
This commit is contained in:
@@ -6,12 +6,10 @@
|
||||
# whose raw volume files risk inconsistency during live restic capture.
|
||||
#
|
||||
# Unique approach for this host: most containers don't bundle sqlite3,
|
||||
# so we run sqlite3 and pg_dump from the HOST against the volume
|
||||
# bind-mount paths. Requires sqlite3 + postgresql-client installed
|
||||
# on esh-docker-vm (apt install sqlite3 postgresql-client).
|
||||
# so we run sqlite3 from the HOST against the volume bind-mount paths.
|
||||
# Requires sqlite3 installed on esh-docker-vm (apt install sqlite3).
|
||||
#
|
||||
# Services handled:
|
||||
# - paperless-ngx (external Postgres on 10.0.50.60 — pg_dump from host)
|
||||
# - home-assistant (local SQLite in volume — sqlite3 .backup from host)
|
||||
# - calibre-web-automated (local SQLite — sqlite3 .backup inside container, has sqlite3)
|
||||
# - pgadmin (local SQLite in volume — sqlite3 .backup from host)
|
||||
@@ -21,8 +19,13 @@
|
||||
# `docker ps | grep` lookup exited 1 and set -e aborted this script, and
|
||||
# resticprofile then skipped the WHOLE host backup (stale 09-22 → 09-23).
|
||||
#
|
||||
# External DB credentials live in /etc/restic/dbcreds.env (root:600).
|
||||
# Template: configs/restic/esh-docker-vm/dbcreds.env.example.
|
||||
# paperless-ngx's Postgres (on esh-vm-db, 10.0.50.60) was also dumped from
|
||||
# here until 2026-09-23, when the block was removed by operator decision. It
|
||||
# is backed up at the source by esh-vm-db's own fail-closed pg_dumpall, and
|
||||
# this copy had failed auth every night since 2026-04-24 behind a WARN
|
||||
# nobody read, leaving a 0-byte paperless.pg_dump in every snapshot. That
|
||||
# block was the only consumer of /etc/restic/dbcreds.env, so the creds
|
||||
# loader went with it.
|
||||
#
|
||||
# Errors in individual blocks log a WARN; whole script doesn't abort.
|
||||
# That only holds if every lookup for an optional service sits inside an
|
||||
@@ -41,11 +44,6 @@ warn() { log "WARN: $*" >&2; }
|
||||
# Purge previous stage so stale dumps don't pile up in the snapshot.
|
||||
find "$STAGE" -mindepth 1 -maxdepth 1 -exec rm -rf {} +
|
||||
|
||||
# Load external-DB creds
|
||||
if [ -r /etc/restic/dbcreds.env ]; then
|
||||
set -a; . /etc/restic/dbcreds.env; set +a
|
||||
fi
|
||||
|
||||
# Helper: host-side sqlite .backup against a volume-bind path.
|
||||
# $1 = source .db path (host absolute, typically under /var/lib/docker/volumes/.../_data/)
|
||||
# $2 = stage filename (just the leaf name)
|
||||
@@ -68,25 +66,6 @@ host_sqlite_backup() {
|
||||
fi
|
||||
}
|
||||
|
||||
# ---------- paperless-ngx (external Postgres on 10.0.50.60) -------------------
|
||||
if docker inspect paperless-ngx-webserver-1 >/dev/null 2>&1; then
|
||||
if [ -z "${PAPERLESS_PGPASS:-}" ]; then
|
||||
warn "paperless-ngx: PAPERLESS_PGPASS unset in /etc/restic/dbcreds.env — skipping"
|
||||
elif ! command -v pg_dump >/dev/null 2>&1; then
|
||||
warn "paperless-ngx: pg_dump not installed — apt install postgresql-client"
|
||||
else
|
||||
log "dumping paperless postgres (${PAPERLESS_PGHOST}:${PAPERLESS_PGPORT:-5432})"
|
||||
PGPASSWORD="$PAPERLESS_PGPASS" pg_dump \
|
||||
-h "$PAPERLESS_PGHOST" -p "${PAPERLESS_PGPORT:-5432}" \
|
||||
-U "$PAPERLESS_PGUSER" -d "$PAPERLESS_PGDB" \
|
||||
-Fc --clean --if-exists \
|
||||
> "$STAGE/paperless.pg_dump" \
|
||||
|| warn "paperless pg_dump failed"
|
||||
fi
|
||||
else
|
||||
log "skip paperless: container not present"
|
||||
fi
|
||||
|
||||
# ---------- home-assistant (SQLite in named volume, host-side .backup) --------
|
||||
# HA's DB is ~50MB and actively written. SQLite .backup is the proper way
|
||||
# to grab a consistent snapshot while HA is running.
|
||||
|
||||
Reference in New Issue
Block a user