esh-pfi-infrastructure/configs/restic/esh-docker-vm/README.md

restic / esh-docker-vm

Primary ESH home-lab Docker host. VM on esh-pve. Covered by vzdump at the hypervisor (3/3 on esh-pve after VM 108 retirement); this adds file-level restic with app-consistent DB dumps.

Cross-site writes to rest-server-ana at 10.250.50.70:8000/esh-docker-vm/.

What's backed up

Path	Purpose
/opt/docker	Compose + bind-mounted conf (~12 MB)
/etc	Host config
/root	Root's scripts / history / keys
/home	User home dirs
/var/lib/docker/volumes	HA / paperless / pgadmin / CWA / dockge / agents
/var/lib/restic/stage	DB dumps from pre-backup.sh

Critically excluded

• NFS mounts from 10.0.50.50 — /mnt/{backup,books,compose,documents}. Debian NAS protects these at its own layer.
• /mnt/backup/docker/esh-vm-docker/* — offen-sidecar tarballs on NFS; once restic proves itself for a week, retire those sidecars.
• Docker internals (backingFsBlockDev, metadata.db).
• Logs, pids, user caches.

Pre-backup hooks

Unique to this host: most containers don't bundle sqlite3, so pre-backup.sh runs sqlite3 and pg_dump from the host against the volume bind-mount paths. Simpler than building custom images for HA, pgadmin, and uptime-kuma.

Service	DB	Approach
paperless-ngx	external Postgres 10.0.50.60 / paperless-ng	host pg_dump
home-assistant	/var/lib/docker/.../homeassistant_v2.db	host sqlite3
pgadmin	/var/lib/docker/.../pgadmin4.db	host sqlite3
uptime-kuma	/var/lib/docker/.../kuma.db	host sqlite3
calibre-web-automated	/config/app.db inside container	in-container sqlite3 (it has the binary)

Host prerequisites

Needs sqlite3 and postgresql-client installed on esh-docker-vm:

ssh -t esh-docker-vm 'sudo apt update && sudo apt install -y sqlite3 postgresql-client restic'

(restic too — not installed on this host yet.)

Deploy (one-time setup)

1. Install restic + db-client tooling

ssh -t esh-docker-vm 'sudo apt update && sudo apt install -y restic sqlite3 postgresql-client'

2. Add esh-docker-vm entry on ana-docker rest-server

ssh ana-docker '
  docker run --rm httpd:2.4-alpine htpasswd -nbB esh-docker-vm "<NEW-PW>" \
    >> /mnt/backup/restic/repo/ana/.htpasswd
'
# no sudo — /mnt/backup is NFS with root_squash; dir is lkraven-owned

3. Install restic creds on esh-docker-vm

ssh -t esh-docker-vm 'sudo install -d -o root -g root -m 0700 /etc/restic /var/lib/restic'

# REST URL
ssh -t esh-docker-vm 'sudo bash -c "cat > /etc/restic/restic.env && chmod 600 /etc/restic/restic.env"'
# paste: RESTIC_REPOSITORY=rest:http://esh-docker-vm:<HTPASSWD-PW>@10.250.50.70:8000/esh-docker-vm/
# Enter, Ctrl-D

# Repo passphrase
ssh -t esh-docker-vm 'sudo bash -c "cat > /etc/restic/password && chmod 600 /etc/restic/password"'
# paste: <REPO-PASSPHRASE>
# Enter, Ctrl-D

# Install DB creds (from dbcreds.env.example — edit in a temp file first)
cp configs/restic/esh-docker-vm/dbcreds.env.example /tmp/dbcreds.env
${EDITOR:-vi} /tmp/dbcreds.env   # set PAPERLESS_PGPASS to real value
scp /tmp/dbcreds.env esh-docker-vm:/tmp/
ssh -t esh-docker-vm '
  sudo install -o root -g root -m 0600 /tmp/dbcreds.env /etc/restic/dbcreds.env &&
  rm /tmp/dbcreds.env
'
shred -u /tmp/dbcreds.env 2>/dev/null || rm -f /tmp/dbcreds.env

4. Init the repo

ssh -t esh-docker-vm '
  sudo bash -c "
    set -a; . /etc/restic/restic.env; set +a
    restic init
  "
'
# Type passphrase twice at the prompt — same as installed in /etc/restic/password.

5. Deploy pre-backup.sh

scp configs/restic/esh-docker-vm/pre-backup.sh esh-docker-vm:/tmp/pre-backup.sh
ssh -t esh-docker-vm '
  sudo install -o root -g root -m 0700 /tmp/pre-backup.sh /etc/restic/pre-backup.sh &&
  rm /tmp/pre-backup.sh
'

6. Test the hook by itself (without running the full backup)

ssh -t esh-docker-vm 'sudo /etc/restic/pre-backup.sh'
ssh -t esh-docker-vm 'sudo ls -lh /var/lib/restic/stage/'

Expected files:

• paperless.pg_dump — should be 1–50 MB depending on doc count
• home-assistant.sqlite3 — ~50 MB (matches live DB size)
• calibre-web-automated.app.db — ~250 KB
• pgadmin4.db — ~200 KB
• uptime-kuma.kuma.db — varies (history retention)

Any WARN: lines in the hook output indicate a block that was skipped — read them, debug one at a time.

7. Install resticprofile

ssh -t esh-docker-vm '
  curl -sfL https://raw.githubusercontent.com/creativeprojects/resticprofile/master/install.sh \
    | sudo sh -s -- -b /usr/local/bin &&
  /usr/local/bin/resticprofile version
'

8. Deploy profile

scp configs/restic/esh-docker-vm/profiles.yaml esh-docker-vm:/tmp/profiles.yaml
ssh -t esh-docker-vm '
  sudo install -o root -g root -m 0644 /tmp/profiles.yaml /etc/restic/profiles.yaml &&
  rm /tmp/profiles.yaml &&
  sudo resticprofile --config /etc/restic/profiles.yaml show
'

9. Enable timers + first backup

ssh -t esh-docker-vm '
  sudo resticprofile --config /etc/restic/profiles.yaml schedule --all &&
  sudo resticprofile --config /etc/restic/profiles.yaml backup --verbose
'

Expected first snapshot: ~1–2 GB (HA DB is the dominant file, plus some Docker volumes). If you see much more, inspect /home and /var/lib/docker/volumes for surprise content.

Restoring

ssh -t esh-docker-vm '
  sudo bash -c "
    set -a; . /etc/restic/restic.env; set +a
    RESTIC_PASSWORD_FILE=/etc/restic/password \
      restic restore --target /tmp/restore latest --path /opt/docker
  "
'

Follow-ups after restic is proven (separate session)

• Rotate paperless-ngx Postgres password. Currently paperless-ng — trivially weak. Update the DB, the compose, and dbcreds.env.
• Retire offen/docker-volume-backup sidecars on paperless-ngx + pgadmin stacks after ~1 week of clean restic runs. Delete the compose's sidecar services + the tarballs under /mnt/backup/docker/esh-vm-docker/.

Recreating the repo

Same pattern as ana-ml2 — see configs/restic/ana-ml2/README.md. Repo path on ana-docker is /mnt/backup/restic/repo/ana/esh-docker-vm/.