esh-pfi-infrastructure/configs/rsync/nh3-nas-to-ana

nh3-nas → ana-nas restic mirror (return leg)

Symmetric counterpart to configs/rsync/ana-nas-to-nh3/. Nightly rsync of the NH3-side restic repo to the Ana NAS, so file-level backups for nh3-docker and nh3-dev have cross-site redundancy.

What + where

Source	nh3-nas:/volume1/Backup/restic/ (Btrfs on cachedev_0, written by rest-server-nh3 on nh3-nas). Holds per-client repos: nh3-docker/, nh3-dev/, repo/.
Target	ana-nas:/mnt/backup/restic-nh3-mirror/ (ZFS NASPool/backupStore, 20 TB pool with ~20 TB free at setup).
Runs on	nh3-nas (Synology RS2418+, DSM 7+ with native systemd).
Runs as	root — required because rest-server-nh3's Docker container writes restic files as admin:users mode 400, and only root (via CAP_DAC_READ_SEARCH) can read them. Running as syncuser was tried first and denied. See gotchas below.
Auth	Dedicated ed25519 keypair /root/.ssh/id_mirror_ana → lkraven@ana-nas. No passphrase.
Schedule	Daily at 05:00, Persistent=true, 300s randomized delay.

Why 05:00

Slots between:

• 04:00 ana-nas → nh3-nas (the other mirror) — lets that finish before NH3 starts pushing anything back.
• 06:00 PBS-ANA → PBS-NH3 sync — avoids WAN contention with the much larger PBS replication.

DSM-specific notes

• DSM 7.x ships a stripped/older systemd. Unit files in /etc/systemd/system/ persist across DSM updates but are NOT guaranteed to — verify after each DSM major update.
• DSM systemd is pre-v247 — it refuses Restart=on-failure on Type=oneshot services, and does not recognize StartLimitBurst or StartLimitIntervalSec. This service deliberately omits those; a failed run just means the daily timer picks up again tomorrow. For a mirror used as DR (not primary), that's acceptable.
• systemctl --now flag is also absent. Use enable + start separately on DSM.
• syncuser has administrators group membership, so sudo works for unit installation. syncuser itself runs the service (no privilege escalation at runtime).
• rsync lives at /usr/bin/rsync on DSM (same as Debian). Older /bin/rsync doesn't exist.
• scp to DSM needs -O (legacy protocol) — sftp subsystem is off by default. Either enable SFTP in DSM (Control Panel → File Services → FTP → SFTP) or keep using scp -O.
• rest-server-nh3 writes restic files as admin:users mode 400 (via DSM ContainerManager). syncuser cannot read these even with admins-group ACL because the newer DSM doesn't ship synoacltool to add per-user ACEs. The service therefore runs as root (CAP_DAC_READ_SEARCH bypasses the 400 mode).
• rsync --archive broke cross-filesystem perm/owner preservation from Btrfs-with-syno_acl → ZFS-on-Debian — the dest dir ended up mode 0000 and rsync couldn't recover on subsequent runs. Switched to -rlt --no-perms --no-owner --no-group --chmod=D755,F644 which copies contents verbatim and forces sensible dest perms. Restic's repo integrity doesn't rely on Unix perms so this is a clean swap.

Files

File	Install path on nh3-nas
restic-mirror-to-ana.service	/etc/systemd/system/restic-mirror-to-ana.service
restic-mirror-to-ana.timer	/etc/systemd/system/restic-mirror-to-ana.timer

Prereqs (one-time setup)

1. Generate the keypair on nh3-nas as syncuser:

   ssh nh3-nas "test -f ~/.ssh/id_mirror_ana || ssh-keygen -t ed25519 -N '' -f ~/.ssh/id_mirror_ana -C 'nh3-nas→ana-nas restic mirror'"
   

2. Install the pubkey on ana-nas:

   ssh nh3-nas 'cat ~/.ssh/id_mirror_ana.pub' | \
     ssh ana-nas 'cat >> ~/.ssh/authorized_keys'
   

3. Create the destination dir on ana-nas:

   ssh ana-nas 'sudo mkdir -p /mnt/backup/restic-nh3-mirror && \
                sudo chown lkraven:lkraven /mnt/backup/restic-nh3-mirror'
   

4. Smoke-test from nh3-nas as syncuser:

   ssh nh3-nas 'ssh -i ~/.ssh/id_mirror_ana -o StrictHostKeyChecking=accept-new \
                  lkraven@10.250.50.50 "hostname && ls -ld /mnt/backup/restic-nh3-mirror"'
   

Deploy

cd configs/rsync/nh3-nas-to-ana
scp restic-mirror-to-ana.{service,timer} nh3-nas:/tmp/
ssh -t nh3-nas 'sudo install -m 644 /tmp/restic-mirror-to-ana.service /etc/systemd/system/ && \
                sudo install -m 644 /tmp/restic-mirror-to-ana.timer   /etc/systemd/system/ && \
                sudo systemctl daemon-reload && \
                sudo systemctl enable --now restic-mirror-to-ana.timer && \
                sudo systemctl start restic-mirror-to-ana.service && \
                sudo systemctl list-timers restic-mirror-to-ana.timer'

Operate

Mirrors the ana-nas side — see that README. Replace restic-mirror-to-nh3 with restic-mirror-to-ana and ana-nas with nh3-nas in the commands there.