diff --git a/README.md b/README.md index 50db1eb..4ee728b 100644 --- a/README.md +++ b/README.md @@ -145,9 +145,10 @@ Goal: **every Docker host + configs + every database** covered, not just VM imag | VM-level (Proxmox vzdump) | ✅ All running guests covered across pfi-pve / nh3-pve / esh-pve-nas; esh-pve has VMID 108 uncovered | | ana-docker restic (host files + DBs) | ✅ `configs/restic/ana-docker/` with pre-backup hooks for synapse / seafile / vaultwarden | | ana-ml2 restic | ❌ **Bare metal — no vzdump**, no restic yet. Highest-priority gap | -| nh3-docker restic | ❌ VM is vzdump'd but no file-level restic | -| esh-docker-vm restic | ❌ VM is vzdump'd; needs pre-backup hooks for paperless-postgres, home-assistant SQLite, calibre-web-automated, pgadmin | -| vm-esh-nas restic | ❌ Light setup needed (low RAM, few stacks) | +| nh3-docker restic | ✅ Light — no DB hooks needed | +| esh-docker-vm restic | ✅ With DB hooks for paperless-postgres (external), home-assistant + pgadmin + uptime-kuma (host-side sqlite3), calibre-web-automated (in-container sqlite3) | +| vm-esh-nas restic | ✅ Light — NFS mounts explicitly excluded | +| nh3-dev (workstation) restic | ✅ `/home/lkraven` + `/etc` with language-toolchain and build-output excludes | | DB dumps: mattermost, openwebui, gitea, beszel-hub | ❌ Pre-backup hooks not written yet | | Cross-site redundancy | ❌ rsync between rest-server-ana ↔ rest-server-nh3 planned | | Prune ceremony | ❌ `scripts/restic-prune.sh` planned (rest-server `--append-only` blocks direct prune) | diff --git a/configs/restic/nh3-dev/README.md b/configs/restic/nh3-dev/README.md new file mode 100644 index 0000000..0f06ae2 --- /dev/null +++ b/configs/restic/nh3-dev/README.md @@ -0,0 +1,155 @@ +# restic / nh3-dev + +**User workstation** at the NH3 site (`nh3-dev.phasefinal.com`). Not a +server — active dev machine with ~12 GB of code in `~/development` that +wasn't being captured by any fleet backup. + +Writes site-local to the Synology rest-server (`10.100.50.50:8000`) as +user `nh3-dev`. Same target as nh3-docker. + +## What's backed up + +| Path | Purpose | +|---|---| +| `/home/lkraven` | Dev code, dotfiles, shell history, notes, Claude Code memory, scripts | +| `/etc` | Host config | +| `/root` | Root's state (ssh keys, scripts) | + +## What's excluded + +Roughly 8 GB of regenerable caches/build-outputs: + +- Language toolchain caches: `.cache`, `.rustup`, `.cargo/registry`, + `.cargo/git`, `.npm`, `.pnpm-store`, `.m2`, `.gradle`, `.conda`, + `go/pkg`, `.dotnet` +- Editor caches: `.vscode-server`, `.vscode` +- Project build output: `node_modules`, `__pycache__`, `.venv`, `venv`, + `target`, `dist`, `build`, `.pytest_cache`, `.tox`, `.next`, `.nuxt` +- Trash, browser caches, Steam + +Expected first snapshot: **~14 GB**. Incrementals should be small once +dedup kicks in. + +## Deploy + +### 1. Install resticprofile (restic already present) + +```bash +sudo bash -c " + curl -sfL https://raw.githubusercontent.com/creativeprojects/resticprofile/master/install.sh \ + | sh -s -- -b /usr/local/bin +" +/usr/local/bin/resticprofile version +``` + +### 2. Add `nh3-dev` entry on the Synology rest-server `.htpasswd` + +The Synology-side `.htpasswd` for `rest-server-nh3` lives wherever its +DATA_DIR points (see `stacks/rest-server-nh3/README.md` — probably +`/volume1/Backup/restic/.htpasswd` or similar). + +Generate the bcrypt line from this workstation using the same pattern +we used for ana side: + +```bash +# Generate locally, print the hash line +docker run --rm httpd:2.4-alpine htpasswd -nbB nh3-dev "" +# Copy the output line to clipboard +``` + +Then append it on the Synology. Options in order of friction: + +- **DSM File Station**: navigate to the restic data dir → open `.htpasswd` + in the built-in text editor → paste the line → save. +- **DSM Container Manager**: open the rest-server-nh3 container's + Terminal tab → `echo '' >> /data/.htpasswd` (exact path + matches the container's volume mount for DATA_DIR). +- **Once SSH to the Synology is set up** (tabled earlier): ssh + + `docker exec rest-server-nh3 sh -c '...'` or direct file append. + +### 3. Install restic creds on nh3-dev (local workstation) + +```bash +sudo install -d -o root -g root -m 0700 /etc/restic /var/lib/restic + +# REST URL +sudo bash -c "cat > /etc/restic/restic.env && chmod 600 /etc/restic/restic.env" +# paste: RESTIC_REPOSITORY=rest:http://nh3-dev:@10.100.50.50:8000/nh3-dev/ +# Enter, Ctrl-D + +# Passphrase (generate in password manager first, then paste) +sudo bash -c "cat > /etc/restic/password && chmod 600 /etc/restic/password" +# paste: +# Enter, Ctrl-D +``` + +### 4. Init the repo + +```bash +sudo bash -c ' + set -a; . /etc/restic/restic.env; set +a + restic init +' +# prompt for passphrase twice — paste the same one you just installed +``` + +If you get `config file already exists`, a repo was created in an earlier +session. Either use the existing passphrase (overwrite +`/etc/restic/password` with it) or wipe and reinit — same dance as we +did for ana-ml2 and esh-docker-vm. + +### 5. Verify + +```bash +sudo bash -c ' + set -a; . /etc/restic/restic.env; set +a + RESTIC_PASSWORD_FILE=/etc/restic/password restic snapshots +' +# expect: no snapshots found +``` + +### 6. Deploy profile + +```bash +sudo install -o root -g root -m 0644 \ + /home/lkraven/development/eshpfi-management/configs/restic/nh3-dev/profiles.yaml \ + /etc/restic/profiles.yaml + +sudo resticprofile --config /etc/restic/profiles.yaml show +``` + +### 7. Schedule + first backup + +```bash +sudo resticprofile --config /etc/restic/profiles.yaml schedule --all +systemctl list-timers "resticprofile*" + +# First backup — this will take a while (14 GB, initial dedup work) +sudo resticprofile --config /etc/restic/profiles.yaml backup --verbose +``` + +Expect the first run to take 5-15 minutes depending on NH3 LAN speed +and how much of `/home/lkraven/development` has high-entropy +(compressed) content. + +## Restoring + +```bash +sudo bash -c ' + set -a; . /etc/restic/restic.env; set +a + RESTIC_PASSWORD_FILE=/etc/restic/password \ + restic restore --target /tmp/restore latest --path /home/lkraven/development +' +``` + +## Watch items after first run + +- **First snapshot size** — if much above 14 GB, something large slipped + past excludes. Inspect with `restic stats latest --host nh3-dev`. +- **Schedule catches missed runs** — the workstation is frequently off at + 01:00. systemd's `Persistent=true` (resticprofile sets this by default + for system-level schedules) catches up after boot. After a few days of + normal use, confirm timer history with: + ```bash + journalctl -u "resticprofile-backup@profile-default.service" --no-pager -n 20 + ``` diff --git a/configs/restic/nh3-dev/profiles.yaml b/configs/restic/nh3-dev/profiles.yaml new file mode 100644 index 0000000..9e1a17e --- /dev/null +++ b/configs/restic/nh3-dev/profiles.yaml @@ -0,0 +1,115 @@ +# resticprofile config for nh3-dev. +# +# User workstation at the NH3 site (nh3-dev.phasefinal.com). Primary +# payload is /home/lkraven — ~12 GB of active development code, shell +# history, dotfiles, notes, Claude Code memory, etc. Not a server; no +# Docker stacks to dump. No databases. +# +# Writes to the Synology rest-server at 10.100.50.50:8000 as user +# `nh3-dev` — site-local, matches the pattern nh3-docker uses. +# +# Schedule accommodates a workstation that may be off/sleeping at the +# scheduled time: systemd timer uses Persistent=true (resticprofile sets +# this by default for schedule-permission: system) so a missed run +# catches up on next boot. +# +# Excludes trim ~8 GB of regenerable caches from the 22 GB home. First +# snapshot should land at ~14 GB; incrementals much smaller. + +version: "1" + +global: + priority: low + ionice: true + ionice-class: 2 + ionice-level: 7 + min-memory: 200 # workstation has more RAM than agents; give restic a bit more + +default: + env-file: /etc/restic/restic.env + env: + RESTIC_PASSWORD_FILE: /etc/restic/password + initialize: false + lock: /var/lock/restic-nh3-dev.lock + + backup: + verbose: 1 + run-after: + - date +%s > /var/lib/restic/last-success + source: + - /home/lkraven # the whole reason this backup exists + - /etc # host config + - /root # root's state, ssh keys, etc. + exclude: + # ---- Language / toolchain caches (all regenerable) ------------------- + - /home/lkraven/.cache + - /home/lkraven/.rustup + - /home/lkraven/.cargo/registry + - /home/lkraven/.cargo/git + - /home/lkraven/.npm + - /home/lkraven/.pnpm-store + - /home/lkraven/.m2 + - /home/lkraven/.gradle + - /home/lkraven/.conda + - /home/lkraven/go/pkg + - /home/lkraven/.dotnet + # ---- Editor / remote-dev caches --------------------------------------- + - /home/lkraven/.vscode-server + - /home/lkraven/.vscode + # ---- Project build outputs (often committed to .gitignore) ------------ + - "**/node_modules" + - "**/__pycache__" + - "**/.venv" + - "**/venv" + - "**/target" # Rust/Java/other — regenerable from source + - "**/dist" + - "**/build" + - "**/.pytest_cache" + - "**/.tox" + - "**/.next" # Next.js + - "**/.nuxt" # Nuxt + # ---- Trash / shell noise --------------------------------------------- + - /home/lkraven/.local/share/Trash + - /home/lkraven/.mozilla/firefox/*/Cache* + - /home/lkraven/.thumbnails + - /home/lkraven/.zcompdump* + # ---- Browser profiles (if any) ---------------------------------------- + - /home/lkraven/snap + - /home/lkraven/.steam + - /home/lkraven/Steam + # ---- Root side --------------------------------------------------------- + - /root/.cache + - /root/.npm + - /root/.local/share/Trash + # ---- Logs / pids ------------------------------------------------------ + - "**/*.log" + - "**/*.log.*" + - "**/*.pid" + tag: + - host:nh3-dev + - site:nh3 + - fleet:pfi + - role:workstation + schedule: "*-*-* 01:00:00" + schedule-permission: system + schedule-log: /var/log/restic-backup.log + + forget: + keep-daily: 7 + keep-weekly: 4 + keep-monthly: 12 + keep-yearly: 3 + # NOTE: forget against an --append-only rest-server FAILS (delete ops + # blocked). Schedule left in for when the prune ceremony flips + # --append-only off temporarily. See open thread. + tag: + - host:nh3-dev + schedule: "*-*-* 03:00:00" + schedule-permission: system + schedule-log: /var/log/restic-forget.log + + check: + read-data-subset: 10% + schedule: "Sun *-*-* 05:00:00" + schedule-permission: system + schedule-log: /var/log/restic-check.log