From f5cc60bcfc894e367a1d435a89e5df6148844313 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 20 Apr 2026 23:26:17 -0700 Subject: [PATCH] docs: document rest-server-ana data layout + repo-recreate flow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit rest-server-ana README now describes the /mnt/backup/restic/repo/ top-level NFS mount and its three per-site subdirs: ana/ — live data served by this rest-server (per-host repos + .htpasswd) — what DATA_DIR points at esh/ — mirror destination for ESH-site backups (pending) nh3/ — mirror destination for NH3 Synology's tree (pending) ana-ml2 README gains a proper "Recreating the repo" section with the correct /mnt/backup/restic/repo/ana/ana-ml2/ path for wiping the old repo after a lost passphrase, and two paths for regenerating keys: - interactive: type a user-generated passphrase at restic's init prompt, then install it into /etc/restic/password via `cat > file` + Ctrl-D (no shell history or transcript exposure) - scripted: openssl rand -base64 48, passphrase prints once and must be captured into the password manager immediately Cross-site replication snippet in rest-server-ana README updated to use the unified /mnt/backup/restic/repo/{esh,nh3}/ destinations instead of the earlier restic-mirror-*/ staging paths. --- configs/restic/ana-ml2/README.md | 86 ++++++++++++++++++++++++++++---- stacks/rest-server-ana/README.md | 33 +++++++++--- 2 files changed, 102 insertions(+), 17 deletions(-) diff --git a/configs/restic/ana-ml2/README.md b/configs/restic/ana-ml2/README.md index 15fe275..5cbe6a7 100644 --- a/configs/restic/ana-ml2/README.md +++ b/configs/restic/ana-ml2/README.md @@ -87,27 +87,91 @@ manager or rotate (see "Recreating the repo" below). **Do not run `restic init`** — the repo is already initialized; init against an existing repo errors out deliberately. -### If no repo exists yet (fresh setup) +### Recreating the repo (lost passphrase / starting fresh) -Only applicable if you're starting from zero — otherwise skip. +If `restic snapshots` returns `wrong password or no key found` and the +passphrase can't be recovered, the old repo's data is unrecoverable and +the cleanest path is to wipe and reinit. If you're truly starting from +zero (no prior htpasswd entry or repo at all), skip the +`htpasswd` step — only the init is needed. + +The rest-server's data root is `/mnt/backup/restic/repo/ana/` on +ana-docker (NFS-mounted from the Debian NAS at 10.250.50.50). Per-host +repos are direct subdirs — for `ana-ml2` that's +`/mnt/backup/restic/repo/ana/ana-ml2/`. See +`stacks/rest-server-ana/README.md` for the full sibling layout +(`ana/`, `esh/`, `nh3/`). ```bash +# 1. Wipe the old repo content on ana-docker. `--append-only` blocks +# restic's own delete ops but not direct filesystem removal — this +# is intentional for exactly this case. ssh -t ana-docker ' - sudo htpasswd -B /path/to/rest-server/data/.htpasswd ana-ml2 && # prompts for password - docker restart rest-server + sudo rm -rf /mnt/backup/restic/repo/ana/ana-ml2 && + sudo docker restart rest-server ' -ssh -t ana-ml2 ' - # Write /etc/restic/restic.env and /etc/restic/password as above, using - # the fresh htpasswd password and a freshly generated passphrase: - openssl rand -base64 48 # save this in your password manager IMMEDIATELY +# 2. Optional: rotate the htpasswd password for ana-ml2 at the same time. +# Produces a new hash; replace in place. (Skip if the existing +# htpasswd password is still trusted.) +ssh ana-docker 'docker run --rm httpd:2.4-alpine htpasswd -nbB ana-ml2 ""' \ + | ssh -t ana-docker 'sudo sed -i "/^ana-ml2:/d" /mnt/backup/restic/repo/ana/.htpasswd && sudo tee -a /mnt/backup/restic/repo/ana/.htpasswd >/dev/null' - sudo env $(cat /etc/restic/restic.env) \ - RESTIC_PASSWORD_FILE=/etc/restic/password \ - restic init +# 3a. Init the repo — interactively prompt for a passphrase you generated +# in your password manager ahead of time. restic asks twice (init + +# confirm). This keeps the passphrase out of shell history / transcripts. +ssh -t ana-ml2 ' + sudo bash -c " + set -a + . /etc/restic/restic.env + set +a + restic init + " +' + +# 3b. Install the same passphrase into /etc/restic/password so timers can +# run unattended. `cat > file` + Ctrl-D avoids the passphrase ever +# landing in your shell history or the ssh command line. +ssh -t ana-ml2 'sudo bash -c "cat > /etc/restic/password && chmod 600 /etc/restic/password"' +# Terminal waits for input: +# - paste the same passphrase +# - press Enter +# - press Ctrl-D on an empty line +# (restic strips the trailing newline when reading the file.) + +# 3c. Verify the file-based passphrase agrees with what init registered. +ssh -t ana-ml2 ' + sudo bash -c " + set -a + . /etc/restic/restic.env + set +a + RESTIC_PASSWORD_FILE=/etc/restic/password restic snapshots + " +' +# Expect: "no snapshots found" — repo is initialized and both paths +# (interactive and file-based) decrypt it. +``` + +**Alternative (random machine-generated passphrase):** if you want restic +to generate the passphrase rather than supplying one from your password +manager, replace 3a/3b with: + +```bash +ssh -t ana-ml2 ' + sudo bash -c " + openssl rand -base64 48 | tr -d \"\\n\" > /etc/restic/password + chmod 600 /etc/restic/password + cat /etc/restic/password # copy this to your password manager NOW + echo + set -a; . /etc/restic/restic.env; set +a + RESTIC_PASSWORD_FILE=/etc/restic/password restic init + " ' ``` +The passphrase prints to the terminal exactly once — don't close the +window before copying it. Losing it = losing this repo. + ### 3. Install resticprofile on ana-ml2 The host has `restic` but not `resticprofile`. From the resticprofile diff --git a/stacks/rest-server-ana/README.md b/stacks/rest-server-ana/README.md index d5ffe52..5ca49d2 100644 --- a/stacks/rest-server-ana/README.md +++ b/stacks/rest-server-ana/README.md @@ -4,7 +4,21 @@ Anaheim-site restic backup endpoint. Replaces the older `restic` stack on ana-do **Server:** ana-docker (`10.250.50.70`) **Port:** `http://10.250.50.70:8000` -**Data:** `/mnt/backup/restic/repo/ana/` (NFS mount on the host, served by the Debian file server at `10.250.50.50`) +**Data root:** `/mnt/backup/restic/repo/` (NFS mount on the host, served by the Debian file server at `10.250.50.50`) + +## Data layout + +The NFS export at `/mnt/backup/restic/repo/` is partitioned by site into three sibling subdirs: + +| Path | Role | +|---|---| +| `/mnt/backup/restic/repo/ana/` | **Live data served by this rest-server.** Each Anaheim-side host writes its repo under here (e.g. `ana/ana-docker/`, `ana/ana-ml2/`). `.htpasswd` also lives here. | +| `/mnt/backup/restic/repo/esh/` | Destination for mirroring ESH-site backups to Anaheim (cross-site redundancy; not yet wired up). | +| `/mnt/backup/restic/repo/nh3/` | Destination for mirroring NH3-site backups (from the Synology `rest-server-nh3`) into Anaheim. Same — pending. | + +The rest-server container is configured with `/mnt/backup/restic/repo/ana` as its data root (see compose's `DATA_DIR` env), so clients use URLs of the form `rest:http://user:pw@10.250.50.70:8000//` — the `` path is relative to `ana/`. + +The two mirror dirs (`esh/`, `nh3/`) aren't served by restic at all; they're rsync destinations. Paired with: - **`rest-server-nh3`** on the Synology (`10.100.50.50:8000`, data on Btrfs). @@ -131,13 +145,20 @@ If you go the second-endpoint route, copy this stack to `stacks/rest-server-ana- ## Off-site replication -Scheduled on ana-docker (to be written): +Scheduled on ana-docker (to be written). Mirror destinations live in the +sibling subdirs described under "Data layout" above: ```bash -# Pull NH3's tree down to this side -rsync -avz --delete admin@10.100.50.50:/volume1/Backup/restic/ /mnt/backup/restic-mirror-nh3/ -# Push our tree to NH3 +# Pull NH3's tree into ana-docker's /mnt/backup/restic/repo/nh3/ +rsync -avz --delete admin@10.100.50.50:/volume1/Backup/restic/ /mnt/backup/restic/repo/nh3/ + +# Pull the ESH rest-server's tree into /mnt/backup/restic/repo/esh/ +# (once an ESH rest-server is set up — currently ESH hosts write directly +# to this Anaheim rest-server at ana/esh-*) + +# Push our tree (ana/) to NH3 for redundancy in the other direction rsync -avz --delete /mnt/backup/restic/repo/ana/ admin@10.100.50.50:/volume1/Backup/restic-mirror-ana/ ``` -Two unidirectional syncs, each running in the direction its data flows. Prune runs only at the origin so the mirror shrinks correctly. +Each sync unidirectional, running in the direction its data flows. Prune +runs only at the origin so the mirror shrinks correctly.