feat(backups): assert restic CONTENT, and add the host that was never checked

Operator ruling 2026-09-22 (relayed via svos-dev): have the freshness check
assert snapshot content rather than REST-server reachability, so the green
light is a statement about DATA rather than about a daemon.

⚠ FIRST, A CORRECTION TO MY OWN REPORT. I ran `grep -ic restic` against
backup-freshness-alert.sh -- the WRAPPER -- got 0, and told the operator that
nothing on the fleet verified restic. Wrong. check-backup-freshness.sh has
always checked restic snapshot ages, for seven hosts. I grepped one file of a
two-file tool and generalised, which is the identical error a peer had just
made with dev-backup, made while correcting them.

THE REAL GAP was narrower and still real: the NH3 host list read
`irv-ml1 nh3-docker` and omitted nh3-dev -- the repo holding every Claude Code
session transcript, althing routes, hermes run history and Miranda's
conversation. /volume1/Backup/restic/nh3-dev/ has always existed and always
been written nightly; it was simply never enumerated, so a total failure of the
one repo nobody could reconstruct would have gone unreported indefinitely.

THE CONTENT ASSERTION, built on svos-dev's ladder (list proves the repo
answers; ls proves the index; check proves structure; RESTORE proves the bytes
come back):
  - CONJUNCTIVE: latest snapshot newer than MAX_AGE_H AND the probe path
    present AND a restore returns non-zero bytes. "A snapshot exists containing
    X" is satisfied by a three-month-old one; "the latest is recent" is
    satisfied by an empty one. Age alone was the old problem; content alone is
    the same problem rotated.
  - IT RESTORES. Metadata is what survives the failures worth fearing -- a
    pruned or partially-written repo can list a path whose blobs are gone.
    ~493KB from ~/.local/state/svos, sub-second. Large paths are not restored:
    a repo that returns one file will return others.
  - IDENTITY: the timer runs as lkraven, which has no NOPASSWD sudo on nh3-dev,
    so the probe hops through infra-ops@localhost. /etc/restic is root-only by
    design.

⚠ restic ls ALWAYS PRINTS A HEADER LINE, matched or not. A path absent from the
repo returns 1 line; a real one returned 6. Counting with `grep -c .` read the
header as a hit, so the ABSENT case fell through and reported "blobs gone" --
telling an operator the repository was corrupt when the truth was a mistyped
path. Now `grep -c '^/'`.

TESTED BY MAKING IT FAIL, because a check only ever seen passing is untested:
bogus probe path -> "absent from snapshot"; BACKUP_MAX_AGE_HOURS=1 -> "12h old
(>1h)"; healthy -> snapshot id, age, entries, bytes restored.

scripts/check-backup-freshness.sh

@@ -133,7 +133,14 @@ for c in ana-docker ana-ml2 esh-docker-vm esh-vm-db vm-esh-nas; do
   report "restic/ana/$c" "$(newest_epoch ana-nas "/mnt/backup/restic/repo/ana/$c/snapshots/*")"
 done
 # --- Layer: restic file+DB, NH3 side (rest-server-nh3) ---
-for c in irv-ml1 nh3-docker; do
+# ⚠ nh3-dev WAS MISSING FROM THIS LIST until 2026-09-22, and it is the host
+# whose restic repo holds the most irreplaceable data on the fleet: every
+# Claude Code session transcript, althing routes, hermes run history, and
+# Miranda's conversation. Its repo has always existed at
+# /volume1/Backup/restic/nh3-dev/ and has always been written nightly -- it was
+# simply never enumerated here, so a total failure of the one repo nobody could
+# reconstruct would have gone unreported indefinitely.
+for c in irv-ml1 nh3-docker nh3-dev; do
   report "restic/nh3/$c" "$(newest_epoch nh3-nas "/volume1/Backup/restic/$c/snapshots/*")"
 done
 # --- Layer: PBS VM images (newest per guest, all namespaces) ---
@@ -192,6 +199,76 @@ for entry in "${PVE_NODES[@]}"; do
   done <<<"$bad"
 done
 
+# --- restic CONTENT assertion (operator ruling 2026-09-22) -------------------
+#
+# The checks above read a snapshot directory's MTIME on the NAS. That proves a
+# file was written recently. It does not prove the repository holds the data,
+# and metadata is exactly what survives the failures worth fearing: a pruned or
+# partially-written repo can still list a path whose blobs are gone.
+#
+# ⚠ THE ASSERTION IS CONJUNCTIVE ON PURPOSE. "a snapshot exists containing X"
+# is satisfied by a three-month-old snapshot; "the latest snapshot is recent" is
+# satisfied by an empty one. Age alone was the old check's problem; content
+# alone is the same problem rotated. Both, or neither is worth printing.
+#
+# The rungs, and what each actually proves (svos-dev, 2026-09-22):
+#     snapshots list    the repo answers
+#     ls a path in it   the path is in the INDEX  <- metadata only
+#     restic check      the repo's structure is intact
+#     RESTORE a file    the bytes come back       <- the only proof of recovery
+# We take the top rung for the small irreplaceable set, because it is the only
+# one that answers the question a green light is taken to mean. Restoring from
+# the multi-gigabyte paths is not worth it: a repo that returns one file is
+# overwhelmingly likely to return others, and one that cannot is broken for
+# everything.
+#
+# ⚠ IDENTITY: this runs as a --user timer (lkraven), which has no NOPASSWD sudo
+# on nh3-dev, and the repo credentials live under /etc/restic. So the probe hops
+# through infra-ops@localhost, which does. Not a workaround -- infra-ops is the
+# documented ops identity and the credentials are deliberately root-only.
+RESTIC_PROBE_PATHS="${RESTIC_PROBE_PATHS:-/home/lkraven/.local/state/svos}"
+probe=$($SSH infra-ops@10.100.10.50 "sudo -n bash -s" <<PROBE 2>/dev/null
+set -a; . /etc/restic/restic.env 2>/dev/null; set +a
+export RESTIC_PASSWORD_FILE=/etc/restic/password
+id=\$(restic snapshots --latest 1 --json 2>/dev/null | python3 -c 'import json,sys
+d=json.load(sys.stdin)
+print(d[0]["short_id"], int(__import__("datetime").datetime.fromisoformat(d[0]["time"][:19]).timestamp())) if d else print("NONE 0")' 2>/dev/null)
+sid=\$(echo "\$id" | cut -d" " -f1); ep=\$(echo "\$id" | cut -d" " -f2)
+[ "\$sid" = "NONE" ] && { echo "NOSNAP 0 0 0"; exit 0; }
+# ⚠ grep -c '^/' NOT grep -c '.' — \`restic ls\` always prints a HEADER line
+# ("snapshot <id> of [...] filtered by [...]") whether or not anything matched,
+# so a path absent from the repo returns 1 line and a naive count reads it as
+# found. Measured 2026-09-22: real path 6 lines, bogus path 1. That made the
+# ABSENT case fall through and report "blobs gone" — telling an operator the
+# repository was corrupt when the truth was a mistyped path. Only lines that
+# begin with / are entries.
+n=\$(restic ls "\$sid" $RESTIC_PROBE_PATHS 2>/dev/null | grep -c '^/')
+tmp=\$(mktemp -d); restic restore "\$sid" --target "\$tmp" --include $RESTIC_PROBE_PATHS >/dev/null 2>&1
+bytes=\$(find "\$tmp" -type f -printf '%s\n' 2>/dev/null | awk '{s+=\$1} END {print s+0}')
+rm -rf -- "\$tmp"
+echo "\$sid \$ep \$n \$bytes"
+PROBE
+)
+set -- ${probe:-ERR 0 0 0}
+r_sid="$1"; r_ep="$2"; r_entries="$3"; r_bytes="$4"
+if [ "$r_sid" = "ERR" ] || [ -z "$probe" ]; then
+  stale+=("restic/nh3-dev CONTENT: probe could not run (repo unreachable or creds unreadable)")
+elif [ "$r_sid" = "NOSNAP" ]; then
+  stale+=("restic/nh3-dev CONTENT: repository holds NO SNAPSHOTS")
+else
+  r_age=$(( (now - r_ep) / 3600 ))
+  # conjunctive: recent AND indexed AND restorable
+  if [ "$r_age" -gt "$MAX_AGE_H" ]; then
+    stale+=("restic/nh3-dev CONTENT: latest snapshot $r_sid is ${r_age}h old (>${MAX_AGE_H}h)")
+  elif [ "${r_entries:-0}" -lt 1 ]; then
+    stale+=("restic/nh3-dev CONTENT: $RESTIC_PROBE_PATHS absent from snapshot $r_sid")
+  elif [ "${r_bytes:-0}" -lt 1 ]; then
+    stale+=("restic/nh3-dev CONTENT: $RESTIC_PROBE_PATHS listed in $r_sid but RESTORED ZERO BYTES — index intact, blobs gone")
+  else
+    fresh+=("restic/nh3-dev CONTENT: $r_sid ${r_age}h old, $r_entries entries, restored ${r_bytes}B")
+  fi
+fi
+
 # --- rest-server endpoint health (401 = up & serving) ---
 for ep in "rest-server-ana http://10.250.50.70:8000/" "rest-server-nh3 http://10.100.50.50:8000/"; do
   set -- $ep