Every /etc/sudoers.d/infra-ops carries Defaults:infra-ops log_output, so sudo records each infra-ops session (command line, timing, stdin/stdout/stderr) under /var/log/sudo-io/XX/YY/ZZ, with no retention. fv-ml1 had 16 G and esh-docker-vm 15 G by 2026-10-04. Prime: "those audits should be archived and backed up, then cleared. That should be a regular thing."

What runs

Weekly, Sunday 05:40 PT: sudo-io-archive.timer → sudo-io-archive.service (nh3-dev user units, installed from this folder into ~/.config/systemd/user/). A failure pages infra-ops via althing-notify-failure@.
Collector sudo-io-archive (nh3-dev): for each host in hosts, asks the host to pack every session older than 30 days (SUDO_IO_DAYS), pulls the archive, checks sha256 + gzip integrity, keeps it as ~/archive/sudo-io/<host>/<host>-<UTC stamp>-older-than-30d.tar.gz (+ .sha256, mode 600), then tells the host to delete exactly what it archived. One ops-log line per host.
Host side sudo-io-archive-host.sh, streamed over ssh into sudo -n bash -s, phases pack <days>, commit <sha256>, status. Works under /var/log/sudo-io-archive/ (outbox, list, cutoff).

Safety properties

A session is eligible only when nothing in it changed after the cutoff, so running sessions are never touched.
The host deletes nothing until the collector has the archive and has matched its sha256. Removal is tar --remove-files over the same list the archive was built from (no rm on a variable; the list is refused if any entry is absolute or contains ..). A failed run leaves the host's logs and outbox; the next run re-ships it.
The archives are covered by nh3-dev's backup: VM 102 is in nh3-pve's nightly vzdump to pbs-ana, mirrored to pbs-nh3.

Run by hand

services/sudo-io-archive/sudo-io-archive                  # all hosts
services/sudo-io-archive/sudo-io-archive fv-ml1           # one host
ssh infra-ops@<ip> 'sudo -n bash -s -- status' < services/sudo-io-archive/sudo-io-archive-host.sh

Replay an archived session: untar it anywhere, then sudoreplay -d <dir> <XX/YY/ZZ> (as root).

Bulk data is not kept (Prime 2026-10-04: "we don't need to durably back up weights and rsyncs forever")

Almost all of the volume was DATA piped through sudo, not commands: a 14.45 G recording of sudo tar -C /tank/aimodels -cf - erp-tune-v6-nvfp4a16 on fv-ml1, a ~380 M sudo rsync --server --sender of the Home Assistant volume every night on esh-docker-vm, Worldtree volume rsyncs on corviduo-dev, a docker save. Three layers now keep it out:

Not recorded. Every infra-ops sudoers file carries a BULK_DATA exemption (rsync, tar, cat, dd, zstd, zcat, gzip, pv, docker save|export|cp): Defaults!BULK_DATA !log_output. sudo still logs who ran the command; it no longer records the bytes. playbooks/sudoers-infra-ops-bulk-no-log-output.yaml rolled it out to all 19 recording hosts on 2026-10-04 (controls on each: sudo id adds a session, sudo tar --version does not), and playbooks/bootstrap-infra-ops-user.yaml writes it for new hosts.
Not archived. pack leaves out any stdout/stdin/tty stream over 64 MiB (the session's log, log.json, timing and stderr are still archived) and commit deletes it with the rest. Proven with a labelled 70 MB test session (TE/ST/OV in the 2026-10-04 pfi-pteradactyl archive).
Pruned once (2026-10-04). The 36 existing bulk streams on hosts (≈31 G: fv-ml1 15 G, esh-docker-vm 15 G, corviduo-dev, nh3-dev, nh3-docker) were deleted with literal-path rm, keeping each session's metadata, and the three collected archives that held bulk streams were rebuilt without them (*-bulk-pruned.tar.gz; the store went from 4.8 G to 83 M).

Lessons kept in the scripts: pull the outbox WITHOUT sudo (sudo ≥ 1.9.14 truncated piped output, and log_output re-recorded the archive); read the host list on fd 3 (ssh swallowed it from stdin); GNU tar reads -X NUL-separated under --null and only honours it BEFORE -T. Each of those was caught by a check, not by luck.