Files
2026-04-15 21:06:58 -07:00

662 lines
18 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
created: '2026-04-16T04:06:58+00:00'
id: 8fc5d55f-1466-4414-860f-ae84c1f39ebf
keywords:
- restic
- backup
- AES-256-CTR
- Poly1305
- scrypt
- Rabin Fingerprints
- content-defined chunking
- deduplication
- S3
- B2
- Azure Blob
- Google Cloud Storage
- rclone
- REST server
- erasure coding
- zstandard
- Blake3
- snapshot
- prune
- forget
- append-only
links: []
modified: '2026-04-16T04:06:58+00:00'
summary: Comprehensive reference for restic 0.18.1 — a fast, secure, efficient backup
program written in Go. Covers installation, all storage backends, backup/restore/forget
workflows, encryption, deduplication, repository internals, environment variables,
and troubleshooting.
tags:
- backup
- restic
- encryption
- deduplication
- CLI
- reference
title: Restic Backup Reference
---
---
id: restic
title: Restic Backup Reference
created: 2025-07-10
modified: 2025-07-10
tags:
- backup
- restic
- encryption
- deduplication
- CLI
- reference
links:
- reference/docker-compose.md
- reference/crowdsec.md
---
# Restic — Backups Done Right
**Version:** 0.18.1
**License:** BSD 2-Clause
**Language:** Go
**URL:** https://restic.net/
**Docs:** https://restic.readthedocs.io/en/stable/
## Overview
Restic is a fast, secure, efficient backup program. It supports Linux, macOS, Windows, FreeBSD, OpenBSD. It is a single binary with no server component required. Key features:
- **Deduplication** — Content-defined chunking with Rabin Fingerprints; only changed data is transferred/stored
- **Encryption** — AES-256-CTR + Poly1305-AES MAC on all data; scrypt key derivation from user password
- **Compression** — zstandard compression (repository format v2, default since 0.14)
- **Verification** — SHA-256 hashing of all blobs; `restic check` validates integrity
- **Multiple backends** — Local, SFTP, REST, S3, Azure Blob, Google Cloud Storage, Backblaze B2, OpenStack Swift, rclone
- **No server required** — Single static binary; no daemon
## Installation
### Package Managers
| Platform | Command |
|---|---|
| Debian/Ubuntu | `apt-get install restic` |
| Fedora/RHEL/CentOS | `dnf install restic` (or via EPEL for RHEL 8/9) |
| Arch Linux | `pacman -S restic` |
| Alpine | `apk add restic` |
| macOS (Homebrew) | `brew install restic` |
| macOS (MacPorts) | `port install restic` |
| macOS (pkgx) | `pkgx install restic` |
| Nix/NixOS | `nix-env --install restic` |
| FreeBSD | `pkg install restic` |
| OpenBSD | `pkg_add restic` |
| openSUSE | `zypper install restic` |
| Windows (winget) | `winget install --exact --id restic.restic --scope Machine` |
| Windows (Scoop) | `scoop install restic` |
| Solus | `eopkg install restic` |
| Gentoo | `emerge restic` |
### Official Binaries
Download from https://github.com/restic/restic/releases — reproducible builds with GPG verification (key `0x91A6868BD3F7A907`). Update in-place with `restic self-update`.
### Docker
```bash
docker pull restic/restic
# or from GHCR:
docker pull ghcr.io/restic/restic
```
> **Note:** Set a static hostname with `--hostname` when running in Docker.
### From Source
Requires Go 1.23+:
```bash
git clone https://github.com/restic/restic && cd restic && go run build.go
```
Produces a statically linked binary. Cross-compilation supported.
### Autocompletion
```bash
restic generate --bash-completion /path/to/completion
restic generate --zsh-completion /path/to/completion
restic generate --fish-completion /path/to/completion
restic generate --man /path/to/manpages
restic generate --powershell-completion /path/to/completion
```
## Quickstart
```bash
# Set environment variables
export RESTIC_REPOSITORY=/srv/restic-repo
export RESTIC_PASSWORD=some-strong-password
# Initialize repository (first time only)
restic init
# Create backup
restic backup ~/work
# List snapshots
restic snapshots
# Restore
restic restore --target /tmp/restore-work your-snapshot-ID
# Verify integrity
restic check
restic check --read-data # full data verification
```
## Preparing a New Repository
### Repository Versions
| Version | Min restic | Major Features |
|---|---|---|
| `1` | Any | Original format |
| `2` | 0.14.0+ | Compression support (current default) |
Specify with `restic init --repository-version 2` or `--repository-version latest`.
### Password Management Options
- Environment variable: `RESTIC_PASSWORD`
- Password file: `--password-file` or `RESTIC_PASSWORD_FILE`
- Password command: `--password-command` or `RESTIC_PASSWORD_COMMAND`
- Interactive prompt (default)
### Backend Types
#### Local
```bash
restic init --repo /srv/restic-repo
```
#### SFTP
```bash
restic -r sftp:user@host:/srv/restic-repo init
# With port and IPv6:
restic -r sftp://user@[::1]:2222//srv/restic-repo init
# Use SSH config host aliases:
restic -r sftp:my-backup-host:/srv/restic-repo init
# Custom SFTP command:
restic -r sftp:foo:/repo -o sftp.command="foobar"
```
SSH keepalive recommended:
```
ServerAliveInterval 60
ServerAliveCountMax 240
```
#### REST Server
```bash
restic -r rest:http://host:8000/ init
restic -r rest:https://user:pass@host:8000/ init
restic -r rest:http+unix:///tmp/rest.socket:/repo init
```
TLS cert override: `--cacert`. Env vars: `RESTIC_REST_USERNAME`, `RESTIC_REST_PASSWORD`.
#### Amazon S3
```bash
export AWS_ACCESS_KEY_ID=<KEY>
export AWS_SECRET_ACCESS_KEY=<SECRET>
export AWS_SESSION_TOKEN=<TOKEN> # for temporary credentials
restic -r s3:s3.us-east-1.amazonaws.com/bucket_name init
```
> **Note:** restic uses **path-style URLs**, not virtual-hosted style.
#### S3-Compatible (Minio, Wasabi, Alibaba OSS, etc.)
```bash
export AWS_ACCESS_KEY_ID=<KEY>
export AWS_SECRET_ACCESS_KEY=<SECRET>
restic -r s3:https://server:port/bucket_name init
# Options:
# -o s3.region="us-east-1"
# -o s3.bucket-lookup=auto|dns|path
# -o s3.list-objects-v1=true # for old Ceph
```
#### Backblaze B2
```bash
# Recommended: use S3-compatible API (see B2 docs for S3 keys)
# Legacy B2 backend (less reliable error handling):
export B2_ACCOUNT_ID=<ID>
export B2_ACCOUNT_KEY=<KEY>
restic -r b2:bucketname:path/to/repo init
```
#### Microsoft Azure Blob Storage
```bash
export AZURE_ACCOUNT_NAME=<NAME>
export AZURE_ACCOUNT_KEY=<KEY> # or AZURE_ACCOUNT_SAS=<SAS>
# Or use: az login (needs Storage Blob Data Contributor role)
restic -r azure:container:/ init
# Options: -o azure.connections=10, -o azure.access-tier=Hot|Cool|Cold
```
#### Google Cloud Storage
```bash
export GOOGLE_PROJECT_ID=<ID>
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
restic -r gs:bucket:/ init
# Options: -o gs.connections=10, -o gs.region=us
```
#### OpenStack Swift
```bash
# Keystone v1/v2/v3 auth via environment variables
# (ST_AUTH, ST_USER, ST_KEY for v1; OS_AUTH_URL, OS_USERNAME, etc. for v2/v3)
restic -r swift:container_name:/path init
```
#### rclone (any supported remote)
```bash
restic -r rclone:remote_name:path init
# Options: -o rclone.program, -o rclone.args, -o rclone.timeout
# Remote rclone via SSH:
restic -o rclone.program="ssh user@remotehost rclone" -r rclone:b2:foo/bar
```
### Group-Accessible Repositories
Since v0.14, local and SFTP repos can be shared with a group:
```bash
# Read-only: chmod -R g+r /srv/restic-repo
# Read-write: requires root to set setgid
# find /srv/restic-repo -type d -exec chmod g+s '{}' \;
# chmod -R g+rw /srv/restic-repo
```
## Backing Up
### Basic Backup
```bash
restic -r /repo backup ~/work
restic -r /repo backup ~/work.txt # individual files
```
### File Change Detection
On Unix, a file is considered unchanged if **all** of these match a previous backup:
- Modification timestamp (mtime)
- Metadata change timestamp (ctime)
- File size
- Inode number
Options to modify:
- `--force` — rescan all files
- `--ignore-ctime` — only require mtime match
- `--ignore-inode` — allow inode and ctime to differ (for FUSE filesystems)
Parent snapshot selection defaults to `--group-by host,paths`.
### Skip Unchanged Snapshots
```bash
restic backup ~/work --skip-if-unchanged
```
### Dry Run
```bash
restic backup ~/work --dry-run -vv
```
### Excluding Files
Options:
- `--exclude "pattern"` / `--iexclude "pattern"` (case-insensitive)
- `--exclude-file file` / `--iexclude-file file`
- `--exclude-caches` — skip folders with CACHEDIR.TAG
- `--exclude-if-present foo` — skip folders containing file `foo`
- `--exclude-larger-than 1M` — skip files over size
- `--exclude-cloud-files` — skip cloud-only files (Windows)
Pattern syntax uses Go's `filepath.Match`:
- `foo` matches `/dir/foo` and `/dir/foo/file` but not `/dir/foobar`
- Leading `/` anchors to root
- `**` matches across directories
- `!pattern` cancels a previous match (like gitignore)
- Environment variables expanded with `$VAR`
- `--one-file-system` prevents crossing mount boundaries
### Including Files
```bash
restic backup --files-from /path/to/patterns
restic backup --files-from-verbatim /path/to/list # no glob expansion
restic backup --files-from-raw /path/to/list # NUL-delimited
```
### Comparing Snapshots
```bash
restic diff snapshot1 snapshot2
restic diff snapshot1:/subfolder snapshot2:/subfolder
restic diff snapshot1 snapshot2 --metadata
```
Status codes: `+` added, `-` removed, `T` type changed, `M` content changed, `U` metadata changed, `?` bitrot detected.
### Backing Up from Commands/Stdin
```bash
# Recommended: from command (detects failures)
restic backup --stdin-filename db.sql --stdin-from-command -- mysqldump --host example mydb
# Stdin pipe (masks command failures — use with set -o pipefail)
mysqldump mydb | restic backup --stdin --stdin-filename db.sql
```
### Tags
```bash
restic backup --tag projectX --tag foo ~/work
```
Tags used for snapshot selection with `forget` and `tag` commands.
### Windows VSS
```bash
restic backup C:\ --use-fs-snapshot
# Options:
# -o vss.timeout=2m30s
# -o vss.exclude-all-mount-points=true
# -o vss.exclude-volumes="d:;c:\mnt"
```
### Exit Status Codes
- `0` — success (snapshot created with all files)
- `1` — fatal error (no snapshot created)
- `3` — some source files unreadable (incomplete snapshot created)
### Scheduling Backups
Restic has no built-in scheduler. Options:
- **Linux/BSD:** systemd timers, cron
- **Windows:** Task Scheduler
- **Wrapper tools:** resticprofile
Always detect already-running instances before starting a backup.
## Environment Variables
| Variable | Purpose |
|---|---|
| `RESTIC_REPOSITORY` | Repository location |
| `RESTIC_REPOSITORY_FILE` | File containing repository location |
| `RESTIC_PASSWORD` | Repository password |
| `RESTIC_PASSWORD_FILE` | File containing password |
| `RESTIC_PASSWORD_COMMAND` | Command to print password |
| `RESTIC_CACHE_DIR` | Cache directory location |
| `RESTIC_COMPRESSION` | Compression mode (v2 only) |
| `RESTIC_HOST` | Set/override hostname for snapshots |
| `RESTIC_PACK_SIZE` | Target pack file size |
| `RESTIC_READ_CONCURRENCY` | File read concurrency |
| `RESTIC_PROGRESS_FPS` | Progress bar refresh rate |
| `RESTIC_CACERT` | TLS CA certificate(s) |
| `RESTIC_TLS_CLIENT_CERT` | TLS client certificate |
| `RESTIC_KEY_HINT` | Key ID to try first |
| `AWS_ACCESS_KEY_ID` | S3 access key |
| `AWS_SECRET_ACCESS_KEY` | S3 secret key |
| `AWS_SESSION_TOKEN` | S3 temporary session token |
| `AWS_DEFAULT_REGION` | S3 default region |
| `AWS_PROFILE` | AWS credentials profile |
| `AZURE_ACCOUNT_NAME` | Azure account name |
| `AZURE_ACCOUNT_KEY` | Azure account key |
| `AZURE_ACCOUNT_SAS` | Azure SAS token |
| `GOOGLE_PROJECT_ID` | GCP project ID |
| `GOOGLE_APPLICATION_CREDENTIALS` | GCP service account JSON key |
| `B2_ACCOUNT_ID` | Backblaze B2 account ID |
| `B2_ACCOUNT_KEY` | Backblaze B2 application key |
| `RESTIC_REST_USERNAME` | REST server username |
| `RESTIC_REST_PASSWORD` | REST server password |
## Working with Repositories
### Listing Snapshots
```bash
restic snapshots
restic snapshots --group-by host,paths # default
restic snapshots --host myhost --tag foo
```
### Listing Files in a Snapshot
```bash
restic ls latest
restic ls snapshot-id /path/within/snapshot
```
### Restoring
```bash
restic restore --target /tmp/restore snapshot-id
restic restore --target /tmp/restore --include "*.txt" snapshot-id
restic restore --target /tmp/restore --exclude "*.c" snapshot-id
# Mount as filesystem:
restic mount /mnt/restic
# Print to stdout:
restic dump snapshot-id path/to/file
```
### Copying Snapshots Between Repositories
```bash
restic copy --repo-copy /other-repo snapshot-id
```
### Removing Files from Snapshots
```bash
restic forget --keep-last 5 snapshot-id # remove from repo
restic rewrite --exclude "pattern" snapshot-id # create new snapshot without matching files
```
### Modifying Snapshot Metadata
```bash
restic tag --tag newtag snapshot-id
restic tag --remove tagname snapshot-id
```
### Checking Integrity
```bash
restic check # check metadata
restic check --read-data # verify all data
restic check --check-unused # find unused blobs
```
## Removing Backup Snapshots
### Manual Removal
```bash
restic forget snapshot-id
restic prune # cleanup unreferenced data
```
### Policy-Based Removal
```bash
restic forget --keep-last 7 --keep-daily 14 --keep-weekly 8 --keep-monthly 12 --keep-yearly 5 --prune
```
Policy options:
| Option | Description |
|---|---|
| `--keep-last N` | Keep last N snapshots |
| `--keep-hourly N` | Keep most recent snapshot for each of last N hours with snapshots |
| `--keep-daily N` | Keep most recent snapshot for each of last N days with snapshots |
| `--keep-weekly N` | Keep most recent snapshot for each of last N weeks |
| `--keep-monthly N` | Keep most recent snapshot for each of last N months |
| `--keep-yearly N` | Keep most recent snapshot for each of last N years |
| `--keep-tag TAG` | Keep all snapshots with specified tag(s) |
| `--keep-within DURATION` | Keep snapshots within duration of latest (e.g., `2y5m7d3h`) |
| `--keep-within-{hourly,daily,weekly,monthly,yearly} DURATION` | Duration-based variants |
Options work on natural time boundaries (weeks are Mon–Sun, hours are :00–:59). Results are ORed — a snapshot only needs to match one option to be kept. Grouped by `--group-by host,paths` by default.
All policies support `--dry-run`. `--unsafe-allow-remove-all` allows removing all snapshots for a specific filter (v0.17+).
### Append-Only Mode Security
When using append-only repositories (e.g., via rest-server), use `--keep-within` policies. Calendar-based policies (`--keep-daily`, etc.) can be gamed by attackers injecting fake snapshots. `--keep-within` keeps both legitimate and attacker snapshots, allowing detection.
### Customizing Pruning
```bash
restic prune # default: 5% max unused
restic prune --max-unused 200M # absolute size limit
restic prune --max-unused 10% # relative to repo size
restic prune --max-unused unlimited # never repack (fast, but wastes space)
restic prune --max-repack-size 10G # limit total repack size
restic prune --repack-cacheable-only # only repack cached metadata (fast)
restic prune --repack-small # repack packs below 80% target size
restic prune --repack-smaller-than 5M # repack packs below this size
```
### Recovering from "No Free Space"
```bash
# Step 1: forget some snapshots
restic forget --keep-last 1
# Step 2: minimal prune
restic prune --max-repack-size 0
# Step 3: if still stuck (last resort):
restic prune --unsafe-recover-no-free-space <ID>
```
## Tuning Backup Parameters
| Parameter | Option | Default |
|---|---|---|
| Progress estimation | `--no-scan` / `RESTIC_PROGRESS_FPS` | Enabled |
| Backend connections | `-o backend.connections=N` | 5 |
| CPU usage | `GOMAXPROCS=N` env var | All cores |
| Compression | `--compression auto\|max\|off` | `auto` |
| Data verification | `--insecure-no-verify` (dangerous) | Enabled |
| File read concurrency | `--read-concurrency N` or `RESTIC_READ_CONCURRENCY` | 2 |
| Pack size | `--pack-size N` or `RESTIC_PACK_SIZE` | 8 MiB |
| Feature flags | `-o feature.<name>=true` | — |
## Encryption
- **Algorithm:** AES-256-CTR + Poly1305-AES MAC
- **KDF:** scrypt (N=65536, r=8, p=1)
- **Key management:** Multiple keys supported; password change doesn't require re-encryption
- All data encrypted except informational metadata in key files
- Cache is also encrypted
### Key Management
```bash
restic key list
restic key add
restic key remove <ID>
restic key passwd
restic key set <ID> # set default key hint
```
## Repository Design
### Repository Layout
```
restic-repo/
├── config # encrypted JSON (version, ID, chunker params)
├── data/ # pack files (two-char prefix dirs)
│ ├── 21/
│ ├── 32/
│ └── ...
├── index/ # index files mapping blobs to packs
├── keys/ # encrypted key files (scrypt-derived)
├── locks/ # lock files for concurrent access
├── snapshots/ # snapshot metadata
└── tmp/ # temporary files
```
### Pack Format
Each pack file contains:
```
EncryptedBlob1 || ... || EncryptedBlobN || EncryptedHeader || Header_Length
```
Blob types: `0b00` data, `0b01` tree, `0b10` compressed data, `0b11` compressed tree (v2 only).
### Deduplication
Content-defined chunking with Rabin Fingerprints (64-byte sliding window). Blob sizes: 512 KiB to 8 MiB (target 1 MiB average). Only modified blobs stored in subsequent backups — works even with byte insertions/removals at arbitrary positions.
### Snapshots
Each backup creates a snapshot — a JSON document recording paths, hostname, username, time, tags, and the root tree hash. Snapshots are immutable once created.
### Locking
Two lock types: exclusive (for prune/rewrite) and non-exclusive (for backup/restore). Locks older than 30 minutes are considered stale. Use `--retry-lock <duration>` to wait for locks.
## Scripting
```bash
# JSON output
restic snapshots --json
restic backup ~/work --json
# Check if repo is initialized
restic cat config
# Exit codes for scripting
# 0 = success, 1 = fatal error, 3 = partial (some files unreadable)
# 10 = repo locked, 11 = connection error (and others)
```
## Troubleshooting
1. Find what's damaged: `restic check`
2. Backup the repository (copy data files)
3. Repair index: `restic repair index`
4. Run all backups
5. Remove missing data from snapshots: `restic repair snapshots`
6. Check again: `restic check`
## See Also
- [Docker Compose Reference](reference/docker-compose.md)
- [CrowdSec Reference](reference/crowdsec.md)