restic/ana-docker: gitea dump — switch zip → uncompressed tar for dedup
The compressed zip defeated restic's content-defined chunking: each day's dump looked completely different to restic even when the repo content barely changed, causing repo growth of ~full dump size (821 MB at last measurement) every day until forget/prune aged snapshots out. Uncompressed tar is dedup-friendly. After the first snapshot, daily incrementals cost only the actual new-data delta — typically a few MB for an active repo. Tradeoff: stage file on the client host is ~2-3x the zip size while the dump is in flight, but that's transient (purged at the start of each run). Repo-side storage is much smaller over time.
This commit is contained in:
@@ -91,19 +91,30 @@ else
|
||||
fi
|
||||
|
||||
# ---------- gitea (native `gitea dump`) ---------------------------------------
|
||||
# `gitea dump` produces a single ZIP with the DB dump, repo trees, config,
|
||||
# LFS objects and attachments. The in-container command knows its own DB
|
||||
# creds (from GITEA__database__* env vars), so no creds needed here.
|
||||
# `gitea dump` produces a single archive with the DB dump, repo trees,
|
||||
# config, LFS objects and attachments. The in-container command reads
|
||||
# its own DB creds (from GITEA__database__* env vars) — no external
|
||||
# creds needed.
|
||||
#
|
||||
# Size scales with the repo tree. Trim with --skip-lfs-data,
|
||||
# --skip-repository, --skip-attachment-data if the dump gets too large.
|
||||
# `--type tar` produces an UNCOMPRESSED tarball. Compressed formats
|
||||
# (zip/tar.gz) defeat restic's content-defined chunking: each day's
|
||||
# dump looks completely different to restic even when the underlying
|
||||
# data barely changed, so repo grows by ~full dump size every day.
|
||||
# Uncompressed tar lets restic dedup aggressively — after the first
|
||||
# snapshot, daily incrementals only cost the actual new-data delta.
|
||||
#
|
||||
# Tradeoff: on-disk stage file is larger (~2-3x the zip size) but that's
|
||||
# transient (deleted next run). Repo-side storage is much smaller.
|
||||
#
|
||||
# Size trimming flags if the tar becomes excessive: --skip-lfs-data,
|
||||
# --skip-repository, --skip-attachment-data.
|
||||
if docker inspect gitea >/dev/null 2>&1; then
|
||||
log "dumping gitea (gitea dump, zip)"
|
||||
log "dumping gitea (gitea dump, uncompressed tar)"
|
||||
if docker exec -u git gitea sh -c \
|
||||
'rm -f /tmp/gitea-dump.zip && gitea dump -c /data/gitea/conf/app.ini -f /tmp/gitea-dump.zip' \
|
||||
'rm -f /tmp/gitea-dump.tar && gitea dump -c /data/gitea/conf/app.ini -f /tmp/gitea-dump.tar --type tar' \
|
||||
>/dev/null 2>&1; then
|
||||
docker cp gitea:/tmp/gitea-dump.zip "$STAGE/gitea-dump.zip" \
|
||||
&& docker exec -u git gitea rm -f /tmp/gitea-dump.zip \
|
||||
docker cp gitea:/tmp/gitea-dump.tar "$STAGE/gitea-dump.tar" \
|
||||
&& docker exec -u git gitea rm -f /tmp/gitea-dump.tar \
|
||||
|| warn "gitea dump copy/cleanup failed"
|
||||
else
|
||||
warn "gitea dump command failed"
|
||||
|
||||
Reference in New Issue
Block a user