vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(blender-run): unique remote job dir per local path, mirrored with --delete

Browse Source 
draupnir review: the remote job dir was keyed on the basename alone, so two
local dirs with the same name shared one remote dir, and a rerun inherited
stale files. The dir is now <basename>-<8 hex of sha256(abs path)>, and it is
mirrored with rsync --delete, confined to that one directory. There is no rm on
a computed path. Verified: a file deleted locally is gone from the rerun's
remote dir. The /defaults stderr line is documented as harmless noise.
This commit is contained in:
vh
2026-09-28 08:43:55 -07:00
parent d0f68a3b18
commit 46276611a7
2 changed files with 23 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/blender-run
+15 -7
View File
@@ -16,10 +16,12 @@
# output path against the working directory: "cannot save 'out.png'". Python file I/O and
# importers do use the cwd.
#
# Files: fv-ml1 does NOT mount /mnt/smithy. With --job DIR, DIR is copied to
# fv-ml1:/tank/blender/jobs/<name>/, Blender runs WITH THAT AS ITS WORKING DIRECTORY, and new or
# changed files are copied back into DIR afterwards (nothing is deleted on either side). Use
# relative paths inside the job. Without --job, the only paths Blender sees are under
# Files: fv-ml1 does NOT mount /mnt/smithy. With --job DIR, DIR is mirrored to
# fv-ml1:/tank/blender/jobs/<basename>-<hash of DIR's absolute path>/, Blender runs WITH THAT AS ITS
# WORKING DIRECTORY, and new or changed files are copied back into DIR afterwards (nothing is ever
# deleted locally). Use relative paths inside the job. Running the SAME DIR twice concurrently
# still shares one remote dir, so don't do that. Remote job dirs are never swept: one copy is kept
# per distinct DIR (small files, on /tank). Without --job, the only paths Blender sees are under
# /work (= fv-ml1:/tank/blender).
#
# Budget: GPU 3 (96 GB, borrowed from the vLLM reserve: this ends if a full-size seat moves in).
@@ -42,10 +44,16 @@ done
WORKDIR=/work
if [ -n "$JOB" ]; then
[ -d "$JOB" ] || { echo "blender-run: --job $JOB is not a directory" >&2; exit 2; }
NAME=$(basename "$(realpath "$JOB")")
[[ $NAME =~ ^[A-Za-z0-9._-]+$ ]] || { echo "blender-run: job dir name '$NAME' must be [A-Za-z0-9._-]" >&2; exit 2; }
ABS=$(realpath "$JOB")
BASE=$(basename "$ABS")
[[ $BASE =~ ^[A-Za-z0-9._-]+$ ]] || { echo "blender-run: job dir name '$BASE' must be [A-Za-z0-9._-]" >&2; exit 2; }
# Keyed on basename + a hash of the ABSOLUTE local path (draupnir, 2026-09-28): two different
# dirs that share a basename no longer share a remote dir. `--delete` makes the remote an exact
# mirror of DIR, so a rerun never inherits an earlier run's leftovers. rsync removes only
# inside that one remote dir; there is no rm on a computed path.
NAME=$BASE-$(printf '%s' "$ABS" | sha256sum | cut -c1-8)
ssh -n -o BatchMode=yes "$HOST" "mkdir -p /tank/blender/jobs/$NAME"
rsync -a "$JOB"/ "$HOST:/tank/blender/jobs/$NAME/"
rsync -a --delete "$JOB"/ "$HOST:/tank/blender/jobs/$NAME/"
WORKDIR=/work/jobs/$NAME
fi