fix(hrafn-ci): staging dir inside the rsync target froze host source silently
Root cause of nevermore-claude's report that v1.0.0 deployed green while the host kept serving 0.1.0. The staging dir was $compose_dir/.stage -- INSIDE the rsync target. So `rsync -a --delete $compose_dir/.stage/ $compose_dir/` deleted .stage from the destination (absent from the source listing) DURING the transfer, destroying the source mid-copy. Reproduced exactly: before: app.py="OLD" leftover.txt .stage/app.py="NEW" after: app.py="OLD" leftover.txt GONE, .stage GONE Deletion worked; the copy silently did not. So the directory looked converged while host source stayed frozen at the first manual rsync, and because the build's COPY inputs never changed, Docker full-cache-hit and every SHA tag aliased one image. The provenance guarantee was false. Nothing caught it because the verify steps asserted the marker, health, and a 200 from /readyz -- all of which pass on a frozen host. None measured content. Fixes: - stage at /tmp/hrafn-deploy-stage, outside the target - CI computes context_sha256 over the shipped file list; the playbook recomputes it on the host post-converge and fails on mismatch - compare the running container's src/**/*.py against the host's, catching a SHA tag naming layers the image does not contain - checkout clean:true + assert HEAD == GITHUB_SHA so a reused runner workspace fails the job rather than shipping a stale tree Declined --no-cache: a cache hit is correct when the context is genuinely unchanged, and the new assertions prove the property directly rather than brute-forcing it. The container-vs-host check compares only *.py -- `pip install .` generates src/hrafn.egg-info/* inside the image and __pycache__ appears at runtime, so a naive `find src -type f` compare false-fails on every healthy deploy. Verified against the live container before shipping (12 host files, 18 in container, 0 content differences).
This commit is contained in:
@@ -44,6 +44,22 @@ jobs:
|
||||
|
||||
- name: Checkout hrafn (triggering repo)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
# Self-hosted runners reuse workspaces. Without a clean checkout a
|
||||
# stale tree can be tarred while GITHUB_SHA claims the new commit —
|
||||
# exactly the kind of drift the SHA tagging is supposed to prevent.
|
||||
clean: true
|
||||
|
||||
- name: Assert the checkout really is the triggering commit
|
||||
# Cheap, and it converts "the runner shipped stale files" from a
|
||||
# silent deploy into a failed job.
|
||||
run: |
|
||||
head=$(git rev-parse HEAD)
|
||||
echo "checkout HEAD : $head"
|
||||
echo "GITHUB_SHA : $GITHUB_SHA"
|
||||
test "$head" = "$GITHUB_SHA" || {
|
||||
echo "CHECKOUT DRIFT — refusing to deploy a tree that is not $GITHUB_SHA"
|
||||
exit 1; }
|
||||
|
||||
- name: Checkout management repo (for elway)
|
||||
uses: actions/checkout@v4
|
||||
@@ -83,10 +99,24 @@ jobs:
|
||||
echo "context contents:"
|
||||
tar tzf dist/hrafn-context.tgz
|
||||
|
||||
# Content hash over the same file list, in the same order the
|
||||
# playbook recomputes it on the host. This is what turns "the
|
||||
# deploy said OK" into "the bytes on the host are the bytes CI
|
||||
# built" — the assertion whose absence let the host source stay
|
||||
# frozen at 0.1.0 through every green deploy.
|
||||
CTX=$(find Dockerfile compose.yaml pyproject.toml README.md .env.example src \
|
||||
-type f | LC_ALL=C sort | xargs sha256sum | sha256sum | cut -d' ' -f1)
|
||||
echo "context_sha256=$CTX" | tee -a "$GITHUB_ENV"
|
||||
echo " version in this context:"
|
||||
grep -h '__version__' src/hrafn/__init__.py || true
|
||||
|
||||
- name: Deploy hrafn (in-repo elway playbook)
|
||||
# hrafn_sha becomes the image tag and is written to
|
||||
# /opt/docker/compose/hrafn/.deployed on the host.
|
||||
# /opt/docker/compose/hrafn/.deployed on the host. context_sha256
|
||||
# is asserted against the host tree AFTER the converge, so a deploy
|
||||
# that fails to land new source fails the job instead of passing.
|
||||
run: |
|
||||
_mgmt/scripts/elway ana-docker \
|
||||
--playbook playbooks/deploy.yaml \
|
||||
--var hrafn_sha=${GITHUB_SHA::12}
|
||||
--var hrafn_sha=${GITHUB_SHA::12} \
|
||||
--var context_sha256=${context_sha256}
|
||||
|
||||
Reference in New Issue
Block a user