fix(hrafn-ci): staging dir inside the rsync target froze host source silently

Root cause of nevermore-claude's report that v1.0.0 deployed green while the
host kept serving 0.1.0.

The staging dir was $compose_dir/.stage -- INSIDE the rsync target. So
`rsync -a --delete $compose_dir/.stage/ $compose_dir/` deleted .stage from
the destination (absent from the source listing) DURING the transfer,
destroying the source mid-copy. Reproduced exactly:

  before:  app.py="OLD"  leftover.txt  .stage/app.py="NEW"
  after:   app.py="OLD"  leftover.txt GONE, .stage GONE

Deletion worked; the copy silently did not. So the directory looked
converged while host source stayed frozen at the first manual rsync, and
because the build's COPY inputs never changed, Docker full-cache-hit and
every SHA tag aliased one image. The provenance guarantee was false.

Nothing caught it because the verify steps asserted the marker, health, and
a 200 from /readyz -- all of which pass on a frozen host. None measured
content.

Fixes:
- stage at /tmp/hrafn-deploy-stage, outside the target
- CI computes context_sha256 over the shipped file list; the playbook
  recomputes it on the host post-converge and fails on mismatch
- compare the running container's src/**/*.py against the host's, catching
  a SHA tag naming layers the image does not contain
- checkout clean:true + assert HEAD == GITHUB_SHA so a reused runner
  workspace fails the job rather than shipping a stale tree

Declined --no-cache: a cache hit is correct when the context is genuinely
unchanged, and the new assertions prove the property directly rather than
brute-forcing it.

The container-vs-host check compares only *.py -- `pip install .` generates
src/hrafn.egg-info/* inside the image and __pycache__ appears at runtime, so
a naive `find src -type f` compare false-fails on every healthy deploy.
Verified against the live container before shipping (12 host files, 18 in
container, 0 content differences).
This commit is contained in:
vh
2026-08-22 21:57:16 -07:00
parent bb19a96f39
commit b38c369313
3 changed files with 156 additions and 15 deletions
+32 -2
View File
@@ -44,6 +44,22 @@ jobs:
- name: Checkout hrafn (triggering repo)
uses: actions/checkout@v4
with:
# Self-hosted runners reuse workspaces. Without a clean checkout a
# stale tree can be tarred while GITHUB_SHA claims the new commit —
# exactly the kind of drift the SHA tagging is supposed to prevent.
clean: true
- name: Assert the checkout really is the triggering commit
# Cheap, and it converts "the runner shipped stale files" from a
# silent deploy into a failed job.
run: |
head=$(git rev-parse HEAD)
echo "checkout HEAD : $head"
echo "GITHUB_SHA : $GITHUB_SHA"
test "$head" = "$GITHUB_SHA" || {
echo "CHECKOUT DRIFT — refusing to deploy a tree that is not $GITHUB_SHA"
exit 1; }
- name: Checkout management repo (for elway)
uses: actions/checkout@v4
@@ -83,10 +99,24 @@ jobs:
echo "context contents:"
tar tzf dist/hrafn-context.tgz
# Content hash over the same file list, in the same order the
# playbook recomputes it on the host. This is what turns "the
# deploy said OK" into "the bytes on the host are the bytes CI
# built" — the assertion whose absence let the host source stay
# frozen at 0.1.0 through every green deploy.
CTX=$(find Dockerfile compose.yaml pyproject.toml README.md .env.example src \
-type f | LC_ALL=C sort | xargs sha256sum | sha256sum | cut -d' ' -f1)
echo "context_sha256=$CTX" | tee -a "$GITHUB_ENV"
echo " version in this context:"
grep -h '__version__' src/hrafn/__init__.py || true
- name: Deploy hrafn (in-repo elway playbook)
# hrafn_sha becomes the image tag and is written to
# /opt/docker/compose/hrafn/.deployed on the host.
# /opt/docker/compose/hrafn/.deployed on the host. context_sha256
# is asserted against the host tree AFTER the converge, so a deploy
# that fails to land new source fails the job instead of passing.
run: |
_mgmt/scripts/elway ana-docker \
--playbook playbooks/deploy.yaml \
--var hrafn_sha=${GITHUB_SHA::12}
--var hrafn_sha=${GITHUB_SHA::12} \
--var context_sha256=${context_sha256}