esh-pfi-infrastructure/stacks/hrafn/ci

hrafn CI deploy — mirror of what runs in vh/hrafn

These two files are hrafn's deploy pipeline. The canonical copy now lives in vh/hrafn — as of 2026-08-23 claude-bot holds write there, so infra-ops maintains the pipeline it owns directly instead of routing patches through the repo holder.

file here	canonical location in vh/hrafn
playbooks-deploy.yaml	playbooks/deploy.yaml
gitea-workflows-deploy.yaml	.gitea/workflows/deploy.yaml

This copy is a mirror kept for review and history, verified byte-identical to the live files at commit 22e0eb9d75a6. If you change one, change both — or check with:

curl -s "https://gitea.phasefinal.com/api/v1/repos/vh/hrafn/contents/playbooks/deploy.yaml?ref=main" \
  -H "Authorization: token $(cat ~/.config/claude-bot/gitea-token-repo-create)" \
  | python3 -c 'import json,sys,base64,hashlib; print(hashlib.sha256(base64.b64decode(json.load(sys.stdin)["content"])).hexdigest()[:12])'

What the change buys

The pre-CI shape was: rsync a working tree into /opt/docker/compose/hrafn/, then docker compose build && up. Two problems, both fixed here.

1. No provenance. The image was always local/hrafn:v1, so nothing on the box could answer "what commit is running". The image is now tagged with the commit SHA, and /opt/docker/compose/hrafn/.deployed records the SHA and timestamp. Rollback becomes a retag.
2. The whole repo lived in the compose directory — tests/, docs/, ROADMAP.md, persistent-memory.md, CLAUDE.md. Only the build context ships now (Dockerfile, compose.yaml, pyproject.toml, README.md, src/).

Two design calls worth knowing

• Tarball, not per-file upload steps. nevermore's playbook enumerates every source file as its own upload: step. That is explicit, but it fails open: add src/hrafn/newthing.py, forget the matching step, and the deploy silently ships without it. hrafn's build context travels as one archive so it cannot go partial.
• .env is never deployed. It is host-owned, 0600, and holds the bearer token. The playbook refuses to run if it is missing or not 0600 — a guard added because the file arrived at 0644 on handoff.
• The deploy converges, it does not accrete. The first version unpacked the tarball in place, which overwrote tracked files but never removed anything — so leftovers from the pre-CI hand-rsync (tests/, docs/, ROADMAP.md, persistent-memory.md, CLAUDE.md, LICENSE) survived the first CI deploy and had to be cleaned off the host by hand. Now the tarball unpacks to a staging dir and rsync --delete converges the compose directory onto it, so a stray file cannot outlive the next deploy. The consequence: the tar list in the workflow is authoritative — anything omitted from it is deleted from the host, except .env and .deployed.

The frozen-source defect (fixed 2026-08-22)

The converge fix above had a defect that made every deploy a no-op for source content, while still reporting success. Worth reading before touching this playbook.

The bug: the staging directory was $compose_dir/.stage — inside the rsync target. rsync -a --delete $compose_dir/.stage/ $compose_dir/ then deleted .stage from the destination (it is not in the source listing) during the transfer, destroying the source mid-copy. Reproduced exactly:

before:  app.py="OLD"   leftover.txt   .stage/app.py="NEW"
after:   app.py="OLD"   (leftover.txt GONE, .stage GONE)

Note which half worked. Deletion succeeded, so the directory looked converged; the copy silently did not happen. The host source sat frozen at the first manual rsync while .deployed and the image tag advanced with every commit — and because the build's COPY inputs never changed, Docker full-cache-hit and every SHA tag aliased one image. The provenance the SHA tagging exists to provide was false the whole time.

Why nothing caught it: the verify steps asserted the marker, container health, and a 200 from /readyz. All three pass on a frozen host. None of them measured content. A deploy that reports success without asserting the bytes changed is verifying an uptime, not a deploy.

The fixes:

• Stage outside the target (/tmp/hrafn-deploy-stage).
• CI computes a context_sha256 over the shipped file list; the playbook recomputes it on the host after the converge and fails if they differ. End-to-end from the CI checkout to the host filesystem.
• Compare the running container's src/**/*.py against the host's, catching a SHA tag that names layers the image does not contain.
• clean: true on the checkout plus an explicit HEAD == GITHUB_SHA assertion, so a reused runner workspace fails the job instead of shipping a stale tree.

No --no-cache. A cache hit is correct when the build context is genuinely unchanged, and rebuilding a Chromium base image every deploy to paper over a bug is the wrong trade. The content assertions prove the property directly instead of brute-forcing it.

Gotcha in the check itself: compare only *.py. pip install . generates src/hrafn.egg-info/* inside the image (6 files the host lacks), and __pycache__ appears at runtime, so a naive find src -type f compare fails on every healthy deploy. Verified against a known-good container before shipping: 12 host files, 18 in the container, 0 content differences.

Two failed runs getting there (9920, 9921)

Worth reading before adding anything to the workflow's prerequisite step.

The frozen-source fix originally shipped with a HEAD == GITHUB_SHA assertion. It failed twice, both times for reasons the change itself introduced:

• run 9920 — git: command not found (exit 127). act_runner's job image ships no git binary, and actions/checkout@v4 does not need one.
• run 9921 — adding git made the checkout fail with server certificate verification failed. CAfile: none. Installing git flips checkout@v4 from its node implementation — which every working run of this pipeline had used — to the git binary, and the image has no CA bundle.

So a nice-to-have assertion changed the checkout's code path and broke a working pipeline. It was removed rather than patched with ca-certificates: it guarded a hypothesis that turned out to be wrong (the frozen source was a self-referential rsync in the playbook, not a stale checkout), and clean: true plus the host-side content hash already cover the real risk.

Do not add git to the prerequisite step. Run 9922 is green with it out.

Validation

The playbook parses and interpolates clean under elway's own parser:

scripts/elway ana-docker --playbook stacks/hrafn/ci/playbooks-deploy.yaml \
  --var hrafn_sha=abc123def456 --dry-run

That dry-run is worth running after any edit — it caught a real bug during authoring, where a comment containing a literal {{ ... }} identifier was picked up by elway's variable substitution and failed the run.

No new Actions secrets are required: DEPLOY_SSH_KEY and MGMT_REPO_TOKEN already exist at user scope on vh from the nevermore/task-board wiring.