esh-pfi-infrastructure/stacks/gitea-runner/README.md

# gitea-runner

Self-hosted [Gitea Actions](https://docs.gitea.com/usage/actions/overview)
runner. Polls `gitea.phasefinal.com` for jobs from any repo that has a
`.gitea/workflows/` directory and runs them in ephemeral docker
containers on this host.

**Server:** ana-docker (single central runner — see "Topology" below
for when to add more)
**Image:** `gitea/act_runner:latest`
**Outbound only** — no host port published; the runner connects out to
gitea, gitea never connects in.

## Topology

We run **one central runner on ana-docker**. Reasoning:

- gitea is on ana-docker, so runner→API is local
- existing fleet tooling (`elway`, `sync-stacks`, `deploy-stack`,
  `refresh-server-info`) already SSHes from one origin to all hosts;
  the runner inherits that pattern
- single point to manage SSH keys, secrets, and runner upgrades

The deploy playbook (`playbooks/deploy-gitea-runner.yaml`) is
parameterized by host / runner-name / labels, so spinning up
`nh3-docker-runner` or an ESH runner later is a one-line elway
invocation — not a copy-pasted playbook.

**When to add a site-local runner:**

- Cross-site SSH from ana-docker to that site has become unreliable
- A workflow needs LAN access to something only reachable from inside
  that site's network segment
- You want failure isolation (NH3 can deploy itself when Anaheim is down)

Until one of those bites, one runner is enough.

## What a job on this runner can actually do

Measured 2026-09-02 on `pfi-fleet` (throwaway repo, three jobs, since deleted).
Recorded because two of these are commonly assumed the other way.

| capability | result |
|---|---|
| `services:` containers | **yes** — Postgres 16 answered on the service name as hostname after ~6 s; wait on `pg_isready`, not on ordering |
| host docker daemon | **yes, root-equivalent** — `/var/run/docker.sock` is in every job container, `docker ps` showed all 49 host containers, `docker compose v2.33.0` on PATH |
| `uses:` from the local mirrors | **yes** — `uses: https://gitea.phasefinal.com/actions/checkout@v4` resolves and runs, with no `DEFAULT_ACTIONS_URL` change |

⚠ **`container.valid_volumes: []` does not keep docker out of jobs.** act_runner
mounts the daemon socket itself, independently of that list, so a tight-looking
`valid_volumes` is not containment. Any repo the runner serves — it is registered
instance-wide — can control everything on ana-docker, gitea included. Four repos
(`vh/Worldtree`, `vh/soong-lab`, `vh/skaldsong`, `vh/wt-matrix-bridge`) drive
buildx through it, so it is load-bearing and closing it would break their CI.
**Isolate sensitive builds onto a dedicated runner rather than tightening this one.**

⚠ **Job images need a node binary.** JS actions execute as `node /var/run/act/...`,
so `python:3-slim` fails on the first `uses:`. Use `node:20-bookworm`, or
`docker:cli` plus `apk add --no-cache git nodejs` when the job also builds images.

**Full-URL `uses:` is the un-parked half of the github-independence work.** The
global `DEFAULT_ACTIONS_URL=self` flip is still blocked on act_runner's
action-fetch auth, but a per-workflow full-URL ref needs neither the flip nor the
auth path. Mirrors live under the `actions` and `astral-sh` orgs, all public:
checkout, cache, upload-artifact, download-artifact, setup-node, setup-python,
setup-uv.

**Polling a run from the API:** use `/actions/runs`, not `/actions/tasks` — on
gitea 1.26.1 `tasks` returned an empty `workflow_runs` for a run that `runs` listed
and executed.

## Prereqs

Before running the deploy playbook:

1. **Verify Gitea Actions is enabled.** In gitea 1.21+ Actions ships
   on by default, but check `/-/admin/actions` resolves. If not, add
   `GITEA__actions__ENABLED=true` to the gitea stack env and bounce.

2. **Generate a registration token.** Pick the scope:

   | Scope | URL | Use when |
   |---|---|---|
   | Global (admin) | `https://gitea.phasefinal.com/-/admin/actions/runners` | runner serves any repo on the instance (recommended for the central PFI runner) |
   | Org/user | `https://gitea.phasefinal.com/<owner>/-/actions/runners` | runner serves all repos under one owner |
   | Repo | `https://gitea.phasefinal.com/<owner>/<repo>/settings/actions/runners` | runner serves one repo |

   Register-as-admin is right for our use case: one runner, fleet-wide.

3. **Create an SSH deploy key for the runner** that lets it execute
   the elway playbooks against fleet hosts. The key lives only on
   ana-docker (passed in as a workflow secret per repo, or mounted
   into the runner via volume — see "Wiring deploys" below).

4. **(Optional) Create a Gitea PAT** with `read:repository` scope on
   `vh/esh-pfi-infrastructure`. Workflows need to check out the
   management repo to get at the playbooks; the auto-injected
   `GITHUB_TOKEN` only works for the triggering repo.

## Deploy

```bash
# Edit .env first if not using defaults — at minimum paste the registration token
$EDITOR stacks/gitea-runner/.env.example   # template

# First-time deploy
scripts/elway ana-docker --playbook playbooks/deploy-gitea-runner.yaml

# Site-local runner later (NH3 or ESH)
scripts/elway nh3-docker --playbook playbooks/deploy-gitea-runner.yaml \
  --var runner_name=nh3-docker-runner --var runner_labels=pfi-fleet,nh3-docker
```

The playbook seeds `.env` from `.env.example` only if absent; for the
first run, copy `.env.example` to `/opt/docker/compose/gitea-runner/.env`
on the host and paste the registration token in before running, OR
let the playbook seed it and edit on the host before the
`docker compose up -d` step (it's idempotent — second run will pick up
the edited token).

After successful registration, the token is consumed (it's one-time
use). You can clear `GITEA_RUNNER_REGISTRATION_TOKEN` from `.env`;
the runner reads its permanent credentials from `${DATA_DIR}/.runner`
on subsequent starts.

## Wiring deploys

A workflow that runs on the central runner needs three things:

1. **`runs-on:`** matching a runner label — `pfi-fleet` (cross-fleet)
   or `ana-docker` (pin to that host).
2. **An SSH key** to reach the deploy target. Stored as a repo or
   org-level Actions secret named e.g. `DEPLOY_SSH_KEY`. The
   corresponding public key must be in `~lkraven/.ssh/authorized_keys`
   on every host the workflow targets.
3. **A token to clone `vh/esh-pfi-infrastructure`** if the workflow
   wants to invoke an elway playbook from this repo. Stored as
   `MGMT_REPO_TOKEN` (Gitea PAT, `read:repository` scope).

See `stacks/task-board/gitea-workflow-deploy.yaml.example` for a
complete deploy workflow that consumes all three.

## Path layout (on ana-docker)

| Host path | Container path | Purpose | Restic? |
|---|---|---|---|
| `/opt/docker/compose/gitea-runner/` | — | compose.yaml + .env | included (via `/opt/docker`) |
| `/opt/docker/conf/gitea-runner/data/` | `/data` | `.runner` creds, cache, job workspaces | excluded (regenerable; nothing irreplaceable) |

## Operations

```bash
# Tail runner logs
ssh ana-docker docker logs -f gitea-runner

# List currently registered runners (admin)
# https://gitea.phasefinal.com/-/admin/actions/runners

# Re-register (lost the .runner file? regenerate token, then:)
ssh ana-docker docker compose -f /opt/docker/compose/gitea-runner/compose.yaml down
ssh ana-docker rm /opt/docker/conf/gitea-runner/data/.runner
# paste new GITEA_RUNNER_REGISTRATION_TOKEN into .env
scripts/elway ana-docker --playbook playbooks/deploy-gitea-runner.yaml

# Pin to a specific act_runner version
# Edit RUNNER_IMAGE in /opt/docker/compose/gitea-runner/.env, then:
scripts/elway ana-docker --playbook playbooks/deploy-gitea-runner.yaml
```