diff --git a/persistent-memory.md b/persistent-memory.md index 72f561e..545a47c 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -151,6 +151,7 @@ not arrived._ ## Recent decisions +- `[2026-09-02]` **Every CI job on the shared `pfi-fleet` runner is root on ana-docker — and `container.valid_volumes: []` does NOT prevent it.** Measured: a job container is uid 0, `/var/run/docker.sock` is mounted by act_runner independently of that list, `docker ps` returns all 49 host containers (gitea itself, synapse, phasefinal-web, adguardhome), `docker compose v2.33.0` on PATH. ⚠ **LOAD-BEARING** — `vh/Worldtree`, `vh/soong-lab`, `vh/skaldsong`, `vh/wt-matrix-bridge` all drive buildx through that socket, so it cannot simply be closed; **isolate sensitive builds onto a dedicated runner instead.** Also measured the same night: `services:` containers work (Postgres 16), and **full-URL `uses: https://gitea.phasefinal.com/actions/checkout@v4` resolves from the local mirrors** — the un-parked half of the github-independence work, needing neither `DEFAULT_ACTIONS_URL=self` nor the act_runner auth path that blocked it on 2026-08-05. Prompted by vastblue-dev's CI-posture question for a client-funded healthcare repo. → `stacks/gitea-runner/README.md` - `[2026-09-02]` **pfi-gx10 BASELINED: 79.36 s/it median on the run-3c shape, and the training stack works on aarch64/sm_121.** Median across 10 timed steps, 0.19% spread, **peak 75.1 / 121.6 GiB — 46 GiB spare**, `attn_resolved: flex_attention`. **6× slower than ana-ml2 where compute predicts 2.7×** → likely memory-bandwidth-bound; **capacity box, not throughput box.** Ruled **bare metal, not Proxmox** (no aarch64 PVE; the GPU is on-package and cache-coherent, so passthrough would partition the unified memory that is the whole point). ⚠ `sm_121` is NOT in torch's arch list — everything JITs from sm_120 PTX, so **warm up before timing anything** (an unwarmed bench read 27 TFLOP/s against a true 93). → `persistent-memory.d/2026-09-01-pfi-gx10-onboarding.md` - `[2026-09-02]` **I priced a failure in the units I happened to be measuring — operator overruled me, correctly.** Recommended run 3c to ana-ml2 by costing a breaker trip as "≤50 steps ≈ 11 min of recompute". It is a **40-minute drive each way** with **13 Anaheim hosts dark, three of them SureFire CLIENT machines**. `save_steps` caps the recompute, never the outage. ⚠ **General form: a metric in hand will volunteer itself as the unit of risk.** → `persistent-memory.d/2026-09-01-pfi-gx10-onboarding.md` - `[2026-09-02]` **althing 3.2.0→3.2.4 deployed, and ALTHING DEPLOY IS FOUR SURFACES not three.** The fourth (plugin) had no runbook step and was frozen at Aug 28 — **missing the SessionStart/SessionEnd hooks and `pane-route.sh` entirely**, so "CC seats re-declare automatically" was never true here. Now one command (`scripts/deploy-althing.sh`). ⚠ `uv tool install .` **without `--force` is a silent no-op**. ⚠ **A missing deploy surface presents as "the migration needs manual work", not as an error.** → `persistent-memory.d/2026-09-01-althing-320-deploy.md` diff --git a/stacks/gitea-runner/README.md b/stacks/gitea-runner/README.md index acbeffa..16e4f84 100644 --- a/stacks/gitea-runner/README.md +++ b/stacks/gitea-runner/README.md @@ -35,6 +35,40 @@ invocation — not a copy-pasted playbook. Until one of those bites, one runner is enough. +## What a job on this runner can actually do + +Measured 2026-09-02 on `pfi-fleet` (throwaway repo, three jobs, since deleted). +Recorded because two of these are commonly assumed the other way. + +| capability | result | +|---|---| +| `services:` containers | **yes** — Postgres 16 answered on the service name as hostname after ~6 s; wait on `pg_isready`, not on ordering | +| host docker daemon | **yes, root-equivalent** — `/var/run/docker.sock` is in every job container, `docker ps` showed all 49 host containers, `docker compose v2.33.0` on PATH | +| `uses:` from the local mirrors | **yes** — `uses: https://gitea.phasefinal.com/actions/checkout@v4` resolves and runs, with no `DEFAULT_ACTIONS_URL` change | + +⚠ **`container.valid_volumes: []` does not keep docker out of jobs.** act_runner +mounts the daemon socket itself, independently of that list, so a tight-looking +`valid_volumes` is not containment. Any repo the runner serves — it is registered +instance-wide — can control everything on ana-docker, gitea included. Four repos +(`vh/Worldtree`, `vh/soong-lab`, `vh/skaldsong`, `vh/wt-matrix-bridge`) drive +buildx through it, so it is load-bearing and closing it would break their CI. +**Isolate sensitive builds onto a dedicated runner rather than tightening this one.** + +⚠ **Job images need a node binary.** JS actions execute as `node /var/run/act/...`, +so `python:3-slim` fails on the first `uses:`. Use `node:20-bookworm`, or +`docker:cli` plus `apk add --no-cache git nodejs` when the job also builds images. + +**Full-URL `uses:` is the un-parked half of the github-independence work.** The +global `DEFAULT_ACTIONS_URL=self` flip is still blocked on act_runner's +action-fetch auth, but a per-workflow full-URL ref needs neither the flip nor the +auth path. Mirrors live under the `actions` and `astral-sh` orgs, all public: +checkout, cache, upload-artifact, download-artifact, setup-node, setup-python, +setup-uv. + +**Polling a run from the API:** use `/actions/runs`, not `/actions/tasks` — on +gitea 1.26.1 `tasks` returned an empty `workflow_runs` for a run that `runs` listed +and executed. + ## Prereqs Before running the deploy playbook: diff --git a/stacks/gitea-runner/conf/config.yaml b/stacks/gitea-runner/conf/config.yaml index fded738..bae8025 100644 --- a/stacks/gitea-runner/conf/config.yaml +++ b/stacks/gitea-runner/conf/config.yaml @@ -46,7 +46,19 @@ container: # /data volume so workspaces persist briefly between steps. workdir_parent: /data/workspace - # Volumes the runner allows job containers to bind-mount. Keep tight. + # Volumes a WORKFLOW may bind-mount into its job container. Keep tight. + # + # ⚠ This does NOT keep the docker daemon out of jobs, and reading it that + # way is the mistake. act_runner mounts /var/run/docker.sock into every job + # container on its own, independently of this list. Measured 2026-09-02: a + # job running `docker:27-cli` is uid 0, sees all 49 containers on this host + # via `docker ps`, and has `docker compose v2.33.0` on PATH. + # + # So every job on this runner has root-equivalent control of ana-docker — + # which hosts gitea itself, synapse, phasefinal-web and adguardhome. It is + # also LOAD-BEARING: vh/Worldtree, vh/soong-lab, vh/skaldsong and + # vh/wt-matrix-bridge all drive buildx through that socket, so it cannot + # simply be closed. Isolate sensitive work onto its own runner instead. valid_volumes: [] force_pull: false