Release acceptance gate: release.sh refuses to sign without a matching artifact

scripts/acceptance-gate.py recomputes every catalog test's domain fingerprint
from /etc/forgefirm-manifest.json inside the release rootfs and requires the
committed releases/v<version>/acceptance.json to carry a matching PASS
(inherited results not core and newer than the invalidate epoch; the artifact
self-hashed; the catalog identical to the tree). release.sh runs it after the
build and stages the artifact as a release asset; FORGEFIRM_ACCEPTANCE_SKIP=1
bypasses loudly. scripts/manifest-from-tree.py builds the same manifest from
the recipe pins with git for CI and the workstation; forgetest-ci.yml runs the
unit tests and enforces the coverage lint (every manifest path covered by some
test). docs/ACCEPTANCE.md is the contract; the coverage currency rule and the
status live in BRINGUP.
This commit is contained in:
ScottW514
2026-08-15 15:57:12 -04:00
parent c0f53a865f
commit 1179d5e7c1
10 changed files with 644 additions and 9 deletions
+68
View File
@@ -0,0 +1,68 @@
# forgetest: the release acceptance tool's host-side proof.
#
# - unit tests: campaign rules, fingerprints, artifact build/verify (the
# release gate's decision, including the negative fixtures), runner +
# HTTP API end to end with a fake catalog and a fake bench tool
# - coverage lint: every source path of every component pinned by the
# recipes must be selected by some catalog test's coverage globs (the
# tree manifest is generated from the pins with git - no Yocto build);
# enforced: an uncovered path fails the job (docs/ACCEPTANCE.md).
name: forgetest-ci
on:
push:
paths:
- 'forgetest/**'
- 'scripts/acceptance-gate.py'
- 'scripts/manifest-from-tree.py'
- 'meta-forgefirm/recipes-forgefirm/**'
- '.github/workflows/forgetest-ci.yml'
pull_request:
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout forgefirm
uses: actions/checkout@v4
with:
path: forgefirm
- name: Checkout meta-openglow (sibling)
uses: actions/checkout@v4
with:
repository: ScottW514/meta-openglow
ref: scarthgap
path: meta-openglow
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Unit tests
working-directory: forgefirm/forgetest
run: python -m unittest discover -s tests -v
- name: Tree manifest from the recipe pins
working-directory: forgefirm
run: python scripts/manifest-from-tree.py --out tree-manifest.json
- name: Coverage lint (enforced)
working-directory: forgefirm/forgetest
run: python -m forgetest.coverage --manifest ../tree-manifest.json --enforce | tee "$GITHUB_STEP_SUMMARY"
- name: Gate self-check on the tree manifest
working-directory: forgefirm
run: |
# The gate must load the catalog and refuse a missing/unauthorized
# artifact cleanly (exit 1/2, never a traceback).
set +e
echo '{}' > empty.json
python scripts/acceptance-gate.py empty.json tree-manifest.json --quiet
rc=$?
set -e
test "$rc" -eq 1
+1
View File
@@ -10,6 +10,7 @@ panel, and a standard Grbl interface.
* [Build Instructions](https://github.com/ScottW514/forgefirm/blob/master/BUILD.md)
* [Connecting LightBurn](https://github.com/ScottW514/forgefirm/blob/master/docs/LIGHTBURN.md)
* [How the laser safing works](https://github.com/ScottW514/forgefirm/blob/master/docs/SAFETY.md)
* [How a release is accepted](https://github.com/ScottW514/forgefirm/blob/master/docs/ACCEPTANCE.md)
* [Community Support](https://community.openglow.org)
## What it does
+177
View File
@@ -0,0 +1,177 @@
# Release acceptance
A ForgeFIRM release is signed and published only when the **acceptance
catalog** passes on the bench machine and the result is committed with
the release. This document is the contract: what the gate is, how a
campaign runs, how a result stays valid across builds, and what the
release pipeline checks.
## The pieces
| Piece | Where | What it does |
|---|---|---|
| **forgetest** | `forgetest/` in this repo; on the **dev image** as a daemon on HTTP **:8090** | Runs the catalog against the machine from a self-contained web page, keeps the append-only result log under `/data/forgetest/`, exports the release artifact, and carries the bench diagnostics page. Never on a release image. |
| **Image manifest** | `/etc/forgefirm-manifest.json` in every image (`meta-forgefirm/classes/forgefirm-manifest.bbclass`, `forgefirm-image-manifest.bbclass`) | The build's inputs: for every component the pinned revision and one `[path, blob-id]` pair per source file, plus the platform identity (machine, kernel revision + config hash, device tree hashes, layer content hashes). |
| **Artifact** | `releases/v<version>/acceptance.json` (+ `acceptance.md`) committed to this repo, and attached to the GitHub release | What forgetest exported: per catalog test the winning PASS, the fingerprint it ran under, and whether it was inherited. Self-hashed. |
| **Gate** | `scripts/acceptance-gate.py`, called by `scripts/release.sh` | Recomputes every test's fingerprint from the manifest inside the release rootfs and requires the recorded PASS to match. |
## The catalog
Every test declares, in code (`forgetest/forgetest/suite/*.py`):
- **kind** - `auto` (no operator), `operator` (prompts, no emission), or
`live` (laser emission possible: the page requires the eye-protection /
fire-watch / exhaust acknowledgment, and the physical arm press is
required through the controller's normal path - forgetest never touches
the laser latch);
- **hardware** - `api` (forgectrl and the controller stay up) or
`takeover` (forgectrl is stopped for the duration; a marker file makes a
crash recoverable at the next start);
- **covers** - the source paths whose content the test stands for, as
`(component, glob)` pairs;
- **requires** - tests that must be satisfied first (the emission tests
require the motion and readback tests);
- **always** - membership in the **always-required core**, which is run
in every campaign and is never inherited: image health, the kernel
latch/safety readbacks, and one live emission witness with the
armed-window disarm.
`GET /catalog` on the tool lists the definitions; the page shows them under
each test's *details*.
## Domain fingerprints and inheritance
A test's **domain fingerprint** is the hash of the `(component, path,
blob-id)` triples its coverage globs select in the image manifest, plus the
platform identity, plus the hash of the test's own implementation. A PASS
recorded under fingerprint F applies to any build whose recomputed
fingerprint is F - the same code computes it on the board and in the gate.
Consequences:
- A change to a covered file invalidates exactly the tests that cover it.
A panel-only change reruns the core plus the panel tests, not the
cooling drills.
- A platform change (kernel, device tree, a layer's content) invalidates
everything.
- A change to a test's implementation invalidates that test's earlier
passes and no other.
- "Touched" is computed from content hashes carried in the image, never
declared by hand.
## Campaigns
A **campaign** is bound to one image (manifest content hash) and one
catalog (catalog hash). The first Start on an image opens one. It stays
open until a **FAIL** (or an erroring test), an **invalidate-all**, an
explicit **reset**, or a different image or catalog. Reboots into the same
image continue it.
For every test, in order:
1. a PASS in the open campaign with the current fingerprint satisfies it;
2. otherwise, if it is not core, the newest PASS anywhere in the history
with the current fingerprint and newer than the last invalidate-all is
**inherited** (its origin - run time, image, campaign - is kept and
exported);
3. otherwise it is **required** (reason: `always`, `never-passed`, or
`domain-changed`).
**Release authorized** = a campaign is open and every catalog test is
satisfied. There is no SKIP: a test the bench cannot run means the release
cannot be authorized (that is a catalog change, not a skip).
**Invalidate all** (page footer, reason required) records that the bench
itself changed - new tube, driver swap, cable work, a judgment call - and
forces a full campaign; nothing before it can be inherited.
## Running a campaign
1. Boot the dev image on the bench (`forgefirm-image-dev`), open
`http://<machine>:8090/`.
2. The banner shows the image, the manifest identity, and *Release
authorized*. Tests marked **required** need to run; **inherited** ones
do not.
3. Start the required tests. `operator` tests ask questions in the run
pane; `live` tests need the acknowledgment and the physical arm press;
`takeover` tests stop forgectrl for the duration.
4. When *Release authorized: YES*, **Export release artifact**, download
`acceptance.json` and `acceptance.md`, and commit them as
`releases/v<version>/acceptance.json` and `.md`.
The raw log (`/data/forgetest/results.jsonl`, `Raw log` in the footer) is
the bench's own record; the artifact is the release's.
## The gate
`scripts/release.sh <version>` builds the release image, reads
`/etc/forgefirm-manifest.json` out of the release rootfs and runs
scripts/acceptance-gate.py releases/v<version>/acceptance.json <manifest>
which requires: the artifact self-hash intact; `authorized: true`; the
catalog in the tree identical to the artifact's; for every test a recorded
PASS whose fingerprint equals the one recomputed from the release manifest;
inherited results not core and newer than the invalidate epoch. Any
problem dies before signing. `FORGEFIRM_ACCEPTANCE_SKIP=1` bypasses the gate
deliberately and prints a loud warning; it is never the default. The
artifact is staged and attached to the GitHub release next to
`forgefirm.fw`.
Because the dev image and the release image are built from the same tree
in one `bitbake` invocation, their manifests share the same identity; a pin
bumped after the campaign shows up as a fingerprint mismatch on exactly the
tests that cover it.
## Coverage currency rule
Every change is evaluated against the catalog, in addition to its unit
tests:
1. does an existing test exercise the changed behavior - if not, add or
extend one in the same change;
2. does that test's `covers` map name the files touched - if not, widen it
in the same change.
A behavior change with no catalog consequence needs a sentence of
justification in the commit message. Coverage gaps are defects: under the
domain model an uncovered path lets an inherited PASS stay valid across a
change that should have invalidated it.
The mechanical floor is the coverage lint,
python3 -m forgetest.coverage --manifest <manifest.json> [--enforce]
which lists every manifest path no test covers, minus the allowlist of
non-behavioral paths in `forgetest/forgetest/coverage.py` (docs, CI, tests,
licenses). CI (`forgetest-ci.yml`) runs it on a manifest generated from the
recipe pins with `scripts/manifest-from-tree.py` (no Yocto build needed)
and fails the job on any uncovered path. On the board, run it against
`/etc/forgefirm-manifest.json`. The lint proves a
file is *fingerprinted*; whether the test *exercises* the change is the
change author's judgment (rule 1).
## Bench diagnostics page
The same daemon serves `#bench`: the registry of the bench tools
(`scripts/bench`, installed under `/usr/share/forgetest/bench/`), each with
its safety class (`dry`, `takeover`, `live`, `scope`), argument form, and
last run. A ported tool runs as a subprocess with the output on the page;
unported tools are listed with Start disabled. Bench runs are recorded in
`/data/forgetest/bench.jsonl` and never enter a campaign.
## Layout
forgetest/forgetest/ the package (stdlib only)
manifest.py manifest, globs, fingerprints, coverage report
catalog.py @test registry, catalog hash
campaign.py the rules (pure functions)
artifact.py export + gate verification
runner.py one run at a time, prompts, abort, takeover
server.py / page.py HTTP API + the page (forgectrl's access rules)
bench.py / coverage.py bench registry + subprocess runner; the lint
suite/ the catalog, one module per subsystem
forgetest/tests/ host unit tests (python3 -m unittest discover -s tests)
scripts/acceptance-gate.py the gate
scripts/manifest-from-tree.py manifest from the recipe pins (CI, workstation)
releases/v<version>/ the committed artifacts
+81
View File
@@ -1379,6 +1379,69 @@ overrides the IE (DE applied while associated to the US AP), and
clearing reverts to the 00 hint. The UI labels the default
accordingly ("Automatic — AP country, else World").
## Release acceptance (forgetest, port 8090)
The release acceptance tool - the catalog, campaigns, domain
fingerprints, inheritance, the always-required core, invalidate-all,
the release gate, and the coverage currency rule - is specified in
`docs/ACCEPTANCE.md`; the tool lives in `forgetest/` and ships only on
the dev image (`forgetest` recipe, `/etc/init.d/forgetest`, HTTP :8090).
Status: **code landed 2026-08-15, host-verified and build-verified;
bench validation pending - ships with the next full image flash** (the
image manifest is an image change: `forgefirm-manifest.bbclass` entries
from every component recipe, the kernel and the module through
`do_deploy`, assembled by `forgefirm-image-manifest.bbclass` into
`/etc/forgefirm-manifest.json`, also deployed next to the image as
`*.forgefirm-manifest.json`). Build proof (dev image `20260815191634`,
built with the classes): the manifest carries all eight components
(forgectrl, grblhal-glowforge with the core submodule's files,
forgefirm-app merged from its three recipes, python3-gfhardware,
python3-gfutilities, kernel-module-glowforge and linux-fslc through the
deploy path, forgetest through the file mode), the DTB hashes and the
modules directory, and layer content hashes that are **byte-identical to
what `scripts/manifest-from-tree.py` computes on the workstation** - the
identity is content-defined, independent of the checkout's commit or
dirty state; forgetest is installed at S95 with the bench scripts. Host
proof: 44 unit tests (campaign
rules, fingerprints, artifact build + gate verification incl. the
negative fixtures - tampered artifact, covered-file change, platform
change, core inherited, stale invalidate, catalog change, implementation
change - and the runner + HTTP API end to end with a fake catalog and a
fake bench tool), the tree manifest generated from the recipe pins with
`scripts/manifest-from-tree.py` (submodule recursion verified on the
grblHAL core), the coverage lint reporting on it, and the gate refusing an
empty artifact cleanly; `.github/workflows/forgetest-ci.yml` runs the
same and **enforces the coverage lint** (every manifest path is covered:
0 uncovered on both the built manifest and the tree manifest). **Catalog
v1 is complete: 24 tests**, every one a port of a proven bench drill or
of a bench-verified check, with the recorded pass criteria: the core
`image.health`, `kernel.latch-locked-idle`, `kernel.k1-k2`,
`kernel.k3-unlock`, `kernel.fire-abu` (GATE A drills as takeover tests;
K3 and fire B/U prompt for the lid when `laser_pgood` reports HV good)
and `laser.emission-witness` (S400 square, emission peak -> 0, HV rise,
M2 job-based disarm, operator confirms the mark); `forgectrl.auth` /
`settings-bounds` / `panel-serves`, `logs.tree-tail-export` (sanitized
bundle carries no panel token); `motion.pacing`, `jog-roundtrip`,
`liveness-probe`, `cancel-abort`, `deadman` (SIGKILL / SIGSTOP->underrun
/ forgectrl restart mid-move, head returned by the kernel counters);
`cooling.flow-verify` (through forgectrl's diag runner) and
`fans-quiet-after-motion`; `laser.disarm-in-hold`, `expected-stop`
(POST /controller/stop mid-burn, then the operator-judged restart),
`kill-mid-fire`; `camera.snapshot`; `update.slots-and-signature`;
`cloud.mode-switch` (gfcloud comes up and records its service probe) and
`cloud.gfhome-homing`. Not in the catalog by design: the stale-origin
refusal after an underrun (config-dependent - GRBL mode permits unhomed
cutting, see the campaign notes above). The bench tab lists every
`scripts/bench` tool; runnable from the page: `check-pwm`,
`pacing-test`, `bench-m2`, `bench-phase2`, `cp-watchdog`, `accel-fast`,
`bump-seek`, `fire-test`, `gate-a-kernel`, `platform-drills`,
`flow-confirm`, `flow-sampler` (takeover tools get forgectrl stopped and
started around the run); the scope tools, the host-side flow
characterization tools, and the live drills stay ssh/host-run for now.
The coverage currency rule is in `CLAUDE.md`
"Working rules". Bench validation and the bench-tab ports are Next work
item 15.
## Hardware facts bank (measured)
- **DRV8825 stepper drivers wedge on 40 V rail glitches** (factory board;
@@ -2643,3 +2706,21 @@ accordingly ("Automatic — AP country, else World").
goes with the wrapper; a forced daemon crash logs the wrapper's
`exited (N) - respawning in 5 s` line under `forgectrl`.
15. **Release acceptance tool (forgetest) - CODE-COMPLETE 2026-08-15,
host- and build-verified; bench validation pending, ships with the
next full image flash.** Contract: `docs/ACCEPTANCE.md`; catalog v1
complete (24 tests, coverage lint enforced in CI, rule in
`CLAUDE.md`). **Images for the flash are archived under
`images/20260815193946/`** (release `…193946` + dev `…194415`, one
tree; the two manifests share the acceptance identity, the release
image carries no forgetest). Remaining, in order: (a) bench: boot
that dev image, run the catalog from `:8090` - the takeover, motion,
cooling, live
and cloud tests are ports of proven scripts and need their first run
on the machine (expect pass-criteria tuning: fan tach tolerance,
snapshot size floor, timeouts) - export, and drive one UI-only pin
bump to prove the inherited/required split; (b) the remaining
bench-tab ports (scope tools, host-side flow characterization, the
live drills - the catalog carries their acceptance forms); (c) the
first release runs the full
campaign and commits `releases/v<version>/acceptance.json`.
+7 -2
View File
@@ -148,8 +148,13 @@ demonstrably untouched.*
command (`--publish` runs it where gh is authenticated). Gates:
clean tree, version single-source, rootfs-vs-slot size
(warn ≥ 170 MiB / fail ≥ 195 MiB, under bitbake's own hard cap),
**installer-embedded pubkey must match the signing key**, and
factory-era fwup (0.14.2) verification of the packed archive.
**installer-embedded pubkey must match the signing key**,
factory-era fwup (0.14.2) verification of the packed archive, and the
**release acceptance gate**: `releases/v<version>/acceptance.json`
(exported by forgetest on the bench) must authorize the built rootfs
per `docs/ACCEPTANCE.md` - the gate recomputes every catalog test's
domain fingerprint from `/etc/forgefirm-manifest.json` inside the
release ext4. The artifact is attached to the GitHub release.
- One version source: `FORGEFIRM_RELEASE` = git tag =
`/etc/forgefirm-version` = `.fw` meta-version; the script enforces
agreement.
+9 -5
View File
@@ -111,11 +111,15 @@ config move in the right order. The sequence, with current status:
5. **GitHub release**: run `scripts/release.sh <version>` on the build
host. It gates (version single-source, rootfs-vs-slot size,
installer-embedded pubkey vs the signing key, factory-era fwup
verification), builds, packs and signs `forgefirm.fw`, stages the
assets with `sha256sums.txt`, and prints the `gh release create`
command. Assets and their exact names (the installer and the update
manager download them verbatim): `forgefirm.fw`, `sha256sums.txt`,
`forgefirm-image-glowforge.rootfs.wic.gz`. The release tag
verification, and the **acceptance gate** - the committed
`releases/v<version>/acceptance.json` from the bench campaign must
authorize the built rootfs, `docs/ACCEPTANCE.md`), builds, packs and
signs `forgefirm.fw`, stages the assets with `sha256sums.txt`, and
prints the `gh release create` command. Assets and their exact names
(the installer and the update manager download them verbatim):
`forgefirm.fw`, `sha256sums.txt`,
`forgefirm-image-glowforge.rootfs.wic.gz`, plus `acceptance.json` and
`acceptance.md`. The release tag
`v<version>` = `FORGEFIRM_RELEASE` = the rootfs `/etc/forgefirm-version`
= the `.fw` meta-version; `release.sh` enforces the agreement.
+6
View File
@@ -0,0 +1,6 @@
# Release acceptance artifacts
One directory per release, `v<version>/`, holding the `acceptance.json` and
`acceptance.md` that forgetest exported on the bench for that release.
`scripts/release.sh` refuses to sign a release whose artifact does not
authorize the built rootfs; see `docs/ACCEPTANCE.md`.
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
# (C) Copyright 2020-2026
# Scott Wiederhold, s.e.wiederhold@gmail.com
# https://community.openglow.org
# SPDX-License-Identifier: MIT
#
# Release acceptance gate: does the committed acceptance artifact authorize
# THIS release build?
#
# acceptance-gate.py <acceptance.json> <release-manifest.json> [--machine glowforge]
#
# The artifact is what forgetest exported on the bench (releases/v<version>/
# acceptance.json); the release manifest is /etc/forgefirm-manifest.json read
# out of the release rootfs release.sh just built. For every catalog test the
# gate recomputes the domain fingerprint from the release manifest with the
# catalog in this source tree - the same code the bench ran - and requires the
# recorded PASS to match. Inherited results must not be core tests and must
# be newer than the last invalidate-all. Exit 0 = authorized, 1 = refused,
# 2 = usage/load error. release.sh dies on anything but 0.
import argparse
import json
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "forgetest"))
from forgetest import artifact, catalog, manifest # noqa: E402
def main(argv=None):
ap = argparse.ArgumentParser(description="ForgeFIRM release acceptance gate")
ap.add_argument("artifact")
ap.add_argument("release_manifest")
ap.add_argument("--machine", default="glowforge")
ap.add_argument("--quiet", action="store_true")
args = ap.parse_args(argv)
try:
with open(args.artifact, "r", encoding="utf-8") as f:
art = json.load(f)
rel = manifest.Manifest.load(args.release_manifest)
except (OSError, ValueError) as e:
print("acceptance-gate: cannot load inputs: %s" % e, file=sys.stderr)
return 2
registry = catalog.load_suite()
tests = catalog.all_tests(registry)
ok, rows, problems = artifact.verify(art, rel, tests, catalog.catalog_hash(registry),
expect_machine=args.machine)
if not args.quiet:
print("acceptance artifact: image %s, campaign %s, exported %s, authorized=%s"
% (art.get("image", {}).get("version"), (art.get("campaign") or {}).get("id"),
art.get("exported_at"), art.get("authorized")))
print("release manifest: %s identity %s" % (rel.version, rel.identity_sha()[:16]))
if art.get("identity_sha") == rel.identity_sha():
print("identity: the release build's inputs are identical to the bench image's")
else:
print("identity: the release build differs from the bench image (per-test check decides)")
for r in rows:
flag = "ok " if r["ok"] else "FAIL"
print(" %s %-34s %s%s%s" % (flag, r["id"], "core " if r["always"] else "",
"inherited " if r["inherited"] else "",
("; ".join(r["why"]) if r["why"] else r.get("ts", ""))))
for p in problems:
print("PROBLEM: %s" % p)
print("acceptance gate: %s" % ("AUTHORIZED" if ok else "REFUSED"))
return 0 if ok else 1
if __name__ == "__main__":
sys.exit(main())
+188
View File
@@ -0,0 +1,188 @@
#!/usr/bin/env python3
# (C) Copyright 2020-2026
# Scott Wiederhold, s.e.wiederhold@gmail.com
# https://community.openglow.org
# SPDX-License-Identifier: MIT
#
# Build a ForgeFIRM image manifest from the source tree - the same file lists
# forgefirm-image-manifest.bbclass puts in the image, computed from the recipe
# pins with git instead of a Yocto build. For the coverage lint in CI and for
# checking a coverage map on a workstation; NOT a substitute for the image's
# manifest in the release gate (the platform section carries placeholders
# where only a build knows the answer: kernel config hash, modules dir, DTB).
#
# manifest-from-tree.py [--meta-openglow PATH] [--out manifest.json]
# [--cache DIR] [--kernel-srcrev REV]
#
# Component revisions come from the recipes in meta-forgefirm and the sibling
# meta-openglow checkout (default ../meta-openglow relative to this repo).
# Each pinned commit is fetched shallowly into --cache (default
# .manifest-cache/, gitignored) and listed with `git ls-tree`; a submodule
# gitlink is followed through .gitmodules.
import argparse
import hashlib
import json
import os
import re
import subprocess
import sys
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, os.path.join(REPO, "forgetest"))
from forgetest import manifest as manifest_mod # noqa: E402
RECIPES = [
# (component, recipe path relative to the repo or meta-openglow, layer)
("forgectrl", "meta-forgefirm/recipes-forgefirm/forgectrl/forgectrl.bb", "forgefirm"),
("grblhal-glowforge", "meta-forgefirm/recipes-forgefirm/grblhal-glowforge/grblhal-glowforge.bb", "forgefirm"),
("forgefirm-app", "meta-forgefirm/recipes-forgefirm/forgefirm-app/forgefirm-app.inc", "forgefirm"),
("kernel-module-glowforge", "meta-glowforge-bsp/recipes-kernel/kernel-modules/kernel-module-glowforge.bb", "meta-openglow"),
("python3-gfhardware", "meta-glowforge-bsp/recipes-devtools/python/python3-gfhardware.bb", "meta-openglow"),
("python3-gfutilities", "meta-openglow-core/recipes-devtools/python/python3-gfutilities_git.bb", "meta-openglow"),
]
CONTENT_LAYERS = {"meta-forgefirm": ("forgefirm", "meta-forgefirm"),
"meta-glowforge-bsp": ("meta-openglow", "meta-glowforge-bsp"),
"meta-openglow-core": ("meta-openglow", "meta-openglow-core")}
def git(args, cwd=None, input=None):
return subprocess.run(["git"] + args, cwd=cwd, input=input, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, check=True).stdout
def parse_recipe(path):
text = open(path, encoding="utf-8").read()
uri = re.search(r'^SRC_URI\s*\+?=\s*"([^"]+)"', text, re.M)
rev = re.search(r'^SRCREV\s*\??=\s*"([0-9a-fA-F]+)"', text, re.M)
if not uri or not rev:
raise SystemExit("cannot find SRC_URI/SRCREV in %s" % path)
first = uri.group(1).split()[0]
url = first.split(";")[0]
params = dict(p.split("=", 1) for p in first.split(";")[1:] if "=" in p)
proto = params.get("protocol", "https")
if url.startswith("git://") or url.startswith("gitsm://"):
url = proto + "://" + url.split("://", 1)[1]
return url, rev.group(1)
def fetch(url, rev, cache):
"""A bare cache repo containing rev (fetched shallowly)."""
name = hashlib.sha1(url.encode()).hexdigest()[:16]
repo = os.path.join(cache, name)
if not os.path.isdir(repo):
os.makedirs(repo)
git(["init", "-q", "--bare"], cwd=repo)
try:
git(["cat-file", "-e", rev + "^{commit}"], cwd=repo)
except subprocess.CalledProcessError:
git(["fetch", "-q", "--depth", "1", url, rev], cwd=repo)
return repo
def ls_tree(repo, rev, url, cache, prefix, files):
out = git(["ls-tree", "-r", "--full-tree", rev], cwd=repo).decode()
modules = None
for line in out.splitlines():
if not line.strip():
continue
meta, path = line.split("\t", 1)
typ, obj = meta.split()[1], meta.split()[2]
files.append([prefix + path, obj])
if typ == "commit":
if modules is None:
modules = {}
try:
gm = git(["show", "%s:.gitmodules" % rev], cwd=repo).decode()
except subprocess.CalledProcessError:
gm = ""
cur = None
for l in gm.splitlines():
l = l.strip()
m = re.match(r'^path\s*=\s*(.+)$', l)
if m:
cur = m.group(1).strip()
m = re.match(r'^url\s*=\s*(.+)$', l)
if m and cur:
modules[cur] = m.group(1).strip()
sub_url = modules.get(path)
if sub_url:
if sub_url.startswith("../") or sub_url.startswith("./"):
base = url.rsplit("/", 1)[0]
sub_url = base + "/" + sub_url.lstrip("./")
sub_repo = fetch(sub_url, obj, cache)
ls_tree(sub_repo, obj, sub_url, cache, prefix + path + "/", files)
def layer_content(path):
out = git(["ls-files", "-z", "--cached", "--others", "--exclude-standard", "--", "."], cwd=path)
paths = sorted(set(p.decode("utf-8", "replace") for p in out.split(b"\0") if p))
paths = [p for p in paths if os.path.isfile(os.path.join(path, p)) and not p.endswith(".md")]
if not paths:
return None
# hash-object --stdin-paths resolves against the repository top level
prefix = git(["rev-parse", "--show-prefix"], cwd=path).decode().strip()
ids = git(["hash-object", "--stdin-paths"], cwd=path,
input=("\n".join(prefix + p for p in paths) + "\n").encode()).decode().split()
h = hashlib.sha256()
for p, i in zip(paths, ids):
h.update(p.encode("utf-8") + b"\0" + i.encode("ascii") + b"\n")
return h.hexdigest()
def main(argv=None):
ap = argparse.ArgumentParser()
ap.add_argument("--meta-openglow", default=os.path.join(os.path.dirname(REPO), "meta-openglow"))
ap.add_argument("--out", default="-")
ap.add_argument("--cache", default=os.path.join(REPO, ".manifest-cache"))
ap.add_argument("--kernel-srcrev", default=None,
help="linux-fslc SRCREV (default: read from layers/meta-freescale if present)")
args = ap.parse_args(argv)
os.makedirs(args.cache, exist_ok=True)
components = {}
for name, rel, layer in RECIPES:
base = REPO if layer == "forgefirm" else args.meta_openglow
path = os.path.join(base, rel)
url, rev = parse_recipe(path)
repo = fetch(url, rev, args.cache)
files = []
ls_tree(repo, rev, url, args.cache, "", files)
files.sort()
components[name] = {"srcrev": rev, "source": url, "files": files, "recipes": [os.path.basename(rel)]}
print("%s: %s (%d files)" % (name, rev[:12], len(files)), file=sys.stderr)
ksrc = args.kernel_srcrev
if not ksrc:
for cand in ("layers/meta-freescale/recipes-kernel/linux/linux-fslc_6.12.bb",):
p = os.path.join(REPO, cand)
if os.path.exists(p):
m = re.search(r'^SRCREV\s*=\s*"([0-9a-f]+)"', open(p, encoding="utf-8").read(), re.M)
if m:
ksrc = m.group(1)
components["linux-fslc"] = {"srcrev": ksrc, "source": "git://github.com/Freescale/linux-fslc.git",
"config_sha256": None, "recipes": ["linux-fslc"],
"files": [["@config", "unknown-without-a-build"], ["@srcrev", ksrc or "unknown"]]}
layers = {}
for lname, (repo_key, sub) in CONTENT_LAYERS.items():
base = REPO if repo_key == "forgefirm" else args.meta_openglow
lpath = os.path.join(base, sub)
if os.path.isdir(lpath):
layers[lname] = {"content_sha256": layer_content(lpath)}
platform = {"machine": "glowforge", "layers": layers, "kernel_modules": [], "dtb": {}}
canonical = manifest_mod.canonical({"components": components, "platform": platform})
out = {"format": 1, "image": {"name": "tree", "version": "tree (no build)"},
"content_sha256": hashlib.sha256(canonical.encode()).hexdigest(),
"components": components, "platform": platform}
text = json.dumps(out, sort_keys=True, indent=1) + "\n"
if args.out == "-":
sys.stdout.write(text)
else:
with open(args.out, "w", encoding="utf-8") as f:
f.write(text)
print("wrote %s" % args.out, file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main())
+37 -2
View File
@@ -8,6 +8,8 @@
#
# release.sh <version> [--publish] full release: gates, build, pack,
# sign, checksums, stage, publish cmd
# (the acceptance gate reads
# releases/v<version>/acceptance.json)
# release.sh --dev build + pack a dev-signed .fw for
# the GUI upload path; no staging
#
@@ -21,6 +23,8 @@
# FORGEFIRM_DEV_KEY private key for --dev mode (REQUIRED for --dev)
# RELEASE_STAGING_DIR where release assets are staged
# (default: <repo>/release-staging)
# FORGEFIRM_ACCEPTANCE_SKIP set to 1 to bypass the acceptance gate
# deliberately (never the default; docs/ACCEPTANCE.md)
#
# Version contract: <version> == FORGEFIRM_RELEASE in forgefirm-image.bb
# == /etc/forgefirm-version ("v<version>") in the built rootfs == .fw
@@ -135,6 +139,27 @@ STAMP=$(debugfs -R "cat /etc/forgefirm-version" "$EXT4" 2>/dev/null)
[ "$STAMP" = "v$VERSION" ] \
|| die "rootfs stamp is '$STAMP', expected 'v$VERSION'"
# Acceptance gate: the committed acceptance artifact must authorize THIS
# build. scripts/acceptance-gate.py recomputes every catalog test's domain
# fingerprint from the manifest inside the release rootfs and requires the
# recorded PASS to match (docs/ACCEPTANCE.md). A release is never signed
# without it; FORGEFIRM_ACCEPTANCE_SKIP=1 bypasses deliberately and loudly.
ART="$REPO/releases/v$VERSION/acceptance.json"
if [ -n "${FORGEFIRM_ACCEPTANCE_SKIP:-}" ]; then
warn "acceptance gate SKIPPED by FORGEFIRM_ACCEPTANCE_SKIP - this release carries no acceptance proof"
else
[ -f "$ART" ] \
|| die "no acceptance artifact at releases/v$VERSION/acceptance.json - run the campaign on the bench, export, commit"
REL_MANIFEST=$(mktemp)
debugfs -R "cat /etc/forgefirm-manifest.json" "$EXT4" > "$REL_MANIFEST" 2>/dev/null
[ -s "$REL_MANIFEST" ] \
|| { rm -f "$REL_MANIFEST"; die "release rootfs carries no /etc/forgefirm-manifest.json"; }
python3 "$REPO/scripts/acceptance-gate.py" "$ART" "$REL_MANIFEST" --machine glowforge \
|| { rm -f "$REL_MANIFEST"; die "acceptance gate refused this build (see the table above)"; }
rm -f "$REL_MANIFEST"
echo "acceptance gate OK ($ART)"
fi
# Back-door gate: the release image must not ship a passwordless root. A
# debug-tweaks image sets root's password field empty (root::...); a
# hardened image leaves it locked (root:*: / root:!:) or hashed. Read the
@@ -177,6 +202,16 @@ fi
echo "== stage assets =="
cp -L "$DEPLOY/forgefirm-image-glowforge.rootfs.wic.gz" "$STAGE/forgefirm-image-glowforge.rootfs.wic.gz"
# The acceptance artifact travels with the release (docs/ACCEPTANCE.md).
ASSETS="forgefirm.fw sha256sums.txt forgefirm-image-glowforge.rootfs.wic.gz"
if [ -f "$ART" ]; then
cp "$ART" "$STAGE/acceptance.json"
ASSETS="$ASSETS acceptance.json"
if [ -f "${ART%.json}.md" ]; then
cp "${ART%.json}.md" "$STAGE/acceptance.md"
ASSETS="$ASSETS acceptance.md"
fi
fi
( cd "$STAGE" && sha256sum forgefirm.fw forgefirm-image-glowforge.rootfs.wic.gz > sha256sums.txt )
ls -la "$STAGE"
@@ -193,14 +228,14 @@ Publish (from a directory with an authenticated gh):
cd "$STAGE"
gh release create "v$VERSION" --repo ScottW514/forgefirm \\
--title "ForgeFIRM v$VERSION" --generate-notes \\
forgefirm.fw sha256sums.txt forgefirm-image-glowforge.rootfs.wic.gz
$ASSETS
EOF
if [ "$PUBLISH" = "1" ]; then
command -v gh >/dev/null || die "--publish requested but gh is not on PATH"
( cd "$STAGE" && gh release create "v$VERSION" --repo ScottW514/forgefirm \
--title "ForgeFIRM v$VERSION" --generate-notes \
forgefirm.fw sha256sums.txt forgefirm-image-glowforge.rootfs.wic.gz ) \
$ASSETS ) \
|| die "gh release create failed"
echo "== published v$VERSION =="
fi