manifest: component pins are not layer content

A component pin bump counted as a platform change: the layer content hash
in the platform identity covered the recipe carrying the SRCREV, the
platform is folded into every acceptance fingerprint, so every image
that carried any component update invalidated the whole catalog (dev
image 20260816191951: every test domain-changed after a one-line
forgectrl bump; the two manifests differ only in
platform.layers.meta-forgefirm). The component entry already identifies
the pinned source file by file; the pin double-counted it.

Component pins now live in <recipe>-pin.inc (SRCREV and the PV that
moves with it, nothing else) - forgectrl, grblhal-glowforge and
forgefirm-app here, the BSP components in meta-openglow - and
forgefirm-image-manifest.bbclass leaves *-pin.inc out of the layer
content (FORGEFIRM_MANIFEST_PIN_SUFFIX). Recipe bodies, patches, config
fragments, init scripts and third-party pins with no manifest entry stay
layer content; a pin written into a recipe body still hashes (the safe
direction). manifest-from-tree.py mirrors the rule and reads pins
through the recipe's requires; test_tree_manifest.py proves both
(pin bump: hash unchanged; recipe body or inline pin: changed).
Bitbake resolves the same SRCREV/PV for every pinned recipe.

Docs: ACCEPTANCE.md (what layer content is), kas/README.md (the pin
files in the push order), BRINGUP.md (the finding and the bench
consequence: the first image built with the pin files is itself a
platform change, so its campaign is a full one; pin bumps inherit
after it).

No catalog consequence: nothing in the image's behavior changes; the
change is to the acceptance identity computation, proven by the unit
tests and the CI lint on the tree manifest.
This commit is contained in:
ScottW514
2026-08-16 16:06:58 -04:00
parent bf066d4e27
commit b51e695fb1
12 changed files with 254 additions and 36 deletions
@@ -22,6 +22,19 @@
# host-specific (paths, user, time) is recorded: the manifest travels in
# a public release artifact.
#
# Two kinds of file are left out of a layer's content: documentation
# (*.md) and component pin files (*FORGEFIRM_MANIFEST_PIN_SUFFIX,
# "<recipe>-pin.inc"). A pin file holds nothing but the SRCREV (and the PV
# that rides with it) of a component that has its own manifest entry; the
# entry already identifies that source file by file, so hashing the pin as
# layer content would turn every component update into a platform change
# and invalidate every acceptance result instead of the tests that cover
# the component. Everything else a recipe carries - build flags, patches,
# config fragments, init scripts, the pins of third-party sources that
# have no manifest entry - is layer content and stays in the hash. A pin
# written into a recipe body instead of its pin file still hashes: the
# safe direction (a full campaign, not a missed one).
#
# content_sha256 is the identity of the build's inputs: sha256 over the
# canonical JSON (sorted keys, no whitespace) of {"components", "platform"}.
# The image name, version string and the build section are metadata
@@ -30,19 +43,22 @@
FORGEFIRM_MANIFEST_DIR ?= "${sysconfdir}/forgefirm-manifest.d"
FORGEFIRM_MANIFEST_CONTENT_LAYERS ?= "meta-forgefirm meta-glowforge-bsp meta-openglow-core"
FORGEFIRM_MANIFEST_PIN_SUFFIX ?= "-pin.inc"
do_rootfs[depends] += "virtual/kernel:do_deploy kernel-module-glowforge:do_deploy"
ROOTFS_POSTPROCESS_COMMAND += "forgefirm_manifest_assemble;"
forgefirm_manifest_assemble[vardepsexclude] += "DATETIME"
def forgefirm_manifest_layer_content(path):
def forgefirm_manifest_layer_content(path, skip_suffixes=('.md',)):
"""sha256 over (path, git blob id) of every file under the layer
directory except those whose name ends in one of skip_suffixes."""
import hashlib, os, subprocess
out = subprocess.run(['git', 'ls-files', '-z', '--cached', '--others', '--exclude-standard', '--', '.'],
cwd=path, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True).stdout
paths = sorted(set(p.decode('utf-8', 'replace') for p in out.split(b'\0') if p))
paths = [p for p in paths
if os.path.isfile(os.path.join(path, p)) and not p.endswith('.md')]
if os.path.isfile(os.path.join(path, p)) and not p.endswith(tuple(skip_suffixes))]
if not paths:
return None
# hash-object --stdin-paths resolves against the repository top level
@@ -68,6 +84,7 @@ def forgefirm_manifest_layers(d):
and dirty flag of every layer checkout (informational)."""
import os, subprocess
content_layers = (d.getVar('FORGEFIRM_MANIFEST_CONTENT_LAYERS') or '').split()
skip = ('.md',) + tuple((d.getVar('FORGEFIRM_MANIFEST_PIN_SUFFIX') or '').split())
identity, build = {}, {}
for layer in (d.getVar('BBLAYERS') or '').split():
name = os.path.basename(layer.rstrip('/'))
@@ -83,7 +100,7 @@ def forgefirm_manifest_layers(d):
if name in content_layers:
if rev is None:
bb.fatal("forgefirm-image-manifest: layer %s must be a git checkout to be content-hashed" % name)
identity[name] = {'content_sha256': forgefirm_manifest_layer_content(layer)}
identity[name] = {'content_sha256': forgefirm_manifest_layer_content(layer, skip)}
else:
identity[name] = {'rev': rev}
return identity, build
@@ -0,0 +1,6 @@
# forgectrl pin. Bump deliberately after pushing forgectrl changes; keep
# only SRCREV and PV here - the image manifest leaves *-pin.inc out of the
# layer content hash because the component entry already identifies the
# pinned source (forgefirm-image-manifest.bbclass).
SRCREV = "c8f655813ba2b9a44fdf032e84435d9454909d7c"
PV = "0.1.0"
@@ -5,11 +5,10 @@ LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=19ed4e3e8c28a4311c16b0b2b91357ec"
PE = "1"
PV = "0.1.0"
SRC_URI = "git://github.com/ScottW514/forgectrl.git;protocol=https;branch=main"
# Pinned; bump deliberately after pushing forgectrl changes.
SRCREV = "c8f655813ba2b9a44fdf032e84435d9454909d7c"
# SRCREV and PV live in the pin file (forgefirm-image-manifest.bbclass).
require forgectrl-pin.inc
S = "${WORKDIR}/git"
@@ -0,0 +1,12 @@
# forgefirm-app pin (python3-gfhardware repository, forgefirm-app/ tree).
# Bump deliberately (AUTOREV is not reproducible); keep only SRCREV and PV
# here - the image manifest leaves *-pin.inc out of the layer content hash
# because the component entry already identifies the pinned source
# (forgefirm-image-manifest.bbclass). The python3-gfhardware recipe in
# meta-glowforge-bsp pins the same repository; move both together.
SRCREV = "a0a174d8e9a6b4b6e3abe63acc51018199df4202"
# Bump PV with every SRCREV move: the hash-derived package version is not
# monotonic on its own and buildhistory QA fails the build when it sorts
# backwards.
PV = "0.1.6+git"
@@ -10,13 +10,8 @@ LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=62f8bb455fcc4bf177ecab380f71cd5d"
SRC_URI = "git://github.com/ScottW514/python3-gfhardware.git;protocol=https;branch=master"
# Pinned; bump deliberately (AUTOREV is not reproducible).
SRCREV = "a0a174d8e9a6b4b6e3abe63acc51018199df4202"
# Bump PV with every SRCREV move: the hash-derived package version is not
# monotonic on its own and buildhistory QA fails the build when it sorts
# backwards.
PV = "0.1.6+git"
# SRCREV and PV live in the pin file (forgefirm-image-manifest.bbclass).
require forgefirm-app-pin.inc
S = "${WORKDIR}/git"
@@ -0,0 +1,6 @@
# grblHAL-glowforge pin. Bump deliberately after pushing grblHAL-glowforge
# changes; keep only SRCREV and PV here - the image manifest leaves
# *-pin.inc out of the layer content hash because the component entry
# already identifies the pinned source (forgefirm-image-manifest.bbclass).
SRCREV = "67d026d39ac03286937774f5e28946b88c44d490"
PV = "0.1.0"
@@ -4,14 +4,12 @@ HOMEPAGE = "https://github.com/ScottW514/grblHAL-glowforge"
LICENSE = "GPL-3.0-or-later"
LIC_FILES_CHKSUM = "file://COPYING;md5=3237e48bcef3455c7bea5c0ce16206f6"
PV = "0.1.0"
# gitsm: the grblHAL core rides as a submodule (ScottW514/core, branch
# forgefirm = upstream master + the step_us_min buffer fix pending
# upstream).
SRC_URI = "gitsm://github.com/ScottW514/grblHAL-glowforge.git;protocol=https;branch=main"
# Pinned; bump deliberately after pushing grblHAL-glowforge changes.
SRCREV = "67d026d39ac03286937774f5e28946b88c44d490"
# SRCREV and PV live in the pin file (forgefirm-image-manifest.bbclass).
require grblhal-glowforge-pin.inc
SRC_URI += "file://grblhal.init"