ScottW514
b51e695fb1
A component pin bump counted as a platform change: the layer content hash in the platform identity covered the recipe carrying the SRCREV, the platform is folded into every acceptance fingerprint, so every image that carried any component update invalidated the whole catalog (dev image 20260816191951: every test domain-changed after a one-line forgectrl bump; the two manifests differ only in platform.layers.meta-forgefirm). The component entry already identifies the pinned source file by file; the pin double-counted it. Component pins now live in <recipe>-pin.inc (SRCREV and the PV that moves with it, nothing else) - forgectrl, grblhal-glowforge and forgefirm-app here, the BSP components in meta-openglow - and forgefirm-image-manifest.bbclass leaves *-pin.inc out of the layer content (FORGEFIRM_MANIFEST_PIN_SUFFIX). Recipe bodies, patches, config fragments, init scripts and third-party pins with no manifest entry stay layer content; a pin written into a recipe body still hashes (the safe direction). manifest-from-tree.py mirrors the rule and reads pins through the recipe's requires; test_tree_manifest.py proves both (pin bump: hash unchanged; recipe body or inline pin: changed). Bitbake resolves the same SRCREV/PV for every pinned recipe. Docs: ACCEPTANCE.md (what layer content is), kas/README.md (the pin files in the push order), BRINGUP.md (the finding and the bench consequence: the first image built with the pin files is itself a platform change, so its campaign is a full one; pin bumps inherit after it). No catalog consequence: nothing in the image's behavior changes; the change is to the acceptance identity computation, proven by the unit tests and the CI lint on the tree manifest.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
docs/ACCEPTANCE.md.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine>:8080 python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1:8080, /data/forgefirm/panel.token |
forgectrl client |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, covers, requires,
always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.