ScottW514
5bcc536853
workflow_steps rebuilt the first line of a step with its first key two columns past the dash, whatever followed the dash. A step written with more than one space after its dash (`- run: ./build/tray_test`, its `if: false` at the column of the run key) then had its first key at one column and the others at another: every key after the first was read as a child of the first and dropped, `if:`, `continue-on-error:`, `working-directory:` and `with:` among them. So such a step under `if: false` counted as running its host test, and such a checkout under a condition counted as a checkout. _KEY_RX did not read a quoted key, so a step's `"if": false` was dropped the same way. Both spellings are valid YAML that GitHub skips or tolerates. The parser now replaces the dash with a space and realigns nothing, so the key column of a step is the column its keys use. _KEY_RX reads a key in single or double quotes as the same key. The HOST_TESTED comment states the rule the lint applies: no `if:` other than `true` and no `continue-on-error:` other than `false` on the step or its job. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 38 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12, with test_a_step_counts_only_when_every_run_of_the_workflow_runs_it_and_its_failure_fails_the_run extended by a step with three spaces after its dash under `if: false` and a step under a quoted `"if": false`, neither running the test. Negative controls on scratch copies: the first line rebuilt as before fails the three-space case; _KEY_RX as before fails the quoted case. The 17 workflows of the tree's repositories (205 steps) parse to the same steps and events before and after the change. The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: none. forgetest is dev-only and in no fingerprint (the source_sha of 43 of 43 tests is unchanged); the lint runs in CI, not in a campaign, and tests/ are in no fingerprint.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.