ScottW514
35221de791
The queues route a test by its kind and its declared actions: an auto test runs unattended with or without the bench fixture, and an operator test whose actions the fixture covers runs unattended only while it does. exthost.events opens and closes the lid through Context.act but was declared auto with no actions, so the unattended queue ran it with no fixture on the network, and it waited 180 s for a person who was not there: the fx29 campaign (image 20261010213702) ended on "lid open was not seen on the machine within 180 s" with forgefixture.local silent to mDNS and 172.16.1.171 silent to ping. - exthost.events: operator, actions lid, with the operator's steps. - events.button-telemetry: the same defect with the button; operator, actions button, with the operator's steps. - setup.check-switches: declares interlock beside lid and button; the check works the interlock loop on a Pro. test_fixture's new CatalogTests case reads every catalog test's source and fails a test that calls Context.act on a channel it does not declare, or that is auto and calls it at all. It found these three and no other. Fingerprints: the three decorators are each test's own text, so these three move and no other (diffed across the change, 128 tests). Proof: the whole host suite passes (509 tests) and pyflakes finds no undefined name. RoutingTests already prove the routing these declarations feed: an operator test the fixture covers moves to the unattended queue, one it does not stays attended, and with no fixture the routing is the old one. Acceptance: the change is the three tests' declarations; with the fixture up each runs unattended through it as before, and with the fixture down each stays in the attended list instead of stranding the unattended queue.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.