ScottW514
b4493f5052
The soft-reset check's reference verdict is the homed mask, read before
and after the reset through forgectrl's GET /motion/state
(cb_motion_state in src/main.c, through motion_port to the controller
port's state record), because $# prints the mask only with $22 on,
which the driver leaves off. A test that depends on a forgectrl route
names ("forgectrl", "src/main.c") in its covers, as microstep_modes
and deadman do, so that a change to the route invalidates the test's
inherited PASS. motion.jog-roundtrip's covers are now _MOTION_COVERS
plus that entry.
Proof: a compile and pyflakes 4.0.3 give no message on
forgetest/suite/motion.py, with no undefined name in forgetest or
tests. forgetest unit tests, python3 -B -m unittest discover -s tests
-p test_<name>.py, in a private PID namespace: test_manifest.py 39
OK, test_queue.py 15 OK, test_campaign.py 13 OK. Coverage lint on the
tree manifest from scripts/manifest-from-tree.py (python3 -B -m
forgetest.coverage --manifest ../tree-manifest.json --enforce):
0 uncovered, 0 empty, 0 host-tested row problems, 43 tests; the
catalog loads motion.jog-roundtrip with the five covers entries. A
host dry run of the whole test against the null-sink build of the
grblHAL-glowforge default branch head ("serial_test: the scan's exempt
arguments each match once in the sources"), with the machine side
stood in for as before and a model kernel that ramps from the
controller's cnc/stop write, passes: MPos X 3534.93 steps against
3535, tail 23% of the tick, mask 4 kept, drift 0.000 mm.
Acceptance: the covers are part of the test's own text, so the
fingerprint of motion.jog-roundtrip moves, and no other test's: of
the implementation hashes of the 43 catalog tests, before and after,
only that one differs. From here a change under forgectrl's
src/main.c invalidates motion.jog-roundtrip's inherited PASS as well.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.