ScottW514
b447ae77f1
Three catalog tests leave for the host proofs that carry them; the catalog goes from 95 to 92. - setup.https-only-writes, for forgectrl's tests/host_lan.sh (13273a0): from a LAN peer in a network namespace, reads served over HTTPS and HTTP, three writes over HTTP each a 302 to the same path on HTTPS that acts on nothing, the same write served from loopback, and the token writing over HTTPS with no account. - update.release-check and update.job-locks, together, for forgectrl's tests/host_update.sh (acc3dcf), against a release server of its own: GET /update/release's keys, POST /update/check with a v<semver> tag, a size above 0, notes and new; the dismissal refused for a version that is not a tag, taken for the tag, cleared by an empty one; and a probe download that holds the machine (a settings write and a G21 job each 409 in the job's name, update:download of kind system the holder), ends refused by the signature check leaving no file, and lets the same settings write through after. update.release-check was the last test in suite/update.py; _VERSION_RX and the re import, which only it used, go with it. update.job-locks was the only test in suite/updlock.py: the module and its import in suite/__init__.py go. setup.https-only-writes was the only test of setup.py's "HTTPS boundary" section. tests/test_setup_suite.py drops its id. update.product-gate stays: fwup is on neither this host's PATH nor CI's, so host_update.sh does not prove the five uploads, and no HOST_TESTED row is added for src/fwproduct.*. No other HOST_TESTED row either. test_setup_suite.py passes (31 tests). The coverage lint on the tree manifest finds no uncovered path and no host-tested row problem across the 92 tests; its one finding is setup.check-lens's empty src/lensfind.* entry. pyflakes reports only the unused local plain in update.slots-and-signature, which HEAD already has. The full forgetest suite was not run. Fingerprints: setup.py's and update.py's shared text move, so 33 tests move: the 6 left in setup.py, the 25 in the modules that read it, and update.slots-and-signature and update.product-gate. The pending pin bump from c274f19 already owes 31 of them; setup.machine-name and exthost.platform are the two it adds, as before. releases/ and TEST-AUDIT.md are unchanged.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.