ScottW514
7f2da68e34
Each left after its host proofs passed, with the coverage lint clean at the pins, its only failure the known setup.check-lens entry on src/lensfind.*: - image.license-bundle: the pack half is test_image.py's PackLicensesTests on pack-licenses.sh (forgefirm0d63f87); the serve half is host_routes.sh's license cases, the manifest, the bundle and the Licenses page read with no token (forgectrl c9e3347). Its src/main.c stays covered by image.health, and its three pages by forgectrl.panel-serves (src/ui/**). The 40-package and 5-text counts leave with it. - image.network-boot: the interfaces file's DHCPv6 check is NetworkBootTests' case on the BSP layer's file (forgefirm0d63f87). The board checks move into image.health, unchanged: wlan0 in ifup's state file and no ifup or ifdown running; a getty on ttymxc0; at least one udhcpc, each with -b as its own word and parent pid 1; no udhcpc6, dhcp6c or odhcp6c running or on the PATH, and no /usr/share/udhcpc/default6.script; on wlan0 disable_ipv6 0, accept_ra neither empty nor 0, and a link-local address, the global address logged. image.health's description says so. The interfaces file carries no -b, so -b is checked here, on the client that runs. No HOST_TESTED row: network-boot covered nothing, and license-bundle's paths stay covered by the two tests that keep them. image.health's covers are unchanged. The helpers ifstate_names, dhcp6_in_interfaces, parse_stat, link_addrs, _procs and _read, and the module's constants, stay as they were; only the two functions and the inside of image_health changed. Fingerprints: image.health's moved and no other id's (the module's shared text, which every exthost test reads through its import of kernel_config, is unchanged). image.health is always-required, so the next campaign reruns it. Proof: the coverage lint, 53 tests (forgectrl, src/lensfind.* its one entry); pyflakes; forgetest's unit suite on Linux (python3 -B -m unittest discover -s tests), 335 OK.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.