mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-10-15 17:19:55 -07:00
ScottW514
b90299acdc
Four catalog tests leave for the host proofs that carry them; the
catalog goes from 87 to 83 (83 @test in forgetest/forgetest/suite/).
Each proof passed; none skipped.
- exthost.package-routes, for forgectrl's tests/host_ext.sh (05848ac):
against a compiled stand-in that is a live host, GET /ext/status and
POST /ext/package are 403 without the token with the host not run,
the status passes the host's list through with host.running true and
the stand-in's pid, enable runs as the host's command, a bad action
and a bad id are 400 with nothing run, and a package that is not
installed is 409 in the host's words. extpkg_test holds the relay's
argv; the state file, the required-holds directory and removal are
forgeext's install_test.py and run_test.py; the page card is
check_ui_js.py. REQUIRED_HOLDS, which only it read, and the "the hold"
section header over it go.
- exthost.wizard, for host_ext.sh (05848ac): the check listed, run to its
prompt with no lease holder, answered yes, ended with its summary and
listed done; the setup record's wizards and the gate as they were; a
second run aborted; a check of a package that is not installed 404.
wizpkg_test holds each step to its form. suite/extwizard.py held only
it: the module and its import go.
- exthost.catalog: install_test.py and extpkg_test already assert its
behavioral claims, and the one fact the bench test did not prove, that
the image's extension key is the catalog's published key, is now a
forgetest-ci step: the catalog is checked out at main as a sibling and
meta-forgefirm/recipes-forgefirm/forgefirm-keys/files/ext/forgefirm-ext.pub
is cmp -s'd against its keys/forgefirm-ext.pub, either one missing a
failure. suite/extcat.py held only it: the module and its import go.
- camera.key-read, for forgectrl's tests/host_lan.sh (05848ac): with
panel_open_reads=0, GET /cam/status from the LAN client over HTTP and
HTTPS is 403 with no key or a wrong one and 200 with the key as ?key=
or the header, the key alone writes no setting, and a rotation stops
the old key and the new one reads. camera.py's json import, which only
it used, goes.
The coverage lint then named forgectrl src/wizpkg.c, src/wizpkg.h,
src/camkey.c and src/camkey.h, and nothing else. Two HOST_TESTED rows
take them, to forgectrl:tests/wizpkg_test.c and
forgectrl:tests/camkey_test.c on forgectrl:.github/workflows/build.yml,
which the pin c274f19 runs (./build/wizpkg_test, ./build/camkey_test).
src/extpkg.* stays covered (logs.tree-tail-export, logs.routing), and
forgeext/** by exthost.service.
The key step's run text, taken from the workflow, passes from the tree
root with both files present; controls on a /tmp copy, a published key
that differs and one that is missing: each fails. test_manifest.py (34)
passes. The coverage lint on the tree manifest finds no uncovered path
and no host-tested row problem across the 83 tests; its one finding is
setup.check-lens's empty src/lensfind.* entry. pyflakes reports nothing
on exthost.py, camera.py or manifest.py. The full forgetest suite was
not run.
Fingerprints, measured: 10 tests move, the 5 left in exthost.py
(armed-freeze, motion-jog, motion-job, platform, service) and the 5 left
in camera.py (frame-health, h264-stream, lid-privacy, sensor-profile,
snapshot). The pending pin bumps from forgectrl c274f19 and forgeext
2d0391e already owe 9 of them; exthost.platform is owed already since
30c0a05. This adds none. releases/ and TEST-AUDIT.md are unchanged.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.