ScottW514
7a71dd80b5
_imported walked a relative import only when it names a module (`from .x import y`): a level-1 ImportFrom with no module, `from . import x`, was passed over, so the suite module x joined no hash through it. The guard on the suite's package imports reads levels 2 and 0 alone, so it passed that spelling, and its own message sends an absolute import there: "import it relatively (from .. or from .)". So the guard and the walk did not read the same spellings: on a scratch copy of the catalog, forgectrl.auth's import of setup.py written `from . import setup` passed the guard, and an edit to setup.py's decode(), which the test calls, left forgectrl.auth's implementation hash where it was. homeoff.py imports cloud.py so (`from . import cloud as logs`); its `from .cloud import log_lines_since, log_size` on the next line keeps cloud.py in the walk, so no hash misses a module. _imported now walks every level-1 relative import: one with no module walks each name it imports as <suite>/<name>.py, the name and not its alias (a name that is no module of the suite is passed over, as a module was). The docstrings of _imported and sibling_imports, and the comment of the guard's case, name both spellings. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12. test_a_sibling_imported_inside_a_function_moves_the_tests_that_run_it gains a leg in which m.two imports judge.py as `from . import judge as j`: an edit to judge.py moves m.two and not m.one. Negative controls on scratch copies: catalog.py as it was fails that leg (the two hashes are equal), and so does the walk that reads the alias (j) in place of the name. On a scratch copy of the catalog with forgectrl.auth's import written `from . import setup`, an edit to setup.py's decode() now moves forgectrl.auth's hash. The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: none. The source_sha of 43 of 43 tests is unchanged: the one module-less sibling import outside the registry (suite/__init__.py, which has no tests) is homeoff.py's, and cloud.py is in its walk already. catalog.py is the registry and no shared judge, forgetest is in no fingerprint, and tests/ are in no fingerprint.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine> python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1, /data/forgefirm/panel.token |
forgectrl client (HTTP; the token authorizes writes from the board) |
FORGECTRL_TLS_URL |
https://127.0.0.1 |
forgectrl over HTTPS (self-signed, unverified), for the login test |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.