ScottW514
7605a90946
update.slots-and-signature's apply section required 200 from
POST /update/apply, where the daemon answers 202 with started, like
every job endpoint, so its first bench run on image 20260902144848
ended before the job did; the cleanup then deleted the staged archive
under the running job. The drill requires 202 and started, and looks
for the daemon's refusal ("archive is not signed with the ForgeFIRM
release key"). Bench: the apply started, the job ended with that
refusal, PASS.
A queue started 4 s after a forgetest restart ran 7 tests instead of
10. The bench page's /state poll had a fixture probe in flight (an mDNS
answer), probe_fixture stamped its time at its start, and the queue
start read the stale fixture, none, so the three operator tests the
fixture runs in the unattended queue were routed to nobody. The probe
now runs under a lock and is stamped when it completes: a caller that
arrives during a probe waits for its answer. tests/test_fixture.py
holds the race with a slow scripted probe; it fails on the old code.
Catalog consequence: the update implementation hash moves; the runner
change is dev-only.
forgetest - the ForgeFIRM release acceptance tool
The daemon behind http://<machine>:8090/ on the dev image: runs the
acceptance catalog against the machine, keeps the append-only result log,
decides which results still apply to the image that is running, exports
the release artifact scripts/release.sh gates on, and serves the bench
diagnostics page. The contract - catalog, campaigns, fingerprints,
inheritance, the gate, the coverage rule - is
the Acceptance page of the documentation site.
Run the host tests
cd forgetest
python3 -m unittest discover -s tests -v
Run the daemon on a workstation (against a mock or a manifest file)
FORGETEST_DATA=/tmp/ft FORGETEST_MANIFEST=../tree-manifest.json \
FORGECTRL_URL=http://<machine>:8080 python3 -m forgetest --port 8090
scripts/manifest-from-tree.py produces tree-manifest.json from the recipe
pins; the coverage lint is python3 -m forgetest.coverage --manifest ....
Environment
| Variable | Default | Purpose |
|---|---|---|
FORGETEST_DATA |
/data/forgetest |
results.jsonl, bench.jsonl, token, export/ |
FORGETEST_MANIFEST |
/etc/forgefirm-manifest.json |
the image manifest |
FORGETEST_PORT, FORGETEST_HOST |
8090, 0.0.0.0 | listener |
FORGETEST_BENCH_DIR |
/usr/share/forgetest/bench |
the installed bench scripts |
FORGETEST_BENCH_DATA |
<FORGETEST_DATA>/bench |
passed to bench tools: where they keep their data files (with GF_HOST=127.0.0.1 and the panel token in GF_TOKEN) |
FORGETEST_MARKER |
/run/forgetest.active |
takeover marker |
FORGECTRL_URL, FORGECTRL_TOKEN_FILE |
http://127.0.0.1:8080, /data/forgefirm/panel.token |
forgectrl client |
GF_SYSFS_ROOT |
/sys/glowforge/ |
kernel module sysfs |
GRBL_HOST, GRBL_PORT |
127.0.0.1, 23 | Grbl TCP |
Adding a test
Register it in the subsystem module under forgetest/suite/ with
@test(...): id subsystem.name, kind, hardware, mode (the controller
mode the test needs; the runner switches to it first), covers,
requires, always, steps. The body gets a Context (log, check, fail,
prompt, confirm, instruct, sleep, evidence, forgectrl, sysfs,
grbl, takeover). Return normally for PASS, raise runner.Failed for
FAIL. Then run the unit tests and the coverage lint.