mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-10-16 01:29:55 -07:00
08c7df9cbb8f7aca65348dc620a4834ff9f3c755
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ScottW514
|
01f59914e8 |
Catalog: forgectrl.panel-serves, setup.ssh-until-reboot, setup.first-run-flow and logs.routing leave
Each leaves for a host proof that forgectrl's build.yml runs at the pinned 22a2cfc, and that run passed (38728407038): - forgectrl.panel-serves: tests/host_routes.sh serves the panel page bundled, with the theme and the save bar and the policy frame-src 'none', and answers /status, /cam/status and the dose-curve reads. Rows: src/ui.h. - setup.ssh-until-reboot: tests/host_routes.sh, GET and POST /system/ssh with the flag in the run directory and sshd's init a stand-in told start and stop; enable=2 refused. Rows: src/paths.h. - setup.first-run-flow: tests/host_setup.sh, the first run in the page's order: the four documents accepted at their hashes, the press step breathing teal on a scratch LED tree, the press taken as the acceptance, the account and its cookie, the gate's reason after each form step, the run completed. Rows: src/advisories.*, src/button.*, src/led.*, src/hooks.h and the four first-run documents, one row each (docs/advisories/extensions.md stays with exthost.service); src/sheetid.* to tests/sheetid_test.c, the golden ids, because host_setup.sh does not read the sheet id. - logs.routing: tests/fflog_e2e.sh, the image's rsyslog.conf and the rendered rules, each logger in its own file at its level, a stray tag in system/, the kern facility in kernel/. Rows: src/fflog.*. logs.py held logs.routing alone and goes with its import. The five camera tests no longer require forgectrl.panel-serves. test_setup_suite.py keeps setup.account-login and setup.machine-name in its ids; the cases that described setup.first-run-flow (takeover, fixture, covers) go, with the cases of the helpers that left with the two tests (breathes_teal, first_run_record, sshd_policy). test_manifest's behavioral-path case runs the report with no host-tested rows: the real table now names the first-run documents, and the case is about the allowlist. exthost.motion-job stays. It is the only catalog test that grants motion.job and posts /v0/motion/job, the path that reads machine_host_token() on the default token file, and it stays the bench proof of that path until an image carries forgeext's token-file change. src/jobpost.* stays in its covers, with no row. Fingerprints: every test in forgectrl.py, setup.py and camera.py moves with the module text. The next image cannot inherit a campaign. Proof: the coverage lint over the tree manifest from the pins finds 0 uncovered paths and 0 host-tested row problems (49 tests); pyflakes on the touched files is clean; forgetest's unit suite under Linux passes. |
||
|
ScottW514
|
6594f57362 |
Catalog: motion.port-jog and motion.tray leave; two leftovers of the audit
Two catalog tests leave for the host proofs that carry them; the catalog goes from 80 to 78 (78 @test in forgetest/forgetest/suite/). - motion.port-jog. forgectrl's tests/host_routes.sh (7b2f629 on e69d967) runs the controller port's routes against the port's stand-in: a jog refused without the token, a jog that is no number 400 with nothing on the port, a numeric jog on the port as G91 G21, cancel, release and energize each one line, GET /motion/state passing the port's state on with state and sender, GET /motion/sender 200. The port beside a sender, its polls, the cancel, the client first and the bound are grblHAL's ctlport_test.py and forgectrl's grblport_test; motion.release keeps the physical currents. The helpers before it (_port_state, _port_idle, _words) stay: bedsize, laser and motion.job read them. - motion.tray. host_routes.sh holds POST /motion/tray (tray=in on the port, tray=up 400) and the panel page's switch (id="traybtn", trayToggle, /motion/tray); grblHAL's tray_test.py the mode, its offset, its persistence and its frame; forgectrl's tray_test the daemon's side. suite/tray.py held only it: the module and its import go. The coverage lint then named forgectrl src/tray.c and src/tray.h, and nothing else; grblhal-glowforge's src/glowforge_tray.* stays covered by another catalog test, so it gets no row. One HOST_TESTED row takes forgectrl's, to forgectrl:tests/tray_test.c on forgectrl:.github/workflows/build.yml, which the pin c274f19 runs (./build/tray_test). Two of the audit's leftovers, in the same change: - test_manifest.py's ImplementationHashTests sat after the __main__ guard, so running the file directly skipped it. It is above the guard now, byte for byte: python3 -m unittest test_manifest.py and python3 test_manifest.py (from tests/, where its helpers import resolves) each run its 7 cases, and 34 in all; before, the direct run ran none of them. - laser_lifecycle_test.py's test_lid_open_in_wait said an alarm is raised; the docstring now says what the test asserts: the lid-open cancel is reported and no alarm is raised. The assertions are unchanged and the harness was not run. The coverage lint on the tree manifest finds no uncovered path and no host-tested row problem across the 78 tests; its one finding is setup.check-lens's empty src/lensfind.* entry. pyflakes reports nothing on motion.py, manifest.py, test_manifest.py or laser_lifecycle_test.py. The full forgetest suite was not run. Fingerprints, measured against the parent: 17 tests move with motion.py's shared text: the 11 left in motion.py, and homing.manual, laser's four and setup.check-envelope, whose modules read it. The pending pin bumps from forgectrl c274f19 and forgeext 2d0391e already owe all 17; this deletion adds none to the owed list. The two leftovers move no fingerprint. releases/ is unchanged. |
||
|
ScottW514
|
ed1322518b |
forgetest: the lint takes python -m unittest given a test's file or module
forgectrl's CI runs its dev-server mock test as `python3 -B -m unittest -v tests/test_devserver_mock.py`, and the workflow judge counted python -m unittest only as `unittest discover`: a HOST_TESTED row for that test was refused as not run. _unittest_names: an argument to python -m unittest (not discover) runs the test when it is the test's file (resolved against the step's working-directory) or a dotted name whose module is that file (tests.test_x, tests.test_x.Case.test_one); a -k pattern's value and an argument with a shell variable are not names. Proof: host tests: the CI's own command line, a module, a module with a class and a method; not a module beside it, nor a -k pattern naming it. The manifest tests (34) pass, and the forgetest host suite passes under Linux. Against forgectrl's pinned workflow a row for tests/test_devserver_mock.py is accepted; every earlier verdict on the real workflows is unchanged. Acceptance: none. forgetest is dev-only and in no fingerprint; the lint runs in CI, not in a campaign. |
||
|
ScottW514
|
a9c794be34 |
forgetest: a path a host test proves needs no catalog test
The coverage lint failed CI on any behavioral path no catalog test covered, so every software change pulled a new or wider acceptance test in with it, though the catalog is for what needs the machine or an operator. A path whose behavior a host test in CI proves now has a row in HOST_TESTED (manifest.py): the component, a glob, the host test, and the CI workflow that runs it, the last two as "<repository>:<path>" - a file of a component in the manifest (its tests/ and .github/ files are listed there), or "forgefirm:" and a path in this repository (the null-sink harnesses, which grblhal-glowforge's workflow runs). - coverage_report counts a host-tested path as covered. Fingerprints are unchanged: a path no catalog test covers is in no fingerprint, so a change there makes no acceptance run necessary. - host_row_problems reports a row whose glob selects no behavioral file; a host test or workflow that does not exist (a component file missing from the manifest, a "forgefirm:" file missing from the checkout, the dev-only forgetest, no repository named); a workflow that should be readable and is not; one that does not check out the test's repository; and one with no step that runs the test. - The workflow is read at the pinned revision: read_source takes a component's blob from the bare repo manifest-from-tree.py fetched for its source URL (the checkout's .manifest-cache, or --sources), a "forgefirm:" file from the checkout. With no sources (an image) it is not read and the lint says so; with sources, a read that fails is a problem, never a pass. - workflow_steps reads the workflow's steps by indentation (uses, run, working-directory, with), comments dropped, whole lines and from " #"; a quoted scalar loses its quotes, a quote inside a command stays. checkout_path finds where the test's repository is checked out (its own: the checkout naming no repository; another: the checkout of <owner>/<name>, the name from the component's source URL). - workflow_runs looks at what a step's run commands execute, split into simple commands (;, &&, ||, |, newlines, then/do/else; a backslash continuation joined), with sudo and its flags, env, time, exec and variable assignments passed over, each path resolved against the step's working-directory and compared with the test's workspace path: the command itself (a script run directly, a C test's built binary <dir>/<stem>); the script an interpreter runs (python, bash, sh) or a for loop's list, by path or a glob matching the whole path segment by segment; unittest discover starting in the test's own directory with a pattern its name matches (test*.py unless -p); pytest given the test or its own directory, for a name it collects. An argument to anything else (echo, cat, a build target), a step's name, a comment, a trigger's paths filter, or a path that only ends like the test's does not count. - The lint prints the row problems and fails --enforce on them, as on an uncovered path or an empty covers entry; --json carries them as host_tested, with workflows_read (false when a workflow that should have been read was not). It reads HOST_TESTED when it runs. - The table starts empty: rows come with the tests that leave the catalog. Proof: host tests: a host-tested row takes its paths out of the report and moves no fingerprint; good rows report nothing; each kind of bad row is reported with its reason; a workflow that runs one harness and names another only in its paths filter, a comment line and an inline comment runs the first and not the second; workflow_runs on a built binary, sudo and a binary, a script, a harness under another checkout, pytest on a directory, a for-loop glob and unittest discover under a step's working-directory, and not under another one; and not on a paths entry, an inline comment, a glob or a discover that the path only ends like (tools/tests/..., scripts/bench/tests/...), a longer name, another file, a step's name, or a commented line in a block; not on echo with the stem or the binary's path, a build target, cat of a script, unittest discover from a directory above the test or with a pattern its name misses, or pytest on a directory above it; on discover with -p '*_test.py', pytest given the file, a script run directly and by bash, and sudo -E with assignments continued onto the next line; checkout_path for its own and another repository and none; repo_name from a source URL; a component's workflow read from a bare repo's blob, judged, not judged without the sources, and a blob that cannot be read reported; the command line with a workflow it cannot read reports workflows_read false and the read failure, and fails --enforce. The manifest tests (34) pass on Windows and under Linux; the forgetest host suite passes under Linux. Against the recipe pins and their fetched workflows: rows for forgectrl's tray_test, forgeext's holdkeep_test, sandbox_test (under sudo) and run_test.py (sudo, env and assignments over three lines), the laser_stream_test harness (grblhal-glowforge's checkout of forgefirm), gfhardware's test_machine_lid_button (a for-loop glob under its working-directory), gfutilities' test_frame (pytest on tests) and forgetest's own test_manifest (unittest discover under forgefirm/forgetest) pass; rows for forgectrl's fflog_e2e.sh, the bench script fire_test.py and kas/source-bundle.yml (only in forgetest-ci's paths filter) are refused. The lint on that manifest reports 0 uncovered paths and 0 host-tested row problems. Acceptance: none. forgetest is dev-only and in no fingerprint; the lint runs in CI, not in a campaign. |
||
|
ScottW514
|
0eb0a16452 |
forgetest: forgeext's packages/ exemption goes with the packages
OpenGlow's packages left forgeext for repositories of their own, so the
non-behavioral entry ("forgeext", "packages/**") names nothing. The
manifest test checks the shared package workflow's kit action as
non-behavioral in its place (".github/**" covers it). The manifest tests
pass (26); manifest.py is in no fingerprint.
|
||
|
ScottW514
|
dc9bede034 |
forgetest: the first /state parses each suite module once
The first GET /state after forgetest starts hashes every test's implementation, and it took from 81 s to more than 12 minutes on the bench reference. Two causes: - sibling_imports() read and parsed a test's module and every sibling it imports, transitively, once per test: 280 ast.parse calls for 26 suite modules. module_parts() kept its own parse, sibling_imports() kept none. - The page gives up on a poll after 20 s and polls again; the server thread goes on computing. Each new poll started the same cold work beside the first, all of it on the one CPU, so the longer the first took the more copies ran. That is the spread between 81 s and 12 minutes. Each module's text and tree are now read and parsed once and kept, as are its direct sibling imports, and the implementation hash is filled by one thread at a time: a poll that arrives while it is computed waits for it instead of repeating it. catalog.forget(path) drops what is kept about a file, for the unit tests that edit their modules. The hashes do not move: every test's implementation hash and domain fingerprint on the image manifest of 20260923220034 is byte-identical before and after, so no result is invalidated. On the host the cold computation went from 7.00 s to 0.29 s (280 parses to 25). On the bench reference (the file bind-mounted on image 20260923220034, the page open) the first /state answered in 8 to 12 s after a restart, where the same restart earlier in the evening had not answered after 7 minutes. tests/test_responsiveness.py pins both: each suite module parsed at most once while every test's implementation hash is computed, and three threads reading one test's hash compute it once. With the old behavior put back by a patch both fail (280 parses for 26 modules; the hash computed 3 times). forgetest's unit tests 454 OK. forgetest is the dev-only harness, outside the catalog's coverage; its catalog consequence is none, since no fingerprint moves. |
||
|
ScottW514
|
4e47b7fbf1 |
forgetest: exthost.catalog, and the coverage lint no longer allows an advisory away
exthost.catalog (suite/extcat.py, its own module): GET /ext/catalog answers the index the host keeps and the one address it is fetched from. On a scratch root under /tmp, with a throwaway key standing in for the OpenGlow extension key, the machine's own forgeext keeps an index signed with it, and the author key it names for one id makes a package of that id read as community and endorsed, where before it was unverified; the same key on another id counts for nothing. On the machine's own root that index is refused in words, a package handed over as an index is refused by the product gate, and the index kept is left as it was. The relay refuses an id with no such form (400) and one the kept index does not list (404, or 409 with none kept) before anything is fetched, and a refresh from the fixed address keeps OpenGlow's index when one is published there and is 502 in curl's words when none is, the kept index left as it was. Nothing is left in the staging directory. The coverage lint had a gap: coverage_report() let the allowlist's docs/** and **/*.md take out a path the BEHAVIORAL list keeps in every fingerprint, so the four first-run advisory documents were covered by no test and the lint passed. A change to the privacy advisory would have invalidated nothing. A behavioral path is now never allowed away, and setup.advisories-rehash, which accepts every first-run document at its current hash, covers the four. Proof: on the bench reference, with forgectrl 848ccc1 and forgeext a64b933 bind-mounted and the privacy document accepted again at its new hash with the fixture's press, exthost.catalog PASS (the refresh was 502: GitHub answered 404, nothing is published at the address yet), and setup.advisories-rehash PASS with the rest of the campaign. The lint's new unit test reports the uncovered advisory, and the old reading (the override ignored) reports nothing for it, which is the gap. forgetest's unit tests pass (452), and the coverage lint passes with --enforce. |
||
|
ScottW514
|
b7e7c0fb2c |
Manifest: the advisories are behavior, and forgeext's kit and packages are not
forgectrl embeds its advisory documents in the binary, serves them, and records the operator's consent to one by its hash, so an edited advisory is a changed consent. They are Markdown under docs/, which the non-behavioral list takes out of every fingerprint, and that left setup.extensions-consent's covers entry for docs/advisories/extensions.md selecting nothing: the enforced coverage lint fails on it (exit 1 on the dev image's manifest of 20260922225653), and an edited advisory moved no fingerprint at all. A BEHAVIORAL list now keeps forgectrl's docs/advisories/** in, ahead of the non-behavioral one. forgeext's recipe installs the binary and its init script and nothing else, so packages/ (the official packages, which carry their own acceptance artifact), sdk/ (the author's kit), template/ and tools/ are non-behavioral for the image: without that, an edit to the alignment page or the kit would make every exthost test stale on the next image. Proof: test_manifest passes its 25 cases, the new ones among them; the enforced coverage lint on the dev image's manifest of 20260922225653 exits 0 with no empty entry and nothing uncovered, where it exited 1 before. The whole forgetest suite ran its 451 tests; one, test_cloud_suite's test_pause_resume_passes_on_the_machines_lines, errored under the suite's load and passes 16 runs of 16 alone, at HEAD and on this tree alike: it replays a print against timed hooks. |
||
|
ScottW514
|
0eb764bf75 | Added SPDX | ||
|
ScottW514
|
b182a5ab0e |
acceptance: helper imports move the fingerprints, and the update test verifies a foreign signature
A test's fingerprint covered its own text and its module's shared text only, so a judge imported from a sibling suite module (laser.py takes its motion judges from motion.py) could change without moving the fingerprints of the tests that call it. The shared text of every sibling module a module imports now rides along, transitively; unit test. update.slots-and-signature claimed to refuse a tampered signature but fed fwup one garbage file. It now makes a throwaway key pair on the machine, signs a tiny archive, checks that the archive verifies with its own key and fails against the shipped release key, and asks the update job to apply it without confirm_unsigned: the job refuses it for its signature before touching the slot. |
||
|
ScottW514
|
335c6dea9d |
forgetest: non-behavioral paths outside every fingerprint; hollow covers fail the lint
The coverage lint already allowed docs, CI, unit tests and licenses to go uncovered; the same list now keeps them out of every fingerprint, so a README edit in any component re-requires nothing. The list moves to the manifest module as NON_BEHAVIORAL, the one place both uses read it. And a coverage entry that selects no file of its component (a glob without the recipe's subdirectory, a component the manifest lacks, a glob naming docs only) fails the lint: such an entry covers nothing and the test's fingerprint ignores the file it meant. The contract says both. Every test whose maps reached a doc or a test file gets a new fingerprint once. |
||
|
ScottW514
|
2547a8eb68 |
forgetest: a test's implementation hash is its own function and its module's shared code
The implementation half of a domain fingerprint was the whole suite
file, so a two-line witness fix in laser.py re-required every laser
test and a rename in cloud.py every cloud test: sixty attended minutes
for changes that touched two test bodies. Now the hash is the test's
own function (its decorator included) together with the module's text
outside every @test function. A body edit moves that test alone; a
helper edit moves the tests of its module, which is what a helper does;
a file that defines no test in the @test form hashes whole. The gate
computes it the same way, from the same code.
Every recorded fingerprint moves once with this, so the next campaign is
a full one: the price of every later fix costing one test.
Also carries the re-targeted cloud replays that the previous commit
left in the working tree (the CI failure on
|
||
|
ScottW514
|
c0f53a865f |
forgetest: the release acceptance tool and the bench diagnostics page
A stdlib-only daemon on the dev image (HTTP :8090) that runs the acceptance catalog against the machine from a self-contained page, keeps the append-only result log under /data/forgetest, and exports the release artifact the gate reads. Tests declare kind (auto / operator / live), hardware (api / takeover), coverage globs, prerequisites, and core membership; a test's domain fingerprint is the hash of the manifest files its globs select plus the platform and its own implementation, so a PASS stays valid exactly while nothing it covers changed. Campaign rules: a FAIL ends the campaign, the core (image health, kernel latch and drills, one live emission witness) is never inherited, invalidate-all forces a full campaign, no SKIP. Live tests need the operator acknowledgment and the physical arm press through the controller; takeover tests stop forgectrl for the duration with a crash-recoverable marker; the tool never touches the laser latch. Catalog v1: 24 tests ported from the proven bench drills with their recorded pass criteria (image, kernel K1-K3 and fire A/B/U, forgectrl API and logs, motion incl. dead-man, cooling, live laser, camera, update, cloud). The bench tab lists every scripts/bench tool and runs the board-side ones as subprocesses (takeover tools wrapped). 44 host unit tests, including the gate verification fixtures. Installed only by forgefirm-image-dev, with the bench scripts under /usr/share/forgetest/bench. |