mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-10-19 11:01:43 -07:00
8fc411f50fd592a2f5a747c31b09ab25351bfb2d
26
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ScottW514
|
4fa03f7e2b |
forgetest: the lint counts a push or pull request only when no filter is under it
A HOST_TESTED row's workflow has to run on push or pull_request, and
the lint read the events of its `on:` and nothing under them. A
`paths:` or `paths-ignore:` under the event keeps the workflow from
the pushes that change the other files, a `branches:` or
`branches-ignore:` from the pushes to the other branches, and a push
with only a `tags:` or `tags-ignore:` runs on no branch push at all.
So a workflow whose push and pull_request each carried such a filter
counted as running on every change, and a host test behind one kept
its row green and its paths covered.
workflow_filters reads what is under each event of the `on:` block
mapping: each `branches:`, `branches-ignore:`, `tags:`, `tags-ignore:`,
`paths:` or `paths-ignore:` key at the event's own level, by name; a
value on the event's line (`push: {branches: [main]}`); and a key
with one stray quote under the event. Each is a filter the lint does
not read: the simple rule, fail closed. The manifest carries no
branch (a component's record is its srcrev, source, files and
recipes), so a `branches:` naming the default branch is read as any
other filter, and a `paths:` is not matched against the row's covered
paths. host_row_problems counts a push or a pull_request only with no
filter under it, and lists a workflow whose every push and
pull_request carries one as "every push or pull_request in its own
on: carries a filter (push: paths-ignore)", with each event's filters
named. One push or pull_request with no filter is enough: the
workflow then runs on every change, through that event. "Runs" for
a row's workflow now means: on a push or a pull_request with no
filter under it, in a step on every run of the workflow, in a job
whose needs are met on every run, on a line of run text whose
failure ends the step. The HOST_TESTED comment and the docstrings
say so. A key deeper than the event's own level (a dispatch's
inputs) is no filter, and an `on:` list or scalar carries none.
The workflows the rows name, forgectrl's and forgeext's build.yml,
run on `push: branches: [main]` with a plain `pull_request:`, as
grblhal-glowforge's does: the pull request keeps each row counting,
and the push to main is the push that moves a pin.
Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s
tests -v` in forgetest): test_manifest.py runs 41 tests, OK, on
Ubuntu 24.04 with python 3.12, in a private PID namespace. The new
case test_a_push_or_pull_request_under_a_filter_runs_on_not_every_change:
workflow_filters reads a `branches:` on the push with a plain
pull_request beside it; a `branches-ignore:` and a `paths-ignore:` on
the push with a `paths:` on the pull_request; `tags:`; `tags-ignore:`;
a value on the event's line and a stray-quoted key under the event as
filters the lint does not read; a dispatch's `inputs: paths:` as
none; and an `on:` flow list, scalar and block list as none. Through
host_row_problems, a push to main with every pull request passes, a
push with `paths-ignore:` alone is listed, and a push under
`branches:` with a pull_request under `paths:` is listed, each with
its filters named. The CI_YML fixture of the two older cases moves
its `paths:` from the push to the pull_request, so the workflow
still runs on every push and still names the second harness in a
filter alone. Controls: the parent,
|
||
|
ScottW514
|
9c97f05db9 |
forgetest: the lint counts a step as run only when every job its job needs runs
The coverage lint judges a HOST_TESTED row by a step of its workflow
that runs on every run of the workflow. It read the `if:` and the
`continue-on-error:` of a step and of its job, and not the job's
`needs:`. GitHub skips a job whose needed job did not run (an `if:`
that held it back, or a failure), unless the job says `always()`, so
a test step in a job that needs a conditional job runs on not every
push, and counted all the same: a host test behind a conditional
build keeps its row green and its paths covered. No workflow a row
names uses `needs:`.
workflow_steps now reads each job's `needs:` (_needs: a flow list,
one scalar, or a block list), keeps the jobs by name, and resolves
them when the workflow is read (_needs_run): a job's needs are met
on every run when every job it names runs on every run, that is,
carries no `if:` other than the literal true and has its own needs
met, transitively. A job the workflow does not have, a cycle, and a
`needs:` in a form the lint does not read each fail closed. A needed
job's `continue-on-error:` does not bear: its failure fails neither
the run nor the jobs after it. The job's dict carries the result as
"needs_run", and _on_every_run counts no step of a job whose needs
are not met, so workflow_runs and checkout_path refuse it as they
refuse a conditional step. The reason stays "does not run it". The
lint evaluates no condition, so a dependent job under `always()`
counts as no job, as a step under `always()` counts as no step. The
HOST_TESTED comment and the docstrings say what the lint reads.
Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s
tests -v` in forgetest): test_manifest.py runs 40 tests, OK, on
Ubuntu 24.04 with python 3.12, in a private PID namespace. The new
case test_a_job_runs_only_when_every_job_it_needs_runs: a job that
needs an unconditional job runs its test, with the `needs:` as a
scalar, a flow list, a quoted flow list and a block list; the same
four with the needed job under `if: false` or `if: github.event_name
== 'push'` do not; a needed job under `continue-on-error: true`
runs it; a chain of three runs it, and does not with the first job
under `if: false`; a needed job the workflow does not have, a cycle
of two, a self-need and a `needs:` holding a mapping each do not;
`if: always()` on the dependent job, and on the middle job of the
chain, does not. test_the_workflow_has_to_run_the_host_test gains a
workflow whose test job needs a job under `if: false`, listed as
"does not run it". Control: the parent,
|
||
|
ScottW514
|
7a71dd80b5 |
forgetest: the implementation hash walks a sibling imported as from . import x too
_imported walked a relative import only when it names a module (`from .x import y`): a level-1 ImportFrom with no module, `from . import x`, was passed over, so the suite module x joined no hash through it. The guard on the suite's package imports reads levels 2 and 0 alone, so it passed that spelling, and its own message sends an absolute import there: "import it relatively (from .. or from .)". So the guard and the walk did not read the same spellings: on a scratch copy of the catalog, forgectrl.auth's import of setup.py written `from . import setup` passed the guard, and an edit to setup.py's decode(), which the test calls, left forgectrl.auth's implementation hash where it was. homeoff.py imports cloud.py so (`from . import cloud as logs`); its `from .cloud import log_lines_since, log_size` on the next line keeps cloud.py in the walk, so no hash misses a module. _imported now walks every level-1 relative import: one with no module walks each name it imports as <suite>/<name>.py, the name and not its alias (a name that is no module of the suite is passed over, as a module was). The docstrings of _imported and sibling_imports, and the comment of the guard's case, name both spellings. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12. test_a_sibling_imported_inside_a_function_moves_the_tests_that_run_it gains a leg in which m.two imports judge.py as `from . import judge as j`: an edit to judge.py moves m.two and not m.one. Negative controls on scratch copies: catalog.py as it was fails that leg (the two hashes are equal), and so does the walk that reads the alias (j) in place of the name. On a scratch copy of the catalog with forgectrl.auth's import written `from . import setup`, an edit to setup.py's decode() now moves forgectrl.auth's hash. The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: none. The source_sha of 43 of 43 tests is unchanged: the one module-less sibling import outside the registry (suite/__init__.py, which has no tests) is homeoff.py's, and cloud.py is in its walk already. catalog.py is the registry and no shared judge, forgetest is in no fingerprint, and tests/ are in no fingerprint. |
||
|
ScottW514
|
614efb23e4 |
forgetest: the lint counts a host test only on a line of run text whose failure ends the step
workflow_runs counted any simple command in a step's run text as
running the test: _SPLIT_RX split the text on &&, ||, ;, |, newlines,
then, do and else, and every part was matched. So a step that hid the
test's failure, or never ran it, kept a HOST_TESTED row green, against
the HOST_TESTED comment's "a step ... whose failure fails it":
`./build/tray_test || true`, `| tee log`, `&`, `true ||
./build/tray_test`, an `exit 0` or a `set +e` line before the test,
`if false; then ./build/tray_test; fi`, a here-document's lines, a
folded block (`run: >`) whose next line is `|| true`, and a step
under `shell: bash {0}` or a `defaults: run: shell:` of pwsh each
counted. The workflow_runs docstring said "Only what a step's run
commands execute counts", which a command the shell skips, or whose
status it drops, does not meet. No workflow of the tree uses these
forms at a row's test.
workflow_runs now reads the run text line by line as bash -e or sh -e
runs it (_run_commands), a backslash at the end of a line joining the
next. _shell_words reads a line's words and operators, with quotes and
expansions ($(...), ${...}, `...`) inside the words. A command counts
on a line that holds it alone, with its redirections, or in a pipeline
under pipefail (shell: bash, or `set -o pipefail` on an earlier line),
and only while errexit holds: a `set` of errexit or pipefail is
followed. A pipeline with no pipefail does not count. The lint stops at
a line with any other operator (`||`, `&&`, `;`, `&`, a subshell, a
here-document), at a command that opens a compound command or negates a
status (if, case, while, until, a group, a function, `!`), at a command
after which it cannot tell what a later line runs (exit, exec, trap,
eval, source, alias, shopt, hash and the like), and at a `set` of an
option it does not follow (noexec): nothing on that line or after it
counts. A for loop counts in one form: `for <name> in <words>; do` on
its own line, body lines that count (a pipeline under pipefail only),
and `done` on its own line. A step counts only when its own shell:,
and the `defaults: run: shell:` of its job and of the workflow, are
each bash, sh or absent: workflow_steps now reads those defaults, and
reads a folded block, or a scalar continued on the lines under its key,
as one line, as YAML folds it. A prefix is taken off only with the
flags with which it runs its command (`sudo -E`, `env -i`), so `command
-v ./build/tray_test`, which prints the path, does not count; exec is a
stop, no longer a prefix. The HOST_TESTED comment and the docstrings of
workflow_steps and workflow_runs say what the lint reads.
The lint does not read, and counts as no step: `./build/x; true`
(bash -e stops at the test, and the lint refuses the `;` all the
same), `exec ./build/x`, a loop on one line or with `do` on a line of
its own, a pipeline with the test last and no pipefail, a test under
`set +e` whose status a later line checks, a test after a guard line
with `||` or `&&`, and a `set` of any other option (`set -E`). It
counts without reading: what a loop's body does with the loop's name
(its list counts), a `cd` before the test, a `defaults: run:
working-directory:`, what PATH or BASH_ENV change, and the runner's
operating system (it does not read runs-on; a Windows runner's default
shell is pwsh).
Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s
tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows
with python 3.14 and on Ubuntu 24.04 with python 3.12.
test_a_step_counts_only_when_every_run_of_the_workflow_runs_it_and_its_failure_fails_the_run
gains these legs, none of which runs the test: `|| true`, `true ||`,
`; true`, `&` and a pipe with no pipefail on the test's line; an `exit
0`, `set +e`, `set -n` or here-document line before it; a `{` group
that `} || true` ends; a loop whose body is `python $t || true`, or a
pipe with no pipefail; a folded block and a continued scalar with `||
true` under the test; a step under `shell: bash {0}`; a job's and a
workflow's `defaults: run: shell: pwsh`. A pipe under `set -o
pipefail` and under shell: bash, and a step under shell: sh, run it.
test_what_running_a_host_test_looks_like gains `command -v
./build/tray_test`, which does not run it; its loop case is the loop as
python3-gfhardware's CI writes it (an assignment from `$(basename "$t"
.py)` in the body), and its glob case is a loop on lines of its own.
Negative controls on scratch copies: manifest.py as it was fails the
first new leg (True is not false), and each new leg reads as running
the test there; 19 mutations of the new reading each fail a case: exit
no stop, errexit not followed, an unknown `set` option followed, a
here-document read as a redirection, no reserved words, pipefail always
on, shell: bash with no pipefail, `set -o pipefail` not followed, the
loop's body not read, a body pipeline with no pipefail read, a folded
block read line by line, a continued scalar's lines dropped, the job's
defaults not read, the workflow's defaults not read, any shell read,
sh not read, `$(` read as an operator, every flag of a prefix taken
off, and `sudo -E` kept. Under bash -e with a failing stand-in test,
the step exits 0 for `|| true`, `true ||`, `&`, a pipe with no
pipefail, an earlier `exit 0`, `set +e` or `set -n`, the `{ } || true`
group, both loop bodies and the folded block; `; true` exits 1. A
here-document's lines are its command's input (`cat <<'EOF'` exits 0),
and `bash {0}` runs with no errexit, so a line after the test decides
the step (exit 0). The 17 workflows of the tree's repositories (211
steps) read to the same events and steps; of their 161 run steps, 20
count fewer commands and 4 the same commands with a redirection taken
off or an expansion kept as written, none of them a step that runs a
row's test. Each
of the 46 HOST_TESTED rows runs its test under the new reading, in the
workflows at the recipe pins and at the heads of forgectrl and
forgeext. The coverage lint on the tree manifest from the recipe pins
reports 0 uncovered paths, 0 host-tested row problems, 43 tests.
pyflakes reports no undefined name.
Acceptance: none. forgetest is dev-only and in no fingerprint (the
source_sha of 43 of 43 tests is unchanged); the lint runs in CI, not in
a campaign, and tests/ are in no fingerprint.
|
||
|
ScottW514
|
3aecf800a5 |
forgetest: the lint's stray-quote rule reads a step's first key, on its dash line, too
The rule that a key with one stray quote makes every step of the
workflow count as no step never saw a step's first key, the one on its
dash line. _STEP_RX reads no key with a stray quote (`- run": make`,
`- name": Build`), and _stray_key cannot match a line that starts with
`- `. So such a step dropped out with no "unread" set, and the
workflow's other steps still counted. YAML reads the key with its quote
(PyYAML gives {'name"': 'Build', 'run': 'make'}), or refuses the file
(`- "run: make`), and GitHub refuses a workflow with a step key outside
the step schema: a HOST_TESTED row passed the lint for a workflow that
runs nothing. The HOST_TESTED comment, the workflow_steps docstring and
the Test page state the rule for a step's key, and every step's first
key is on its dash line. No workflow of the tree has such a key.
workflow_steps now reads the first key of every list item as it reads a
step's other keys: the dash and the spaces after it become spaces, and
a key with one stray quote there makes every step of the workflow carry
"unread", which _on_every_run refuses. The rule also takes in a list
item outside the steps (a matrix include entry): that only lists more.
The docstring says so.
Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s
tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows
with python 3.14 and on Ubuntu 24.04 with python 3.12.
test_a_step_counts_only_when_every_run_of_the_workflow_runs_it_and_its_failure_fails_the_run
gains a step `- run": make` before a step that runs the test: no step
counts. Negative control on a scratch copy: manifest.py as it was fails
that case (True is not false). A probe of workflow_runs: `- name":
Build` with `run: make` under it, and `- "run: make`, each before a
step that runs the test, count as no step, as a stray quote on a
step's second key already did. The 17 workflows of the tree's
repositories (211 steps) read to the same events and steps before and
after the change. The coverage lint on the tree manifest from the
recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43
tests. pyflakes reports no undefined name.
Acceptance: none. forgetest is dev-only and in no fingerprint (the
source_sha of 43 of 43 tests is unchanged); the lint runs in CI, not in
a campaign, and tests/ are in no fingerprint.
|
||
|
ScottW514
|
fc6b7797af |
forgetest: the shared judges' comments say why puls.py is one
The comment over SHARED_JUDGES in catalog.py said that every test runs through all four judges without importing them, puls.py among them, and the comment of test_a_shared_judge_edit_moves_every_test said the same. No test runs puls.py but the three that print the dark job (cloud.lid-interlock-abort, cloud.oversize-stream and exthost.armed-freeze, through offline_job), and suite/cloud.py imports it with `from .. import puls`: it is a judge because the sibling walk does not follow that import, as the Acceptance page of the site says. Both comments now say that every test runs through runner.py, hw.py and baseline.py with no import to follow, and that puls.py comes in by a package import the walk does not follow. Comments only: SHARED_JUDGES does not change. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12. The only import of puls in the package is suite/cloud.py's (grep). pyflakes reports no undefined name. Acceptance: none. catalog.py is the registry and in no fingerprint, and tests/ are in none (source_sha of 43 of 43 unchanged). |
||
|
ScottW514
|
a10fd7782d |
forgetest: the hash cases hold what a test's own import imports in turn, and the guard refuses an absolute package import
No case held the step of implementation_sha that adds what a test's own import imports in turn (`sibling_imports(sib, sibs)`), nor the recursion in sibling_imports itself: the judge.py of test_a_sibling_imported_inside_a_function_moves_the_tests_that_run_it imports nothing. Either line deleted kept test_manifest.py green, while on the real catalog the first drops motion.py's shared text from cloud.mode-switch's hash and setup.py's from motion.release's. The guard on the suite's package imports read the relative spelling alone: a `from forgetest import server`, an `import forgetest.server`, or a `from forgetest.suite.setup import ...` in a suite module passed it, and the sibling walk does not follow any of them, so the module they name joined no hash. The case's judge.py now imports limit.py, which imports base.py: an edit to either moves m.two and not m.one. The guard refuses any import of the package by its absolute name in a suite module, at any depth. No suite module imports the package that way. catalog.py does not change. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12. Negative controls on scratch copies: catalog.py without `sibling_imports(sib, sibs)` in implementation_sha fails the case at the limit.py edit; the walk without its recursion fails it at the base.py edit; in suite/forgectrl.py, a `from forgetest import server` and an `import forgetest.server` inside a function, and forgectrl.auth's import of setup.py spelled `from forgetest.suite.setup import request, decode`, each fail the guard. pyflakes reports no undefined name. Acceptance: none. tests/ are in no fingerprint (source_sha of 43 of 43 unchanged). |
||
|
ScottW514
|
2af9b0277f |
forgetest: the lint's docstrings and its row reason say what it reads, not what a workflow runs on
host_row_problems' docstring gave the reason a row is listed as "a
dispatch, a call or a schedule runs on no change". A workflow on
workflow_call alone runs on every run of the workflow that calls it
(forgeext's package.yml is one); the lint lists it because it does not
follow a call to its caller. The reason it printed, "<ci> runs on no
push or pull request", was false the same way for such a workflow, and
for a flow mapping (`on: {push: ...}`) that names push. The docstrings
of _on_every_run and workflow_runs said that a step under an `if:`
other than true does not run on every run, where the lint evaluates no
condition: a step under `always()` or `success()` runs on every run and
still counts as no step. The Test page of the site says each of these
as the lint applies it.
The docstrings now say what the lint reads: a row is listed when the
workflow's own `on:` names neither push nor pull_request as the lint
reads it (a dispatch, a call or a schedule alone, a call not followed
to its caller, a flow mapping), and a step with a condition counts as
no step because the lint evaluates none. The row's reason reads "<ci>:
the lint reads no push or pull_request in its own on:". The comments
of the two cases that hold these rules say the same. No rule changes:
the same rows are listed, for the same workflows.
Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s
tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows
with python 3.14 and on Ubuntu 24.04 with python 3.12, with
test_the_workflow_has_to_run_the_host_test asserting the new reason. A
probe of host_row_problems: a workflow on workflow_call alone and one
whose flow-mapping `on:` names push are each listed with the new
reason, and a plain `on: [push]` one is not. The coverage lint on the
tree manifest from the recipe pins reports 0 uncovered paths, 0
host-tested row problems, 43 tests. pyflakes reports no undefined name.
Acceptance: none. forgetest is dev-only and in no fingerprint (the
source_sha of 43 of 43 tests is unchanged); the lint runs in CI, not in
a campaign, and tests/ are in no fingerprint.
|
||
|
ScottW514
|
c717ecbcce |
forgetest: the lint reads a quoted key only in a matched pair, and a key with one stray quote runs nothing
_KEY_RX took an opening quote and a closing quote each on its own, so a key with one stray quote read as the bare key: `push":` and `pull_request':` counted as push and pull_request, where YAML reads the keys `push"` and `pull_request'`, which are no events; a step's `run": ...` counted as its run command, and `uses": actions/checkout@v4` as a checkout. A HOST_TESTED row whose workflow was spelled so passed the lint, while that workflow runs on no push and runs no such step. _KEY_RX now reads a quote only as a matched pair. A key with one stray quote is no key the lint reads, and where the lint reads keys it counts as running nothing: an event key spelled so makes the `on:` name no event, and one at the top level, at a job's name or key, or at a step's key below its dash line makes every step of the workflow count as no step (each carries "unread", which _on_every_run refuses). So a step's `if": false` still keeps the step from counting, as it did under the loose reading, and under a `jobs":` top level, where a job's `if: false` goes unread, no step counts either. The HOST_TESTED comment says so. The lint only lists more: a workflow with no such key reads as before. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12. test_the_events_a_workflow_runs_on gains an `on:` with `push":`, and one with `push:` and `'pull_request:`, each naming no event, and test_a_step_counts_only_when_every_run_of_the_workflow_runs_it_and_its_failure_fails_the_run gains a step with `if": false`, a step with `run": ./build/tray_test`, a job with `if": false` and a `jobs":` top level, none of them running the test. Negative controls on scratch copies: _KEY_RX as it was fails the `run":` step and the `push":` event; the matched pairs with no stray-quote rule fail the `if": false` step and the second `on:`; the rule at a step's keys alone fails the job's `if": false`. The 17 workflows of the tree's repositories (211 steps) read to the same events and steps before and after the change. The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: none. forgetest is dev-only and in no fingerprint (the source_sha of 43 of 43 tests is unchanged); the lint runs in CI, not in a campaign, and tests/ are in no fingerprint. |
||
|
ScottW514
|
8d16b93e86 |
forgetest: the shared judges and a test's own imports reach the hash the gate reads
The sibling walk read the top-level statements of a module alone. Five
tests import their judging helpers inside their own functions, so that
no other test of the module moves with them: cloud.mode-switch
(homeoff.py: judge_whole_motions, camera_home_return),
cooling.flow-under-load and cooling.fail-tier-stop (laser.py's sample
and run_and_sample, motion.py's log readers), forgectrl.auth (setup.py's
request and decode) and motion.release (setup_dark.py's run_check).
None of those helpers was in the hash of its test, and homeoff.py was
in no hash at all: an edit to judge_whole_motions moved 0 of 43 tests,
and an edit to laser.sample moved the three laser.* tests and neither
cooling test, so an inherited PASS survived a change to its own judge.
The guard that every package module a suite module imports is a listed
judge read the top level alone too, and no unit test held the
production path: Test.source_sha, which the fingerprint, the campaign
and the gate read, could drop the shared judges with CI green.
implementation_sha now adds to the hash of a test the suite modules its
own @test function imports, at any depth, and what they import in turn;
no other test of the module takes them. The shared text of a module
gives the modules it imports at any depth too: a helper that imports
inside its body brings that module into every test of the module, as
the helper itself does. _test_defs finds the @test functions of a
module for module_parts and for both walks. The guard walks the whole
tree of each suite module, so a package import inside a function or a
block is held to the list as well. test_a_shared_judge_edit_moves_every_test
holds the production path on a suite test: its source_sha equals
implementation_sha with SHARED_JUDGES and differs from it with none.
Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s
tests -v` in forgetest): test_manifest.py runs 39 tests, OK, on Windows
with python 3.14 and on Ubuntu 24.04 with python 3.12, with
test_a_sibling_imported_inside_a_function_moves_the_tests_that_run_it
(a judge imported in the body of one test moves that test and not the
other; imported in the body of a helper, it moves both), the
source_sha assertions, and the guard on the whole tree; test_artifact.py,
test_campaign.py, test_redact.py and test_responsiveness.py pass.
Negative controls on scratch copies: implementation_sha's default
taking no judges fails the source_sha assertion, and so does
Test.source_sha passing judges=(); the new case fails against
catalog.py as it was, and against a shared-text walk of top-level
statements alone; a `from .. import server` inside a function of
suite/forgectrl.py, or in a try block at its top level, fails the
guard, and the function-local one passes the guard that reads the top
level alone. On the real catalog: an edit to judge_whole_motions moves
cloud.mode-switch alone (0 tests before); an edit to laser.sample moves
the three laser.* tests and both cooling tests (the three alone
before); an edit to run_check moves the six setup.* tests and
motion.release (the six alone before); an edit to setup.request moves
forgectrl.auth and motion.release (motion.release through setup_dark.py,
which imports setup.py) beside the 8 it moved before; an edit to
hw.sysfs_int
moves 43 of 43, as before. The coverage lint on the tree manifest
from the recipe pins reports 0 uncovered paths, 0 host-tested row
problems, 43 tests. pyflakes reports no undefined name.
Acceptance: cloud.mode-switch, cooling.flow-under-load,
cooling.fail-tier-stop, forgectrl.auth and motion.release: the
implementation hash of each moves once, its own imports joining it; the
other 38 do not move (source_sha compared before and after). Against
the campaign this series already needs, that adds no run:
|
||
|
ScottW514
|
5bcc536853 |
forgetest: the lint's step parser keeps a step's real key column and reads a quoted key
workflow_steps rebuilt the first line of a step with its first key two columns past the dash, whatever followed the dash. A step written with more than one space after its dash (`- run: ./build/tray_test`, its `if: false` at the column of the run key) then had its first key at one column and the others at another: every key after the first was read as a child of the first and dropped, `if:`, `continue-on-error:`, `working-directory:` and `with:` among them. So such a step under `if: false` counted as running its host test, and such a checkout under a condition counted as a checkout. _KEY_RX did not read a quoted key, so a step's `"if": false` was dropped the same way. Both spellings are valid YAML that GitHub skips or tolerates. The parser now replaces the dash with a space and realigns nothing, so the key column of a step is the column its keys use. _KEY_RX reads a key in single or double quotes as the same key. The HOST_TESTED comment states the rule the lint applies: no `if:` other than `true` and no `continue-on-error:` other than `false` on the step or its job. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 38 tests, OK, on Windows with python 3.14 and on Ubuntu 24.04 with python 3.12, with test_a_step_counts_only_when_every_run_of_the_workflow_runs_it_and_its_failure_fails_the_run extended by a step with three spaces after its dash under `if: false` and a step under a quoted `"if": false`, neither running the test. Negative controls on scratch copies: the first line rebuilt as before fails the three-space case; _KEY_RX as before fails the quoted case. The 17 workflows of the tree's repositories (205 steps) parse to the same steps and events before and after the change. The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: none. forgetest is dev-only and in no fingerprint (the source_sha of 43 of 43 tests is unchanged); the lint runs in CI, not in a campaign, and tests/ are in no fingerprint. |
||
|
ScottW514
|
850498fb00 |
forgetest: the shared judges join every test's implementation hash
Every catalog test judges the machine through package modules it never imports: the Context a test is handed and the takeover (runner.py), the sysfs, LED, switch and HTTP readers behind it (hw.py), and the fresh-boot idle state judged around every run (baseline.py). The cloud and extension tests print the dark job of puls.py, which the suite imports at level 2, and the sibling walk follows level-1 imports only. So none of the four was in any fingerprint: a weakening of sysfs_int, button_lit, Context.check, wait_for or position_quantized moved no hash, and an inherited PASS of kernel.latch-locked-idle, laser.armed-kill or any takeover test survived it unseen by the gate. catalog.py names them in SHARED_JUDGES, with the criterion in the comment above it: a module every test runs through without importing it, or the dark job the cloud tests print. The rest of the package (the server, the page, the campaign rules, the artifact, the manifest, the registry, the bench actuator) judges nothing about the machine and stays out. shared_judge_paths() resolves the four next to the module, and implementation_sha folds their text after the module's shared text and its imported siblings. A listed file that is missing raises, never a hash computed without it. _direct_siblings stays level-1 only, and the tool version does not move: the gate judges by source_sha and the fingerprint. Every catalog test's implementation hash moves once with this change, so every recorded PASS is stale and the next campaign is a full one, 43 tests. The catalog hash does not move, so an open campaign stays open with every test required. From here on, an edit to runner.py, hw.py, baseline.py or puls.py moves every test. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 38 tests, OK, with test_a_shared_judge_edit_moves_every_test (an edit to a judge moves both tests of a module, a package file outside the list moves neither) and test_the_shared_judges_exist_and_take_every_package_import_of_the_suite (the four files exist next to catalog.py, and every level-2 import of a suite module is a listed judge or the registry). Negative controls on scratch copies: `judged = []` in implementation_sha fails the first case alone; "runner.py" dropped from SHARED_JUDGES fails the guard alone, naming homeoff.py's import of runner. On the real catalog the source_sha of 43 of 43 tests differs from HEAD's; with hw.sysfs_int weakened to return the latch bit on an unparsable read, 0 of 43 move at HEAD and 43 of 43 move with this change. The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: every catalog test. The implementation hash of all 43 moves once, so the next campaign runs the whole catalog. |
||
|
ScottW514
|
4d8f0d2e67 |
forgetest: the lint counts a host test as run only by a step every run of the workflow runs, on push or pull request
A HOST_TESTED row names a workflow and a host test, and the coverage lint judges the row by what the workflow's steps run. The parser stored a step's `if:` and `continue-on-error:` and read neither, read no job key, and read no `on:`. So a step under `if: false`, a step whose failure the run tolerates, a job under an `if:`, and a workflow on `workflow_dispatch` alone all counted as running the test, and a checkout under a condition counted as a checkout: a disabled or tolerated host test keeps its row green and its paths covered. The lint now reads the workflow's events and the conditions of a step and of its job. `workflow_triggers` returns the events of the `on:` mapping, list or scalar. What is under an event (its branches, its paths, a dispatch's inputs) is not an event, and a flow mapping names none. A row's workflow must run on push or pull_request, or the lint reports that it runs on no push or pull request: a dispatch or a schedule alone runs on no change, and a call alone is listed as well, since the lint does not follow it to its caller. `workflow_steps` tracks the job each step is in and carries the job's `if:` and `continue-on-error:` on the step as `job`. `_on_every_run` counts a step only when neither it nor its job carries an `if:` other than the literal `true` or a `continue-on-error:` other than the literal `false`. `workflow_runs` and `checkout_path` use it, so a conditional step runs nothing and a conditional checkout is no checkout. The reason for such a step stays "does not run it". `needs:` is not read, nor are the `paths:` and `branches:` filters of an event: the lint stays a floor there. Proof: forgetest-ci "Unit tests" (`python -B -m unittest discover -s tests -v` in forgetest): test_manifest.py runs 36 tests, OK, with the two new cases, test_a_step_counts_only_when_every_run_of_the_workflow_runs_it_and_its_failure_fails_the_run and test_the_events_a_workflow_runs_on, and with test_the_workflow_has_to_run_the_host_test extended by a dispatch-only workflow and a tolerated step. Negative controls on scratch copies: `_on_every_run` forced to return True fails the step case and the host-test case (failures=2); the trigger check deleted from host_row_problems fails the host-test case (failures=1). The coverage lint on the tree manifest from the recipe pins reports 0 uncovered paths, 0 host-tested row problems, 43 tests. pyflakes reports no undefined name. Acceptance: none. forgetest is dev-only and in no fingerprint; the lint runs in CI, not in a campaign. |
||
|
ScottW514
|
01f59914e8 |
Catalog: forgectrl.panel-serves, setup.ssh-until-reboot, setup.first-run-flow and logs.routing leave
Each leaves for a host proof that forgectrl's build.yml runs at the pinned 22a2cfc, and that run passed (38728407038): - forgectrl.panel-serves: tests/host_routes.sh serves the panel page bundled, with the theme and the save bar and the policy frame-src 'none', and answers /status, /cam/status and the dose-curve reads. Rows: src/ui.h. - setup.ssh-until-reboot: tests/host_routes.sh, GET and POST /system/ssh with the flag in the run directory and sshd's init a stand-in told start and stop; enable=2 refused. Rows: src/paths.h. - setup.first-run-flow: tests/host_setup.sh, the first run in the page's order: the four documents accepted at their hashes, the press step breathing teal on a scratch LED tree, the press taken as the acceptance, the account and its cookie, the gate's reason after each form step, the run completed. Rows: src/advisories.*, src/button.*, src/led.*, src/hooks.h and the four first-run documents, one row each (docs/advisories/extensions.md stays with exthost.service); src/sheetid.* to tests/sheetid_test.c, the golden ids, because host_setup.sh does not read the sheet id. - logs.routing: tests/fflog_e2e.sh, the image's rsyslog.conf and the rendered rules, each logger in its own file at its level, a stray tag in system/, the kern facility in kernel/. Rows: src/fflog.*. logs.py held logs.routing alone and goes with its import. The five camera tests no longer require forgectrl.panel-serves. test_setup_suite.py keeps setup.account-login and setup.machine-name in its ids; the cases that described setup.first-run-flow (takeover, fixture, covers) go, with the cases of the helpers that left with the two tests (breathes_teal, first_run_record, sshd_policy). test_manifest's behavioral-path case runs the report with no host-tested rows: the real table now names the first-run documents, and the case is about the allowlist. exthost.motion-job stays. It is the only catalog test that grants motion.job and posts /v0/motion/job, the path that reads machine_host_token() on the default token file, and it stays the bench proof of that path until an image carries forgeext's token-file change. src/jobpost.* stays in its covers, with no row. Fingerprints: every test in forgectrl.py, setup.py and camera.py moves with the module text. The next image cannot inherit a campaign. Proof: the coverage lint over the tree manifest from the pins finds 0 uncovered paths and 0 host-tested row problems (49 tests); pyflakes on the touched files is clean; forgetest's unit suite under Linux passes. |
||
|
ScottW514
|
6594f57362 |
Catalog: motion.port-jog and motion.tray leave; two leftovers of the audit
Two catalog tests leave for the host proofs that carry them; the catalog goes from 80 to 78 (78 @test in forgetest/forgetest/suite/). - motion.port-jog. forgectrl's tests/host_routes.sh (7b2f629 on e69d967) runs the controller port's routes against the port's stand-in: a jog refused without the token, a jog that is no number 400 with nothing on the port, a numeric jog on the port as G91 G21, cancel, release and energize each one line, GET /motion/state passing the port's state on with state and sender, GET /motion/sender 200. The port beside a sender, its polls, the cancel, the client first and the bound are grblHAL's ctlport_test.py and forgectrl's grblport_test; motion.release keeps the physical currents. The helpers before it (_port_state, _port_idle, _words) stay: bedsize, laser and motion.job read them. - motion.tray. host_routes.sh holds POST /motion/tray (tray=in on the port, tray=up 400) and the panel page's switch (id="traybtn", trayToggle, /motion/tray); grblHAL's tray_test.py the mode, its offset, its persistence and its frame; forgectrl's tray_test the daemon's side. suite/tray.py held only it: the module and its import go. The coverage lint then named forgectrl src/tray.c and src/tray.h, and nothing else; grblhal-glowforge's src/glowforge_tray.* stays covered by another catalog test, so it gets no row. One HOST_TESTED row takes forgectrl's, to forgectrl:tests/tray_test.c on forgectrl:.github/workflows/build.yml, which the pin c274f19 runs (./build/tray_test). Two of the audit's leftovers, in the same change: - test_manifest.py's ImplementationHashTests sat after the __main__ guard, so running the file directly skipped it. It is above the guard now, byte for byte: python3 -m unittest test_manifest.py and python3 test_manifest.py (from tests/, where its helpers import resolves) each run its 7 cases, and 34 in all; before, the direct run ran none of them. - laser_lifecycle_test.py's test_lid_open_in_wait said an alarm is raised; the docstring now says what the test asserts: the lid-open cancel is reported and no alarm is raised. The assertions are unchanged and the harness was not run. The coverage lint on the tree manifest finds no uncovered path and no host-tested row problem across the 78 tests; its one finding is setup.check-lens's empty src/lensfind.* entry. pyflakes reports nothing on motion.py, manifest.py, test_manifest.py or laser_lifecycle_test.py. The full forgetest suite was not run. Fingerprints, measured against the parent: 17 tests move with motion.py's shared text: the 11 left in motion.py, and homing.manual, laser's four and setup.check-envelope, whose modules read it. The pending pin bumps from forgectrl c274f19 and forgeext 2d0391e already owe all 17; this deletion adds none to the owed list. The two leftovers move no fingerprint. releases/ is unchanged. |
||
|
ScottW514
|
ed1322518b |
forgetest: the lint takes python -m unittest given a test's file or module
forgectrl's CI runs its dev-server mock test as `python3 -B -m unittest -v tests/test_devserver_mock.py`, and the workflow judge counted python -m unittest only as `unittest discover`: a HOST_TESTED row for that test was refused as not run. _unittest_names: an argument to python -m unittest (not discover) runs the test when it is the test's file (resolved against the step's working-directory) or a dotted name whose module is that file (tests.test_x, tests.test_x.Case.test_one); a -k pattern's value and an argument with a shell variable are not names. Proof: host tests: the CI's own command line, a module, a module with a class and a method; not a module beside it, nor a -k pattern naming it. The manifest tests (34) pass, and the forgetest host suite passes under Linux. Against forgectrl's pinned workflow a row for tests/test_devserver_mock.py is accepted; every earlier verdict on the real workflows is unchanged. Acceptance: none. forgetest is dev-only and in no fingerprint; the lint runs in CI, not in a campaign. |
||
|
ScottW514
|
a9c794be34 |
forgetest: a path a host test proves needs no catalog test
The coverage lint failed CI on any behavioral path no catalog test covered, so every software change pulled a new or wider acceptance test in with it, though the catalog is for what needs the machine or an operator. A path whose behavior a host test in CI proves now has a row in HOST_TESTED (manifest.py): the component, a glob, the host test, and the CI workflow that runs it, the last two as "<repository>:<path>" - a file of a component in the manifest (its tests/ and .github/ files are listed there), or "forgefirm:" and a path in this repository (the null-sink harnesses, which grblhal-glowforge's workflow runs). - coverage_report counts a host-tested path as covered. Fingerprints are unchanged: a path no catalog test covers is in no fingerprint, so a change there makes no acceptance run necessary. - host_row_problems reports a row whose glob selects no behavioral file; a host test or workflow that does not exist (a component file missing from the manifest, a "forgefirm:" file missing from the checkout, the dev-only forgetest, no repository named); a workflow that should be readable and is not; one that does not check out the test's repository; and one with no step that runs the test. - The workflow is read at the pinned revision: read_source takes a component's blob from the bare repo manifest-from-tree.py fetched for its source URL (the checkout's .manifest-cache, or --sources), a "forgefirm:" file from the checkout. With no sources (an image) it is not read and the lint says so; with sources, a read that fails is a problem, never a pass. - workflow_steps reads the workflow's steps by indentation (uses, run, working-directory, with), comments dropped, whole lines and from " #"; a quoted scalar loses its quotes, a quote inside a command stays. checkout_path finds where the test's repository is checked out (its own: the checkout naming no repository; another: the checkout of <owner>/<name>, the name from the component's source URL). - workflow_runs looks at what a step's run commands execute, split into simple commands (;, &&, ||, |, newlines, then/do/else; a backslash continuation joined), with sudo and its flags, env, time, exec and variable assignments passed over, each path resolved against the step's working-directory and compared with the test's workspace path: the command itself (a script run directly, a C test's built binary <dir>/<stem>); the script an interpreter runs (python, bash, sh) or a for loop's list, by path or a glob matching the whole path segment by segment; unittest discover starting in the test's own directory with a pattern its name matches (test*.py unless -p); pytest given the test or its own directory, for a name it collects. An argument to anything else (echo, cat, a build target), a step's name, a comment, a trigger's paths filter, or a path that only ends like the test's does not count. - The lint prints the row problems and fails --enforce on them, as on an uncovered path or an empty covers entry; --json carries them as host_tested, with workflows_read (false when a workflow that should have been read was not). It reads HOST_TESTED when it runs. - The table starts empty: rows come with the tests that leave the catalog. Proof: host tests: a host-tested row takes its paths out of the report and moves no fingerprint; good rows report nothing; each kind of bad row is reported with its reason; a workflow that runs one harness and names another only in its paths filter, a comment line and an inline comment runs the first and not the second; workflow_runs on a built binary, sudo and a binary, a script, a harness under another checkout, pytest on a directory, a for-loop glob and unittest discover under a step's working-directory, and not under another one; and not on a paths entry, an inline comment, a glob or a discover that the path only ends like (tools/tests/..., scripts/bench/tests/...), a longer name, another file, a step's name, or a commented line in a block; not on echo with the stem or the binary's path, a build target, cat of a script, unittest discover from a directory above the test or with a pattern its name misses, or pytest on a directory above it; on discover with -p '*_test.py', pytest given the file, a script run directly and by bash, and sudo -E with assignments continued onto the next line; checkout_path for its own and another repository and none; repo_name from a source URL; a component's workflow read from a bare repo's blob, judged, not judged without the sources, and a blob that cannot be read reported; the command line with a workflow it cannot read reports workflows_read false and the read failure, and fails --enforce. The manifest tests (34) pass on Windows and under Linux; the forgetest host suite passes under Linux. Against the recipe pins and their fetched workflows: rows for forgectrl's tray_test, forgeext's holdkeep_test, sandbox_test (under sudo) and run_test.py (sudo, env and assignments over three lines), the laser_stream_test harness (grblhal-glowforge's checkout of forgefirm), gfhardware's test_machine_lid_button (a for-loop glob under its working-directory), gfutilities' test_frame (pytest on tests) and forgetest's own test_manifest (unittest discover under forgefirm/forgetest) pass; rows for forgectrl's fflog_e2e.sh, the bench script fire_test.py and kas/source-bundle.yml (only in forgetest-ci's paths filter) are refused. The lint on that manifest reports 0 uncovered paths and 0 host-tested row problems. Acceptance: none. forgetest is dev-only and in no fingerprint; the lint runs in CI, not in a campaign. |
||
|
ScottW514
|
0eb0a16452 |
forgetest: forgeext's packages/ exemption goes with the packages
OpenGlow's packages left forgeext for repositories of their own, so the
non-behavioral entry ("forgeext", "packages/**") names nothing. The
manifest test checks the shared package workflow's kit action as
non-behavioral in its place (".github/**" covers it). The manifest tests
pass (26); manifest.py is in no fingerprint.
|
||
|
ScottW514
|
dc9bede034 |
forgetest: the first /state parses each suite module once
The first GET /state after forgetest starts hashes every test's implementation, and it took from 81 s to more than 12 minutes on the bench reference. Two causes: - sibling_imports() read and parsed a test's module and every sibling it imports, transitively, once per test: 280 ast.parse calls for 26 suite modules. module_parts() kept its own parse, sibling_imports() kept none. - The page gives up on a poll after 20 s and polls again; the server thread goes on computing. Each new poll started the same cold work beside the first, all of it on the one CPU, so the longer the first took the more copies ran. That is the spread between 81 s and 12 minutes. Each module's text and tree are now read and parsed once and kept, as are its direct sibling imports, and the implementation hash is filled by one thread at a time: a poll that arrives while it is computed waits for it instead of repeating it. catalog.forget(path) drops what is kept about a file, for the unit tests that edit their modules. The hashes do not move: every test's implementation hash and domain fingerprint on the image manifest of 20260923220034 is byte-identical before and after, so no result is invalidated. On the host the cold computation went from 7.00 s to 0.29 s (280 parses to 25). On the bench reference (the file bind-mounted on image 20260923220034, the page open) the first /state answered in 8 to 12 s after a restart, where the same restart earlier in the evening had not answered after 7 minutes. tests/test_responsiveness.py pins both: each suite module parsed at most once while every test's implementation hash is computed, and three threads reading one test's hash compute it once. With the old behavior put back by a patch both fail (280 parses for 26 modules; the hash computed 3 times). forgetest's unit tests 454 OK. forgetest is the dev-only harness, outside the catalog's coverage; its catalog consequence is none, since no fingerprint moves. |
||
|
ScottW514
|
4e47b7fbf1 |
forgetest: exthost.catalog, and the coverage lint no longer allows an advisory away
exthost.catalog (suite/extcat.py, its own module): GET /ext/catalog answers the index the host keeps and the one address it is fetched from. On a scratch root under /tmp, with a throwaway key standing in for the OpenGlow extension key, the machine's own forgeext keeps an index signed with it, and the author key it names for one id makes a package of that id read as community and endorsed, where before it was unverified; the same key on another id counts for nothing. On the machine's own root that index is refused in words, a package handed over as an index is refused by the product gate, and the index kept is left as it was. The relay refuses an id with no such form (400) and one the kept index does not list (404, or 409 with none kept) before anything is fetched, and a refresh from the fixed address keeps OpenGlow's index when one is published there and is 502 in curl's words when none is, the kept index left as it was. Nothing is left in the staging directory. The coverage lint had a gap: coverage_report() let the allowlist's docs/** and **/*.md take out a path the BEHAVIORAL list keeps in every fingerprint, so the four first-run advisory documents were covered by no test and the lint passed. A change to the privacy advisory would have invalidated nothing. A behavioral path is now never allowed away, and setup.advisories-rehash, which accepts every first-run document at its current hash, covers the four. Proof: on the bench reference, with forgectrl 848ccc1 and forgeext a64b933 bind-mounted and the privacy document accepted again at its new hash with the fixture's press, exthost.catalog PASS (the refresh was 502: GitHub answered 404, nothing is published at the address yet), and setup.advisories-rehash PASS with the rest of the campaign. The lint's new unit test reports the uncovered advisory, and the old reading (the override ignored) reports nothing for it, which is the gap. forgetest's unit tests pass (452), and the coverage lint passes with --enforce. |
||
|
ScottW514
|
b7e7c0fb2c |
Manifest: the advisories are behavior, and forgeext's kit and packages are not
forgectrl embeds its advisory documents in the binary, serves them, and records the operator's consent to one by its hash, so an edited advisory is a changed consent. They are Markdown under docs/, which the non-behavioral list takes out of every fingerprint, and that left setup.extensions-consent's covers entry for docs/advisories/extensions.md selecting nothing: the enforced coverage lint fails on it (exit 1 on the dev image's manifest of 20260922225653), and an edited advisory moved no fingerprint at all. A BEHAVIORAL list now keeps forgectrl's docs/advisories/** in, ahead of the non-behavioral one. forgeext's recipe installs the binary and its init script and nothing else, so packages/ (the official packages, which carry their own acceptance artifact), sdk/ (the author's kit), template/ and tools/ are non-behavioral for the image: without that, an edit to the alignment page or the kit would make every exthost test stale on the next image. Proof: test_manifest passes its 25 cases, the new ones among them; the enforced coverage lint on the dev image's manifest of 20260922225653 exits 0 with no empty entry and nothing uncovered, where it exited 1 before. The whole forgetest suite ran its 451 tests; one, test_cloud_suite's test_pause_resume_passes_on_the_machines_lines, errored under the suite's load and passes 16 runs of 16 alone, at HEAD and on this tree alike: it replays a print against timed hooks. |
||
|
ScottW514
|
0eb764bf75 | Added SPDX | ||
|
ScottW514
|
b182a5ab0e |
acceptance: helper imports move the fingerprints, and the update test verifies a foreign signature
A test's fingerprint covered its own text and its module's shared text only, so a judge imported from a sibling suite module (laser.py takes its motion judges from motion.py) could change without moving the fingerprints of the tests that call it. The shared text of every sibling module a module imports now rides along, transitively; unit test. update.slots-and-signature claimed to refuse a tampered signature but fed fwup one garbage file. It now makes a throwaway key pair on the machine, signs a tiny archive, checks that the archive verifies with its own key and fails against the shipped release key, and asks the update job to apply it without confirm_unsigned: the job refuses it for its signature before touching the slot. |
||
|
ScottW514
|
335c6dea9d |
forgetest: non-behavioral paths outside every fingerprint; hollow covers fail the lint
The coverage lint already allowed docs, CI, unit tests and licenses to go uncovered; the same list now keeps them out of every fingerprint, so a README edit in any component re-requires nothing. The list moves to the manifest module as NON_BEHAVIORAL, the one place both uses read it. And a coverage entry that selects no file of its component (a glob without the recipe's subdirectory, a component the manifest lacks, a glob naming docs only) fails the lint: such an entry covers nothing and the test's fingerprint ignores the file it meant. The contract says both. Every test whose maps reached a doc or a test file gets a new fingerprint once. |
||
|
ScottW514
|
2547a8eb68 |
forgetest: a test's implementation hash is its own function and its module's shared code
The implementation half of a domain fingerprint was the whole suite
file, so a two-line witness fix in laser.py re-required every laser
test and a rename in cloud.py every cloud test: sixty attended minutes
for changes that touched two test bodies. Now the hash is the test's
own function (its decorator included) together with the module's text
outside every @test function. A body edit moves that test alone; a
helper edit moves the tests of its module, which is what a helper does;
a file that defines no test in the @test form hashes whole. The gate
computes it the same way, from the same code.
Every recorded fingerprint moves once with this, so the next campaign is
a full one: the price of every later fix costing one test.
Also carries the re-targeted cloud replays that the previous commit
left in the working tree (the CI failure on
|
||
|
ScottW514
|
c0f53a865f |
forgetest: the release acceptance tool and the bench diagnostics page
A stdlib-only daemon on the dev image (HTTP :8090) that runs the acceptance catalog against the machine from a self-contained page, keeps the append-only result log under /data/forgetest, and exports the release artifact the gate reads. Tests declare kind (auto / operator / live), hardware (api / takeover), coverage globs, prerequisites, and core membership; a test's domain fingerprint is the hash of the manifest files its globs select plus the platform and its own implementation, so a PASS stays valid exactly while nothing it covers changed. Campaign rules: a FAIL ends the campaign, the core (image health, kernel latch and drills, one live emission witness) is never inherited, invalidate-all forces a full campaign, no SKIP. Live tests need the operator acknowledgment and the physical arm press through the controller; takeover tests stop forgectrl for the duration with a crash-recoverable marker; the tool never touches the laser latch. Catalog v1: 24 tests ported from the proven bench drills with their recorded pass criteria (image, kernel K1-K3 and fire A/B/U, forgectrl API and logs, motion incl. dead-man, cooling, live laser, camera, update, cloud). The bench tab lists every scripts/bench tool and runs the board-side ones as subprocesses (takeover tools wrapped). 44 host unit tests, including the gate verification fixtures. Installed only by forgefirm-image-dev, with the bench scripts under /usr/share/forgetest/bench. |