forgetest: the operator's part asked for by name, and fewer hands in a campaign

A campaign asked a person for about eighty things: lid, button and
interlock actions, app jobs, and sixteen confirmations by eye, most of
them as popups to read and answer while the head was already moving.
This is the forgetest-only step of cutting that down.

The operator channel. A test asks for its operator's part in four ways:
ctx.ready() pre-announces a timed step and waits for the click that
starts it; ctx.notice() is a standing instruction with no button, the
test watching the machine for the result; ctx.act(channel, state) is a
machine action by name (lid, interlock, button) - a notice for the
operator today, proven done by the switch reading or an `until`
condition, recorded in evidence.actions with who performed it, and the
seam a bench actuator plugs into through runner.fixture; ctx.confirm()
stays for the yes/no the evidence cannot answer. Tests declare
`actions`; a `precheck` refuses a start the machine cannot honor (a
reason, no result, a queue skips it and carries on).

The page shows what you will do before it is asked: the running test's
steps, a queue's attended tests still waiting, or the test whose title
you clicked while idle; notices and prompts sit under it. The campaign
card no longer carries baseline, queue and leftover notes: those go to
the runner journal (daemon.log, syslog as `forgetest`, and the run in
progress), with a Runner journal button in the footer.

The catalog, 43 tests (27 auto, 8 operator, 8 live; was 45: 25/12/8):
cloud.mode-switch absorbs cloud.hunt-lid-open and cloud.gfhome-homing
(the connect made with the lid open, the hunt judged lid-open with its
Z cycle, the re-hunt waited out after the close, the switch back, then
$H judged by gfhome's own "homing complete" line with its motion
windows; precheck homing_mode = gfcloud). kernel.fire-line is auto with
the HV-not-good precheck, camera.snapshot is auto (a second frame with
the lid lamp off differs and is smaller), motion.jog-roundtrip is auto
(the head accelerometer per leg, the supervisor's own witness). The
remaining attended tests use Ready gates and act(); the head's beam
detector and the button LEDs replace the eye, leaving two confirms: the
emission witness's mark and the app's display in cloud.pause-resume.

Proof: tests/test_operator.py (the channel, the precheck, the journal),
the cloud replays re-targeted to the merged round trip over a fake grbl
and the bench excerpts, 196 host tests green, coverage lint 0 uncovered.
Every re-ported attended test is owed one bench run on the next dev
image (BRINGUP). Catalog consequence: the merged and reclassified
tests' implementation hashes move; nothing else is invalidated.
This commit is contained in:
ScottW514
2026-08-22 16:36:03 -04:00
parent 58d1ac4746
commit de324cc32f
17 changed files with 1529 additions and 433 deletions
+56 -7
View File
@@ -45,7 +45,18 @@ Every test declares, in code (`forgetest/forgetest/suite/*.py`):
- **always** - membership in the **always-required core**, which is run
in every campaign and is never inherited: image health, the kernel
latch/safety readbacks, and one live emission witness with the
armed-window disarm.
armed-window disarm;
- **actions** - the machine actions the test asks for by name (`lid`,
`interlock`, `button`; see "The operator's part"). An `auto` test
declares none. The page lists them before a start; a bench actuator
that covers a channel can perform them;
- **precheck** - a condition the machine must meet for the test to start
at all (`kernel.fire-line` needs HV not reporting good, the kernel's
rule for a zero-duty latch unlock; `cloud.mode-switch` needs
`homing_mode = gfcloud`). A start the precheck refuses is not a result:
the page says why, a queue skips the test with the reason and carries
on, and nothing is recorded. Neither field is part of the gate-visible
definition.
`GET /catalog` on the tool lists the definitions; the page shows them under
each test's *details*.
@@ -127,11 +138,13 @@ bench, or one whose `/data` has been wiped, starts from a full campaign.
one switches from GRBL mode (once, its connect-time hunt waited out)
and the following ones reuse the live session; nothing switches back
after them. The tests that need GRBL mode (`motion.*`, `laser.*`,
`cooling.fans-quiet-after-motion`, `cloud.mode-switch`,
`cloud.gfhome-homing`) declare it, and the runner switches back the
moment one of them starts - so the mode changes only where the next
test asks for it, never between tests of the same mode.
`cloud.mode-switch` is the one round trip.
`cooling.fans-quiet-after-motion`, `cloud.mode-switch`) declare it,
and the runner switches back the moment one of them starts - so the
mode changes only where the next test asks for it, never between
tests of the same mode. `cloud.mode-switch` is the one round trip, and
it carries the two service-driven motions with it: the connect-time
hunt run with the lid open, and the web-service homing (`$H` with
`homing_mode = gfcloud`) after the switch back.
4. Or hand the whole list to a queue. **Run what is left** offers two:
**Unattended** takes every `auto` test the campaign does not already
count as satisfied, and needs nobody in the room; **Operator and live**
@@ -152,7 +165,43 @@ bench, or one whose `/data` has been wiped, starts from a full campaign.
`releases/v<version>/acceptance.json` and `.md`.
The raw log (`/data/forgetest/results.jsonl`, `Raw log` in the footer) is
the bench's own record; the artifact is the release's.
the bench's own record; the artifact is the release's. The runner's own
events - a queue opening, skipping or stopping, a takeover recovered at
start-up, the leftovers a baseline pass found - go to the **journal**
(`Runner journal` in the footer: the daemon's `daemon.log` under the
data directory, also syslog under the `forgetest` name, and the log of
the run in progress), never to the page's campaign card.
### The operator's part
The run card shows **what you will do** before anything is asked: the
running test's `steps`, the attended tests still waiting in a queue, or
the test whose title you clicked while the machine is idle. What follows
during the run is those steps, taken in turn, in one of four forms:
- a **Ready** prompt pre-announces a timed step: what happens on the
click and what you do during it ("On Ready the head starts an 8 s move;
press the button once while it moves"). Nothing moves until you click;
- a **notice** is a standing instruction with no button. The test shows
it and watches the machine for the result - the lid switch reading
open, the interlock loop reading open, the controller entering Hold
after the press, the client's log line - and takes it down when it sees
it. There is nothing to answer and nothing to race;
- a machine **action** (`ctx.act("lid", "open")`, `("interlock",
"close")`, `("button", "press", until=...)`) is a notice the runner
manages: the wording is the action's own, the test adds its context,
the machine's reading proves it done, and the result's
`evidence.actions` records each one with who performed it. This is the
seam a bench actuator plugs into: a runner `fixture` covering a channel
performs the action instead of the notice, and a test reads the same
either way;
- a **confirm** is a yes/no the evidence cannot answer. One is left in
the catalog: the mark `laser.emission-witness` leaves on the scrap, the
once-per-campaign calibration of the sensor witnesses (the head's beam
detector, the HV current, the kernel's LASER_ON count), plus the app's
own display in `cloud.pause-resume`. The head accelerometer stands in
for "did the gantry move", the button LEDs for "is the button dark",
the lid lamp toggled between two snapshots for "is the camera live".
### Every run starts from, and leaves, the fresh-boot idle state
+26 -7
View File
@@ -48,7 +48,7 @@ hardware-validated.**
modes (cancel-and-return on a lid or interlock open, button pause/resume),
bench-validated 2026-08-17.
- **Releases are gated by the acceptance tool** (`forgetest`, dev image only):
a 44-test catalog, domain-scoped inheritance, an always-required safety core,
a 43-test catalog, domain-scoped inheritance, an always-required safety core,
and a release gate that reads the exported artifact. The full campaign on
dev image `20260821181036` (the first built on the `<recipe>-pin.inc`
layout) satisfied 42 of 42 and its export authorizes a release;
@@ -579,17 +579,29 @@ under the domain model from the day's earlier dev images) and the export reads "
YES" for that image's manifest. That authorizes a release; it is not one
until `releases/v<version>/acceptance.json` is committed.
- **Catalog: 45 tests** in `forgetest/forgetest/suite/`, every one a port of a
proven bench drill or a bench-verified check — the always-required core
- **Catalog: 43 tests** in `forgetest/forgetest/suite/`, every one a port of a
proven bench drill or a bench-verified check: the always-required core
(`image.health`, `kernel.latch-locked-idle`, `kernel.k1-k2`,
`kernel.fire-line`), `forgectrl.*`, `logs.*`, `update.*`, `motion.*`
(pacing, jog round-trip, liveness probe, cancel/abort, dead-man, the lid,
interlock and button parity tests), `cooling.*` (flow verification, fans
quiet after motion, a gate setting tripping and off by value, a fan under
its floor), `camera.snapshot`,
its floor), `camera.*`,
`laser.*` (emission witness, arm-wait lid, disarm-in-hold, armed kill,
pause/resume/lid-cancel) and `cloud.*`. Tests that share a setup are merged;
the `auto` tests stay separate for failure isolation.
pause/resume/lid-cancel) and `cloud.*` (the mode round trip with the
lid-open hunt and the web-service homing on it, and the job-behavior
tests). Tests that share a setup are merged; the `auto` tests stay
separate for failure isolation. 27 are `auto`, 8 `operator`, 8 `live`.
- **The operator's part is asked for by name, not by popup**
(`docs/ACCEPTANCE.md` "The operator's part"): a Ready prompt before a
timed step, a standing notice the test takes down when the machine shows
the action done (`ctx.act("lid", "open")` and its kin, the seam a bench
actuator will plug into), and one confirm by eye left in the catalog (the
emission witness's mark). The head accelerometer, the beam detector, the
button LEDs, and a lid-lamp toggle between two snapshots replaced the
other eyeball confirmations; `kernel.fire-line` and `camera.snapshot` are
`auto`. Code-complete 2026-08-22 with host replays; **bench validation
pending** on the next dev image.
- **Machine identity is content-defined.** Every component recipe contributes
`forgefirm-manifest.bbclass` entries (the kernel and the module through
`do_deploy`), `forgefirm-image-manifest.bbclass` assembles them plus the layer
@@ -1134,7 +1146,14 @@ Open items only. Anything closed is in `CAMPAIGN-LOG.md`.
service. Still owed: exercising the ported bench tools from the page
(they are registered and unit-tested, not yet driven from the page), and
the first release, which commits `releases/v<version>/acceptance.json`.
Catalog gaps left from the tool's own plan: `cooling.confirm-escalate` and
Cutting the operator's part of a campaign: the forgetest-only step
(the operator channel, the merged mode-switch, the sensor witnesses,
the steps pane, the journal) is code-complete and host-replayed; it
needs a bench run of every re-ported attended test on the next dev
image. The steps after it, an offline cloud service for the
machine-behavior tests and a bench actuator for the lid, interlock and
button, are planned, not started. Catalog
gaps left from the tool's own plan: `cooling.confirm-escalate` and
`cooling.fire-gate-blocks-arm` are not ported (both need the pump switched
by hand mid-run, so they are bench-tab material first), and whether
`laser.armed-kill` belongs in the always-required core rather than its