From de324cc32f5676417a956fdfbe6198575dedfebc Mon Sep 17 00:00:00 2001 From: ScottW514 Date: Sat, 22 Aug 2026 16:36:03 -0400 Subject: [PATCH] forgetest: the operator's part asked for by name, and fewer hands in a campaign A campaign asked a person for about eighty things: lid, button and interlock actions, app jobs, and sixteen confirmations by eye, most of them as popups to read and answer while the head was already moving. This is the forgetest-only step of cutting that down. The operator channel. A test asks for its operator's part in four ways: ctx.ready() pre-announces a timed step and waits for the click that starts it; ctx.notice() is a standing instruction with no button, the test watching the machine for the result; ctx.act(channel, state) is a machine action by name (lid, interlock, button) - a notice for the operator today, proven done by the switch reading or an `until` condition, recorded in evidence.actions with who performed it, and the seam a bench actuator plugs into through runner.fixture; ctx.confirm() stays for the yes/no the evidence cannot answer. Tests declare `actions`; a `precheck` refuses a start the machine cannot honor (a reason, no result, a queue skips it and carries on). The page shows what you will do before it is asked: the running test's steps, a queue's attended tests still waiting, or the test whose title you clicked while idle; notices and prompts sit under it. The campaign card no longer carries baseline, queue and leftover notes: those go to the runner journal (daemon.log, syslog as `forgetest`, and the run in progress), with a Runner journal button in the footer. The catalog, 43 tests (27 auto, 8 operator, 8 live; was 45: 25/12/8): cloud.mode-switch absorbs cloud.hunt-lid-open and cloud.gfhome-homing (the connect made with the lid open, the hunt judged lid-open with its Z cycle, the re-hunt waited out after the close, the switch back, then $H judged by gfhome's own "homing complete" line with its motion windows; precheck homing_mode = gfcloud). kernel.fire-line is auto with the HV-not-good precheck, camera.snapshot is auto (a second frame with the lid lamp off differs and is smaller), motion.jog-roundtrip is auto (the head accelerometer per leg, the supervisor's own witness). The remaining attended tests use Ready gates and act(); the head's beam detector and the button LEDs replace the eye, leaving two confirms: the emission witness's mark and the app's display in cloud.pause-resume. Proof: tests/test_operator.py (the channel, the precheck, the journal), the cloud replays re-targeted to the merged round trip over a fake grbl and the bench excerpts, 196 host tests green, coverage lint 0 uncovered. Every re-ported attended test is owed one bench run on the next dev image (BRINGUP). Catalog consequence: the merged and reclassified tests' implementation hashes move; nothing else is invalidated. --- docs/ACCEPTANCE.md | 63 ++++- docs/BRINGUP.md | 33 ++- forgetest/forgetest/__main__.py | 5 +- forgetest/forgetest/catalog.py | 51 +++- forgetest/forgetest/hw.py | 132 ++++++++++ forgetest/forgetest/page.py | 44 +++- forgetest/forgetest/runner.py | 172 ++++++++++++- forgetest/forgetest/server.py | 21 ++ forgetest/forgetest/suite/camera.py | 45 +++- forgetest/forgetest/suite/cloud.py | 387 +++++++++++++++------------- forgetest/forgetest/suite/kernel.py | 36 ++- forgetest/forgetest/suite/laser.py | 189 +++++++++----- forgetest/forgetest/suite/motion.py | 144 ++++++++--- forgetest/tests/helpers.py | 93 ++++++- forgetest/tests/test_cloud_suite.py | 230 ++++++++++------- forgetest/tests/test_operator.py | 299 +++++++++++++++++++++ forgetest/tests/test_server.py | 18 +- 17 files changed, 1529 insertions(+), 433 deletions(-) create mode 100644 forgetest/tests/test_operator.py diff --git a/docs/ACCEPTANCE.md b/docs/ACCEPTANCE.md index ed73012..7914c7d 100644 --- a/docs/ACCEPTANCE.md +++ b/docs/ACCEPTANCE.md @@ -45,7 +45,18 @@ Every test declares, in code (`forgetest/forgetest/suite/*.py`): - **always** - membership in the **always-required core**, which is run in every campaign and is never inherited: image health, the kernel latch/safety readbacks, and one live emission witness with the - armed-window disarm. + armed-window disarm; +- **actions** - the machine actions the test asks for by name (`lid`, + `interlock`, `button`; see "The operator's part"). An `auto` test + declares none. The page lists them before a start; a bench actuator + that covers a channel can perform them; +- **precheck** - a condition the machine must meet for the test to start + at all (`kernel.fire-line` needs HV not reporting good, the kernel's + rule for a zero-duty latch unlock; `cloud.mode-switch` needs + `homing_mode = gfcloud`). A start the precheck refuses is not a result: + the page says why, a queue skips the test with the reason and carries + on, and nothing is recorded. Neither field is part of the gate-visible + definition. `GET /catalog` on the tool lists the definitions; the page shows them under each test's *details*. @@ -127,11 +138,13 @@ bench, or one whose `/data` has been wiped, starts from a full campaign. one switches from GRBL mode (once, its connect-time hunt waited out) and the following ones reuse the live session; nothing switches back after them. The tests that need GRBL mode (`motion.*`, `laser.*`, - `cooling.fans-quiet-after-motion`, `cloud.mode-switch`, - `cloud.gfhome-homing`) declare it, and the runner switches back the - moment one of them starts - so the mode changes only where the next - test asks for it, never between tests of the same mode. - `cloud.mode-switch` is the one round trip. + `cooling.fans-quiet-after-motion`, `cloud.mode-switch`) declare it, + and the runner switches back the moment one of them starts - so the + mode changes only where the next test asks for it, never between + tests of the same mode. `cloud.mode-switch` is the one round trip, and + it carries the two service-driven motions with it: the connect-time + hunt run with the lid open, and the web-service homing (`$H` with + `homing_mode = gfcloud`) after the switch back. 4. Or hand the whole list to a queue. **Run what is left** offers two: **Unattended** takes every `auto` test the campaign does not already count as satisfied, and needs nobody in the room; **Operator and live** @@ -152,7 +165,43 @@ bench, or one whose `/data` has been wiped, starts from a full campaign. `releases/v/acceptance.json` and `.md`. The raw log (`/data/forgetest/results.jsonl`, `Raw log` in the footer) is -the bench's own record; the artifact is the release's. +the bench's own record; the artifact is the release's. The runner's own +events - a queue opening, skipping or stopping, a takeover recovered at +start-up, the leftovers a baseline pass found - go to the **journal** +(`Runner journal` in the footer: the daemon's `daemon.log` under the +data directory, also syslog under the `forgetest` name, and the log of +the run in progress), never to the page's campaign card. + +### The operator's part + +The run card shows **what you will do** before anything is asked: the +running test's `steps`, the attended tests still waiting in a queue, or +the test whose title you clicked while the machine is idle. What follows +during the run is those steps, taken in turn, in one of four forms: + +- a **Ready** prompt pre-announces a timed step: what happens on the + click and what you do during it ("On Ready the head starts an 8 s move; + press the button once while it moves"). Nothing moves until you click; +- a **notice** is a standing instruction with no button. The test shows + it and watches the machine for the result - the lid switch reading + open, the interlock loop reading open, the controller entering Hold + after the press, the client's log line - and takes it down when it sees + it. There is nothing to answer and nothing to race; +- a machine **action** (`ctx.act("lid", "open")`, `("interlock", + "close")`, `("button", "press", until=...)`) is a notice the runner + manages: the wording is the action's own, the test adds its context, + the machine's reading proves it done, and the result's + `evidence.actions` records each one with who performed it. This is the + seam a bench actuator plugs into: a runner `fixture` covering a channel + performs the action instead of the notice, and a test reads the same + either way; +- a **confirm** is a yes/no the evidence cannot answer. One is left in + the catalog: the mark `laser.emission-witness` leaves on the scrap, the + once-per-campaign calibration of the sensor witnesses (the head's beam + detector, the HV current, the kernel's LASER_ON count), plus the app's + own display in `cloud.pause-resume`. The head accelerometer stands in + for "did the gantry move", the button LEDs for "is the button dark", + the lid lamp toggled between two snapshots for "is the camera live". ### Every run starts from, and leaves, the fresh-boot idle state diff --git a/docs/BRINGUP.md b/docs/BRINGUP.md index 754e72a..63fb7ec 100644 --- a/docs/BRINGUP.md +++ b/docs/BRINGUP.md @@ -48,7 +48,7 @@ hardware-validated.** modes (cancel-and-return on a lid or interlock open, button pause/resume), bench-validated 2026-08-17. - **Releases are gated by the acceptance tool** (`forgetest`, dev image only): - a 44-test catalog, domain-scoped inheritance, an always-required safety core, + a 43-test catalog, domain-scoped inheritance, an always-required safety core, and a release gate that reads the exported artifact. The full campaign on dev image `20260821181036` (the first built on the `-pin.inc` layout) satisfied 42 of 42 and its export authorizes a release; @@ -579,17 +579,29 @@ under the domain model from the day's earlier dev images) and the export reads " YES" for that image's manifest. That authorizes a release; it is not one until `releases/v/acceptance.json` is committed. -- **Catalog: 45 tests** in `forgetest/forgetest/suite/`, every one a port of a - proven bench drill or a bench-verified check — the always-required core +- **Catalog: 43 tests** in `forgetest/forgetest/suite/`, every one a port of a + proven bench drill or a bench-verified check: the always-required core (`image.health`, `kernel.latch-locked-idle`, `kernel.k1-k2`, `kernel.fire-line`), `forgectrl.*`, `logs.*`, `update.*`, `motion.*` (pacing, jog round-trip, liveness probe, cancel/abort, dead-man, the lid, interlock and button parity tests), `cooling.*` (flow verification, fans quiet after motion, a gate setting tripping and off by value, a fan under - its floor), `camera.snapshot`, + its floor), `camera.*`, `laser.*` (emission witness, arm-wait lid, disarm-in-hold, armed kill, - pause/resume/lid-cancel) and `cloud.*`. Tests that share a setup are merged; - the `auto` tests stay separate for failure isolation. + pause/resume/lid-cancel) and `cloud.*` (the mode round trip with the + lid-open hunt and the web-service homing on it, and the job-behavior + tests). Tests that share a setup are merged; the `auto` tests stay + separate for failure isolation. 27 are `auto`, 8 `operator`, 8 `live`. +- **The operator's part is asked for by name, not by popup** + (`docs/ACCEPTANCE.md` "The operator's part"): a Ready prompt before a + timed step, a standing notice the test takes down when the machine shows + the action done (`ctx.act("lid", "open")` and its kin, the seam a bench + actuator will plug into), and one confirm by eye left in the catalog (the + emission witness's mark). The head accelerometer, the beam detector, the + button LEDs, and a lid-lamp toggle between two snapshots replaced the + other eyeball confirmations; `kernel.fire-line` and `camera.snapshot` are + `auto`. Code-complete 2026-08-22 with host replays; **bench validation + pending** on the next dev image. - **Machine identity is content-defined.** Every component recipe contributes `forgefirm-manifest.bbclass` entries (the kernel and the module through `do_deploy`), `forgefirm-image-manifest.bbclass` assembles them plus the layer @@ -1134,7 +1146,14 @@ Open items only. Anything closed is in `CAMPAIGN-LOG.md`. service. Still owed: exercising the ported bench tools from the page (they are registered and unit-tested, not yet driven from the page), and the first release, which commits `releases/v/acceptance.json`. - Catalog gaps left from the tool's own plan: `cooling.confirm-escalate` and + Cutting the operator's part of a campaign: the forgetest-only step + (the operator channel, the merged mode-switch, the sensor witnesses, + the steps pane, the journal) is code-complete and host-replayed; it + needs a bench run of every re-ported attended test on the next dev + image. The steps after it, an offline cloud service for the + machine-behavior tests and a bench actuator for the lid, interlock and + button, are planned, not started. Catalog + gaps left from the tool's own plan: `cooling.confirm-escalate` and `cooling.fire-gate-blocks-arm` are not ported (both need the pump switched by hand mid-run, so they are bench-tab material first), and whether `laser.armed-kill` belongs in the always-required core rather than its diff --git a/forgetest/forgetest/__main__.py b/forgetest/forgetest/__main__.py index 9bff2a6..3db1149 100644 --- a/forgetest/forgetest/__main__.py +++ b/forgetest/forgetest/__main__.py @@ -18,7 +18,7 @@ from . import catalog as _catalog from . import manifest as _manifest from . import server as _server from .log import Log, data_dir -from .runner import Runner +from .runner import Runner, configure_journal def main(argv=None): @@ -36,6 +36,7 @@ def main(argv=None): return 2 registry = _catalog.load_suite() os.makedirs(data_dir(), exist_ok=True) + configure_journal() log = Log() bench = _bench.Bench() runner = Runner(log, manifest, registry, bench) @@ -52,8 +53,6 @@ def main(argv=None): print("forgetest %s: %d tests, image %s (%s), listening on %s:%d" % (VERSION, len(registry), manifest.version, (manifest.content_sha or "")[:12], args.host, args.port), file=sys.stderr, flush=True) - for m in runner.messages: - print("forgetest: %s" % m, file=sys.stderr, flush=True) th = threading.Thread(target=srv.serve_forever, name="forgetest-http", daemon=True) th.start() try: diff --git a/forgetest/forgetest/catalog.py b/forgetest/forgetest/catalog.py index 93addd4..aecd67b 100644 --- a/forgetest/forgetest/catalog.py +++ b/forgetest/forgetest/catalog.py @@ -5,7 +5,12 @@ what the release gate needs to know without running anything: the id, the subsystem, the kind (auto / operator / live), how it takes the hardware (api / takeover), the controller mode it needs (if any), what source it covers, what it requires, and whether it belongs to the -always-required core. The function body runs +always-required core. Two more fields describe the operator's part +without affecting the gate: `actions`, the machine actions the test asks +for by name (the page lists them before a start; a bench actuator can +perform them), and `precheck`, a condition the machine must meet for the +test to start at all (a reason string refuses the start, the way an +unmet prerequisite does, and records no result). The function body runs under the runner with a Context (log, prompts, evidence, hardware helpers) and reports by returning normally (PASS) or raising runner.Failed (FAIL). @@ -20,6 +25,11 @@ from . import manifest as _manifest KINDS = ("auto", "operator", "live") HARDWARE = ("api", "takeover") MODES = ("grbl", "cloud") +# The machine actions a test may ask of the operator (Context.act): +# the lid, the remote-interlock loop, and the big button. Everything a +# test needs done to the machine is one of these, so a bench actuator +# that covers a channel can stand in for the hands on it. +ACTIONS = ("lid", "interlock", "button") _ID_RX = re.compile(r"^[a-z][a-z0-9-]*\.[a-z][a-z0-9-]*$") REGISTRY = {} @@ -27,7 +37,8 @@ REGISTRY = {} class Test: def __init__(self, id, title, subsystem, kind, hardware, covers, requires, - always, est_min, steps, description, fn, mode=None): + always, est_min, steps, description, fn, mode=None, actions=(), + precheck=None): self.id = id self.title = title self.subsystem = subsystem @@ -39,6 +50,8 @@ class Test: self.always = bool(always) self.est_min = est_min self.steps = tuple(steps) + self.actions = tuple(actions) + self.precheck = precheck self.description = description or (fn.__doc__ or "").strip() self.fn = fn self._source_sha = None @@ -82,9 +95,21 @@ class Test: def describe(self): d = self.definition() d.update({"title": self.title, "est_min": self.est_min, "mode": self.mode, - "steps": list(self.steps), "description": self.description}) + "steps": list(self.steps), "actions": list(self.actions), + "precheck": bool(self.precheck), "description": self.description}) return d + def cannot_start(self): + """The reason the test cannot start on the machine as it is, or + None. Evaluated right before a start; never a result.""" + if self.precheck is None: + return None + try: + reason = self.precheck() + except Exception as e: # noqa: BLE001 - a broken precheck refuses, it never crashes the runner + return "precheck errored: %s: %s" % (type(e).__name__, e) + return reason or None + def source_file_sha(path): with open(path, "rb") as f: @@ -93,13 +118,19 @@ def source_file_sha(path): def test(id, *, title, subsystem, kind="auto", hardware="api", mode=None, covers=(), - requires=(), always=False, est_min=1, steps=(), description=""): + requires=(), always=False, est_min=1, steps=(), description="", actions=(), + precheck=None): """`mode` names the controller mode the test needs live when it starts ("grbl" or "cloud"); the runner switches the machine there before the test and leaves it there, so a queue crosses modes only where a test asks it to. None means the test runs in whatever mode it finds (or manages the mode itself, as the cloud tests do through enter_cloud, - which also waits for the service session).""" + which also waits for the service session). + + `actions` names the machine actions (ACTIONS) the test performs + through Context.act; an `auto` test declares none. `precheck` is a + callable returning a reason string when the machine cannot run the + test as it is (None when it can).""" if not _ID_RX.match(id): raise ValueError("test id %r must look like subsystem.name" % id) if kind not in KINDS: @@ -111,12 +142,20 @@ def test(id, *, title, subsystem, kind="auto", hardware="api", mode=None, covers for c, g in covers: if c in _manifest.DEV_ONLY_COMPONENTS: raise ValueError("test %s: may not cover dev-only component %r" % (id, c)) + for a in actions: + if a not in ACTIONS: + raise ValueError("test %s: action %r" % (id, a)) + if actions and kind == "auto": + raise ValueError("test %s: an auto test asks for no machine actions" % id) + if precheck is not None and not callable(precheck): + raise ValueError("test %s: precheck must be callable" % id) def deco(fn): if id in REGISTRY: raise ValueError("duplicate test id %r" % id) REGISTRY[id] = Test(id, title, subsystem, kind, hardware, covers, requires, - always, est_min, steps, description, fn, mode=mode) + always, est_min, steps, description, fn, mode=mode, + actions=actions, precheck=precheck) return fn return deco diff --git a/forgetest/forgetest/hw.py b/forgetest/forgetest/hw.py index 1d3ebe7..28d188d 100644 --- a/forgetest/forgetest/hw.py +++ b/forgetest/forgetest/hw.py @@ -13,6 +13,7 @@ import json import os import socket import subprocess +import threading import time import urllib.error import urllib.parse @@ -152,6 +153,137 @@ def sysfs_write(attr, value): f.write(str(value)) +# ------------------------------------------------------ head accelerometer + +# The head accelerometer (LIS2HH12 on i2c-3 at 0x1e): the supervisor's +# motion-liveness witness, and the catalog's. Resolved by bus address, +# never by iio index. The raw sysfs read is slow (~150 ms), which still +# lands several samples in a one-second move; the verdict is peak-to-peak +# on X or Y, against the thresholds forgectrl's probe established (a live +# head reads 1800-2900 on its probe move, a wedged one <= 250). +HEAD_ACCEL_I2C = "3-001e" +ACCEL_P2P_MOVING = 800 + + +def head_accel_dir(): + """The head accel's iio directory, or None. GF_IIO_ROOT overrides the + /sys/bus/iio/devices root (host tests).""" + root = os.environ.get("GF_IIO_ROOT") or "/sys/bus/iio/devices" + try: + names = sorted(os.listdir(root)) + except OSError: + return None + for n in names: + d = os.path.join(root, n) + try: + target = os.readlink(d) + except OSError: + target = "" + if HEAD_ACCEL_I2C in target or HEAD_ACCEL_I2C in n: + return d + # a plain directory (host fixture): its name carries the address + try: + with open(os.path.join(d, "name")) as f: + if HEAD_ACCEL_I2C in f.read(): + return d + except OSError: + pass + return None + + +def head_accel_read(d): + """(x, y) raw counts, or None.""" + out = [] + for axis in ("x", "y"): + try: + with open(os.path.join(d, "in_accel_%s_raw" % axis)) as f: + out.append(int(f.read().strip())) + except (OSError, ValueError): + return None + return tuple(out) + + +class AccelSampler: + """Samples the head accelerometer in a thread for as long as it is + running; `p2p(t0, t1)` is the peak-to-peak on X and Y over the samples + in that window and how many there were. Used as a context manager + around a motion the test wants the head to have made.""" + + def __init__(self, period=0.0): + self.dir = head_accel_dir() + self.period = period + self.samples = [] # (t, x, y) + self._stop = threading.Event() + self._th = None + self.errors = 0 + + @property + def available(self): + return self.dir is not None + + def __enter__(self): + if self.dir is not None: + self._th = threading.Thread(target=self._loop, daemon=True, name="forgetest-accel") + self._th.start() + return self + + def __exit__(self, *a): + self._stop.set() + if self._th is not None: + self._th.join(2.0) + return False + + def _loop(self): + while not self._stop.is_set(): + v = head_accel_read(self.dir) + if v is None: + self.errors += 1 + else: + self.samples.append((time.time(), v[0], v[1])) + if self.period: + self._stop.wait(self.period) + + def p2p(self, t0, t1=None): + """(p2p_x, p2p_y, n) over [t0, t1]; (0, 0, 0) with no samples.""" + t1 = time.time() if t1 is None else t1 + xs = [x for t, x, _y in self.samples if t0 <= t <= t1] + ys = [y for t, _x, y in self.samples if t0 <= t <= t1] + if not xs: + return 0, 0, 0 + return max(xs) - min(xs), max(ys) - min(ys), len(xs) + + +# ----------------------------------------------------------- button LEDs + +BUTTON_LEDS = ("button_led_1", "button_led_2", "button_led_3") + + +def leds_root(): + r = os.environ.get("GF_LEDS_ROOT") or "/sys/class/leds/" + return r if r.endswith("/") else r + "/" + + +def button_leds(): + """The three button LED brightnesses, or None where unreadable.""" + out = [] + for name in BUTTON_LEDS: + try: + with open(leds_root() + name + "/brightness") as f: + out.append(int(f.read().strip())) + except (OSError, ValueError): + out.append(None) + return out + + +def button_lit(): + """True when any button LED is on, False when all three read 0, None + when none is readable.""" + vals = [v for v in button_leds() if v is not None] + if not vals: + return None + return any(v > 0 for v in vals) + + # --------------------------------------------------------------- init.d def initd(service, action, timeout=60): diff --git a/forgetest/forgetest/page.py b/forgetest/forgetest/page.py index c3ef1db..a03bbed 100644 --- a/forgetest/forgetest/page.py +++ b/forgetest/forgetest/page.py @@ -87,6 +87,13 @@ input[type=text],input[type=number],select{background:#fff;border:1px solid #c9c .hint{color:var(--dim);font-size:12.5px;line-height:1.55;margin:8px 0 0} pre#log{background:#1d1e26;color:#d7dae0;font-family:ui-monospace,Consolas,monospace;font-size:11.5px;padding:10px;border-radius:4px;height:380px;overflow:auto;margin:8px 0;white-space:pre-wrap;word-break:break-all} #prompt{background:#fdf3e3;border:1px solid #eccb90;border-radius:6px;padding:10px 12px;margin:8px 0} +#notice{background:#e8f1fb;border:1px solid #9dbde6;border-radius:6px;padding:10px 12px;margin:8px 0;font-weight:600} +#steps{background:#f7f8fa;border-radius:6px;padding:8px 12px;margin:8px 0;font-size:12.5px;line-height:1.5} +#steps .sh{font-weight:600;margin-bottom:4px} +#steps .sub{color:var(--dim);margin:6px 0 2px} +#steps ol{margin:2px 0 4px 20px} +#steps .auto{color:var(--dim);font-style:italic} +.tsel{cursor:pointer}.tsel:hover{text-decoration:underline} #prompt .q{font-weight:600;margin-bottom:8px} .details{display:none;background:#f7f8fa;padding:8px 10px;border-radius:4px;font-size:12.5px;line-height:1.55;margin-top:6px} .details.on{display:block} @@ -120,7 +127,7 @@ pre#log{background:#1d1e26;color:#d7dae0;font-family:ui-monospace,Consolas,monos

Campaign

-
+

Run what is left

@@ -139,6 +146,7 @@ pre#log{background:#1d1e26;color:#d7dae0;font-family:ui-monospace,Consolas,monos +
@@ -165,6 +173,8 @@ pre#log{background:#1d1e26;color:#d7dae0;font-family:ui-monospace,Consolas,monos