diff --git a/scripts/bench/README.md b/scripts/bench/README.md index a204377..f58d489 100644 --- a/scripts/bench/README.md +++ b/scripts/bench/README.md @@ -1,7 +1,11 @@ # ForgeFIRM bench tools Hardware-verification tools for the ForgeFIRM bench. All run ON the -target board (dev image, python3 present) unless noted. +target board (dev image, python3 present) unless noted. Host-side tools +take the machine address from `GF_HOST` (or `argv`, where stated); the +ones that shell into the board over ssh use the `ssh` on `PATH`, or the +client named by `GF_SSH` (for example `GF_SSH='wsl -d -- ssh'` +to go through a WSL distro from Windows). | Tool | Purpose | |---|---| @@ -11,14 +15,23 @@ target board (dev image, python3 present) unless noted. | `pwm_sweep.py` | LASER_PWM scope test (runs on the board): `check` = read-only safety readbacks + PWM2 dump; `sweep` = steps PWMSAR through 50/25/75/6/100 % duty with 4 s holds, then restores. Run only in the locked state (controller stopped, cnc disabled, latch locked). | | `pwm_hold.py` | Holds one PWMSAR value for a scope-measurement window (`pwm_hold.py `), then restores. Same locked-state rule. | | `fire_test.py` | FIRE drop-timing scope test (runs on the board): A = latch locked (expects nothing on FIRE/LASER_ON), B = latch unlocked / normal end-of-data, U = true underrun. Duty 0 throughout; refuses to unlock if HV reports good. | +| `pwm_stream_test.py` | LASER_PWM stream-path scope test (runs on the board): streams power bytes only (no step bytes, no FIRE bits, `motor_lock=15`, latch locked) through `/dev/glowforge` so the scope verifies the real power path, including the run-start duty reset and the consecutive-power-byte drop; position counters compared before/after. | +| `gate_a_kernel_drills.py` | Kernel laser-safety drills (run on the board with forgectrl stopped so the pulse device is free): `K1` controlled-stop deceleration floor, `K2` resume waypoint honors the locked latch, `K3` a mid-ramp latch unlock never re-arms the FIRE drive. Software witnesses (`cnc/state`, `laser_enable`, `laser_on`, `laser_on_sampled`, interlock bit 3) plus the PSU-connector LASER_ON scope point; K3 refuses to run if HV reports good. | +| `laser_stream_test.py` | Host-side laser pulse-stream emission harness: runs the native null-sink controller with `GFSINK_DUMP`, drives small laser jobs over TCP, and checks the dumped bytes against the kernel feeder contract (leading power byte, no back-to-back power bytes, FIRE only inside cutting moves, every stream ends FIRE-clear, no FIRE on a stepless gap, no FIRE leak across cycle churn). Runs in the grblHAL repo's CI. | +| `laser_lifecycle_test.py` | Host-side operator-armed-window lifecycle harness (null-sink controller): arm once per job with M5/M3 persistence, the M2 close, sender-change re-consent, the disarm grace counting down in Hold, and arm refusal under a blocking cooling verdict. Runs in the grblHAL repo's CI. | +| `live_fire_drills.py` | **LIVE LASER** drills from a LAN host (`live_fire_drills.py [host]`, or `GF_HOST`): `witness` (emission witness, lid-IR peaks vs the ambient baseline, HV current, job-based disarm on M2), `hold` (disarm grace in Hold), `faultpos` (armed job refuses a stale origin after an underrun). Every drill waits for the operator's physical arm press; eye protection, fire watch, extinguisher, and exhaust are mandatory. | +| `pacing_test.py` | Protocol-loop pacing check (runs on the board, dry motion): idle and parked-in-Hold states are coarse-paced, active motion is tight-paced, and a feed-hold/resume mid-move preserves position with no feeder starve. | | `fan_test.py` | Fan/coolant bench (Windows-side): snapshots fan PWMs/tachs/temps, drives M8 → cut fans, M9 → cooldown → idle, verifying via tach readbacks. | | `flow_characterize.py` | Coolant flow characterization using the factory temperature curve: baseline → flow → no-flow → recovery, printing the ΔT bands and their separation. Takes the heater duty as an argument (`flow_characterize.py 30`); aborts if downstream passes 45 °C. | | `flow_matrix.py` | **The flow-detection design matrix** (with `flow_sampler.py`, which lives on the board at `/data/`): duty × duration × flow/no-flow, every run from a common cooled baseline, interleaved repeats. One heating trace yields the metric at every candidate duration, so cost and precision come from the same 60 runs. Prints a cost table, a precision table (mean±sd, worst-case margin, d′) and a ranked shortlist. Env: `FM_DUTIES`, `FM_REPEATS`, `FM_RESULTS`. | | `flow_sustained.py` | Long-run test of the real re-check cadence via M8: counts verdicts/false faults and tracks whether the loop accumulates heat. | | `flow_warm_validate.py` | Runs the real check from a heater-warmed baseline. Note the ceiling: 100 % duty pushes the downstream sensor past 50 °C in 30 s while the bulk barely moves, so warm-loop validation above ~23 °C needs the laser, not the heater. | | `flow_recheck_char.py` | Characterizes short in-run re-checks and the differential metric; shows why over-temp cannot see a stopped pump and why passive warming trends are ambiguous. | +| `flow_confirm_drill.py` | Coolant flow suspicion/confirmation drill (runs on the board): one continuous M8 session walks the verdict state machine through real pump-off transients — verified → SUSPECT (+ immediate re-check) → cleared → SUSPECT → FAULT (consecutive) → recovered — printing PASS/FAIL per transition. Leaves the machine idle (M9, pump on, heater off). | +| `flow_escalate_drill.py` | Coolant starved-re-check escalation drill (runs on the board against a controller started with a short confirmation budget): with the pump off the job-start check reads SUSPECT, the stagnant loop cannot pass the settle gate inside the budget, and the driver must escalate to FAULT. PASS/FAIL, leaves the machine idle. | +| `flow_sampler.py` | Board-side coolant sampler used by the flow tools (`flow_sampler.py `, prints `elapsed,raw_down,raw_up`); kept on the board at `/data/` so cadence does not depend on ssh latency. | | `temp_calibrate.py` | Coolant temperature spot-check helper (`watch` / `point ` / `fit`) — pairs a measured temperature with averaged raw ADC readings and fits a per-machine line to sanity-check the factory curve against a thermometer. | -| `build-glowforge.sh` | Cross-compiles **grblHAL-glowforge** (the canonical driver repo, `../../../grblHAL-glowforge`) in the forge-yocto WSL distro. Run: `wsl -d forge-yocto -- bash /build-glowforge.sh` (from PowerShell; Git Bash mangles /mnt/c paths). This is the production controller build. | +| `build-glowforge.sh` | Cross-compiles **grblHAL-glowforge** (the canonical driver repo, `../../../grblHAL-glowforge`) in the Yocto build environment, borrowing the recipe toolchain. Run: `bash /build-glowforge.sh` (from Windows, launch it through the WSL distro from PowerShell; Git Bash mangles /mnt/c paths). Env: `FF_SRC_TOP`, `FF_BUILD_TOP`. This is the production controller build. | | `build-forgectrl.sh` | Cross-compiles **forgectrl** (the canonical control-daemon repo, `../../../forgectrl`) the same way, borrowing the toolchain from the forgectrl recipe workdir (regenerate with `bitbake forgectrl` after a clean). | | `accel_fast.py` | Direct-I2C sampler for the two head-bus LIS2HH12s (runs on the board; unbinds/rebinds st-accel around the capture, 800 Hz ODR, ~270 Hz per device polled): optional mid-capture jogs via local grblHAL TCP. CSV to /tmp/accel.csv. The head accel is i2c-3 0x1e. | | `bump_seek.py` | Accelerometer bump-seek homing prototype (runs on the board): creeps toward a rail in bounded jog segments via grblHAL TCP, learns the moving-noise baseline per segment, detects the contact jolt (~530 Hz sampling, 2-sample confirm), jog-cancels (0x85) and backs off. CSV to /tmp/bump.csv. | @@ -26,7 +39,48 @@ target board (dev image, python3 present) unless noted. | `puls_profile.py` | Decodes factory `.puls` streams (raw or GF1-headered) into velocity/accel profiles: peak speeds, ramp-slope fits, per-move segments, Z cadence. Runs anywhere (stdlib only). Source of the factory-true grblHAL defaults: 700/590 mm/s² accel, 200 mm/s max rate, 28160 Hz travel tick. | | `bench_m2.py` | Motion-quality bench, runs against the board over TCP:23: bounded round-trip jogs (sanity, max-rate, diagonal) + feed-hold/resume mid-move, reporting peak feed, state transitions, and position drift. | -The build scripts borrow the Yocto cross toolchain + sysroot from the ulfius -2.7.15 work directory in the WSL build tree; if that path ages out after a -`bitbake -c clean`, point `TC` at any current target recipe workdir (or build -a proper SDK with `bitbake meta-toolchain`). +Data files kept beside the tools: `flow_matrix_results.json` / +`flow_matrix_log.txt` (the 60-run flow-detection matrix), +`flow_warm_results.json` / `flow_warm_log.txt` (warm-baseline validation), +`temp_calibration.json` (coolant sensor spot-checks), and +`lid_ir_ambient_baseline.csv` (lid-IR ambient, lid closed, idle). + +The build scripts borrow the Yocto cross toolchain + sysroot from a target +recipe work directory in the build tree (`FF_BUILD_TOP`, default +`../forgefirm/build`); if that path ages out after a `bitbake -c clean`, +rebuild the named recipe or point `TC` at any current target recipe workdir +(or build a proper SDK with `bitbake meta-toolchain`). + +## How the coolant-flow fire-gate threshold was derived + +forgectrl's cooling engine refuses to let the laser fire unless a +heater-based flow check passes: with the pump commanded on, the loop heater +runs at `COOL_FLOW_HEATER_PCT` (40 %) for `COOL_FLOW_CHECK_S` (50 s) and the +downstream sensor's rise over its settled baseline must stay below +`COOL_FLOW_RISE_C` (14.4 °C) — a stopped pump lets the heater's output pool +at the downstream sensor instead of being carried away. The three constants +live in `forgectrl/src/cool.h` and are the compiled defaults behind the +`cool_flow_*` settings. + +The 14.4 °C threshold and the 40 %/50 s operating point come from +`flow_matrix.py`: 6 duties × 2 flow states × 5 interleaved repeats = 60 +heating runs, every run started from a common cooled baseline, both sensors +sampled at 1 Hz by `flow_sampler.py`, and one heating trace scored at every +candidate check duration. The committed `flow_matrix_results.json` and +`flow_matrix_log.txt` are that data set. The selection rule: the cheapest +duty at which every observed no-flow rise exceeded every observed flow rise +with a comfortable d′ (40 % / 50 s: flow ≤ 12.75 °C over 17 observations, +no-flow ≥ 16.04 °C over 8, d′ 8.4, ~0.8 °C of loop heating per check), with +the threshold set at the balanced midpoint of the two bands (14.4 °C). +`flow_warm_validate.py` then re-ran the real check from heater-warmed +baselines (`flow_warm_results.json`). + +To reproduce on another machine: set `GF_HOST` (and `GF_SSH` if ssh needs a +wrapper), copy `flow_sampler.py` to `/data/` on the board, and run +`flow_matrix.py` (env `FM_DUTIES`, `FM_REPEATS`, `FM_RESULTS`; ~1.6 h for +the full matrix, the controller is stopped for the duration). The same +derivation is also built into forgectrl as the panel's Diagnostics → +**flow-calibrate** tool (3 trials per case at the operating point, reports +both bands and a recommended threshold; the bench value it recommends +lands within a few tenths of a degree of 14.4). Apply a per-machine value +through the `cool_flow_rise` setting rather than editing the constant. diff --git a/scripts/bench/fan_test.py b/scripts/bench/fan_test.py index 6d858a7..1213698 100644 --- a/scripts/bench/fan_test.py +++ b/scripts/bench/fan_test.py @@ -1,14 +1,16 @@ -import socket, subprocess, time +import shlex, socket, subprocess, time import os HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) def board(cmd): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=30) return r.stdout.strip().replace('\n', ' ') diff --git a/scripts/bench/flow_characterize.py b/scripts/bench/flow_characterize.py index c7cc9b1..5cd8455 100644 --- a/scripts/bench/flow_characterize.py +++ b/scripts/bench/flow_characterize.py @@ -16,6 +16,7 @@ leaves this alone. """ import math import os +import shlex import subprocess import sys import time @@ -23,6 +24,9 @@ import time HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) # Factory B-equation conversion (see kernel-module-glowforge/UAPI.md). F = 1024.0 * 1.3 @@ -39,9 +43,8 @@ def degc(raw): def board(cmd): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=30) return r.stdout.strip() diff --git a/scripts/bench/flow_matrix.py b/scripts/bench/flow_matrix.py index e05c8fa..6b2a048 100644 --- a/scripts/bench/flow_matrix.py +++ b/scripts/bench/flow_matrix.py @@ -30,6 +30,7 @@ are still usable) and a summary table at the end. import json import math import os +import shlex import statistics import subprocess import sys @@ -38,6 +39,9 @@ import time HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) HERE = os.path.dirname(os.path.abspath(__file__)) RESULTS = os.path.join(HERE, os.environ.get('FM_RESULTS', 'flow_matrix_results.json')) @@ -69,9 +73,8 @@ def degc(raw): def board(cmd, timeout=120): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=timeout) return r.stdout diff --git a/scripts/bench/flow_recheck_char.py b/scripts/bench/flow_recheck_char.py index e80c7c2..07a110b 100644 --- a/scripts/bench/flow_recheck_char.py +++ b/scripts/bench/flow_recheck_char.py @@ -38,6 +38,7 @@ prints the differential separation. Aborts if downstream passes 45 C. """ import math import os +import shlex import subprocess import sys import time @@ -45,6 +46,9 @@ import time HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) F = 1024.0 * 1.3 RD, BETA = 10000.0, 3380.0 RINF = 10000.0 * math.exp(-3380.0 / 298.15) @@ -57,9 +61,8 @@ def degc(raw): def board(cmd): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=30) return r.stdout.strip() diff --git a/scripts/bench/flow_sustained.py b/scripts/bench/flow_sustained.py index 0cb2720..1936038 100644 --- a/scripts/bench/flow_sustained.py +++ b/scripts/bench/flow_sustained.py @@ -10,6 +10,7 @@ Usage: flow_sustained.py [minutes] (default 30) """ import math import os +import shlex import re import socket import subprocess @@ -19,6 +20,9 @@ import time HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) F = 1024.0 * 1.3 RD, BETA = 10000.0, 3380.0 RINF = 10000.0 * math.exp(-3380.0 / 298.15) @@ -30,9 +34,8 @@ def degc(raw): def board(cmd): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=30) return r.stdout.strip() diff --git a/scripts/bench/flow_warm_validate.py b/scripts/bench/flow_warm_validate.py index c8337fb..7a11cfc 100644 --- a/scripts/bench/flow_warm_validate.py +++ b/scripts/bench/flow_warm_validate.py @@ -15,6 +15,7 @@ Usage: flow_warm_validate.py [cycles_per_case] (default 3) import json import math import os +import shlex import statistics import subprocess import sys @@ -23,6 +24,9 @@ import time HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) HERE = os.path.dirname(os.path.abspath(__file__)) RESULTS = os.path.join(HERE, 'flow_warm_results.json') @@ -52,9 +56,8 @@ def degc(raw): def board(cmd, timeout=120): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=timeout) return r.stdout diff --git a/scripts/bench/temp_calibrate.py b/scripts/bench/temp_calibrate.py index 9866d5f..7cffe90 100644 --- a/scripts/bench/temp_calibrate.py +++ b/scripts/bench/temp_calibrate.py @@ -21,6 +21,7 @@ heater on). import json import math import os +import shlex import subprocess import sys import time @@ -28,6 +29,9 @@ import time HOST = os.environ.get('GF_HOST') if not HOST: raise SystemExit('set GF_HOST to the machine IP address') +# ssh client used to reach the board; override for a wrapper, e.g. +# GF_SSH='wsl -d -- ssh'. +SSH = shlex.split(os.environ.get('GF_SSH', 'ssh')) STORE = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'temp_calibration.json') @@ -42,9 +46,8 @@ def uapi_c(raw): def board(cmd): - r = subprocess.run(['wsl', '-d', 'forge-yocto', '--', 'ssh', - '-o', 'PreferredAuthentications=none', - 'root@' + HOST, cmd], + r = subprocess.run(SSH + ['-o', 'PreferredAuthentications=none', + 'root@' + HOST, cmd], capture_output=True, text=True, timeout=30) return r.stdout.strip()