diff --git a/docs/BRINGUP.md b/docs/BRINGUP.md index 888ab1b..5222f50 100644 --- a/docs/BRINGUP.md +++ b/docs/BRINGUP.md @@ -1358,12 +1358,20 @@ Open items only. Anything closed is in `CAMPAIGN-LOG.md`. and unarmed whenever it is commanded at the run duty; a cloud hunt, sent with the extraction fans off, is measured and not judged. `cooling.fan-gate-trips` and the hunt leg of `cloud.mode-switch` in - the catalog. The shipped floors are 55 percent of the bench machine's - measured steady speeds (facts bank: exhaust 6400, intake 2290, air - assist 6000 rpm, purge current 300, grace 15 s); the bench run of the - gate tests on an image carrying them, and the unplugged-tach drill, - are still owed. The coolant critical tier and the watch-only board - temperatures follow. + the catalog, both PASS on the pinned dev image `20260822135848` + (campaign 18 of 18). The shipped floors are 55 percent of the bench + machine's measured steady speeds (facts bank: exhaust 6400, intake + 2290, air assist 6000 rpm, purge current 300, grace 15 s). The + unplugged-exhaust-fan drill PASS on the same image: `AIRFLOW` at + grace plus three ticks with the exhaust dead, the other fans held, + the reason relayed on the Grbl port, and the replugged fan `ok` inside + the next session's grace (CAMPAIGN-LOG). From the drill: the fault + ends with its run session (decided; a standing hold at idle canceled + GRBL jogs and would have refused the cloud print that re-proves the + fan), since every session judges every fan afresh after the grace; + `cooling.fan-gate-trips` checks it, bench run owed on the next image. + The coolant critical tier and the watch-only board temperatures + follow. Nothing here can put energy where it was not commanded: the hardware chain is the emission boundary and no header field touches it, and forgectrl runs diff --git a/docs/CAMPAIGN-LOG.md b/docs/CAMPAIGN-LOG.md index c37925e..b9b37af 100644 --- a/docs/CAMPAIGN-LOG.md +++ b/docs/CAMPAIGN-LOG.md @@ -3308,6 +3308,63 @@ purge in the purge leg, the exhaust read `off` with floor 0 in the off leg, and the restore showed every fan `ok` at the shipped floors (exhaust 11723, intakes 4157 and 4162, air assist 11078 rpm, purge 628 counts). +## 2026-08-22: the measured floors and the operating-point rule on a pinned image + +Dev image `20260822135848` (forgectrl 47e4256 pinned by forgefirm adcd1ad; +release `20260822135751` built alongside), flashed after a fetch-verified +both-image build. Campaign `c-20260822140659-2f25`, every auto test the +pin bump invalidated plus the rest of the non-operator, non-live catalog: +**18 of 18 PASS** (cooling.flow-verify, image.health, kernel.latch-locked-idle, +kernel.k1-k2, kernel.backtrack-bounds, forgectrl.auth, +forgectrl.settings-bounds, forgectrl.panel-serves, logs.tree-tail-export, +logs.level-settings, motion.deadman, cooling.fans-quiet-after-motion, +cooling.gate-off, cooling.fan-gate-trips, camera.sensor-profile, +camera.frame-health, cloud.mode-switch, kernel.fire-line). + +The two that carry this change, as recorded: `cooling.fan-gate-trips` in +58 s with only the exhaust `TRIPPED` in its leg, only the purge in its, +the exhaust `off` at floor 0 in the same tick `gates_off` named it, and +the restore reading exhaust 11726, intakes 4160 and 4193, air assist +11095 rpm, purge 627 counts, every gate `ok` at the shipped floors; +`cloud.mode-switch` in 29 s with the connect-time hunt's run tick reading +the exhaust at 0 rpm, `unjudged`, the air assist `unjudged`, verdict `OK` +throughout, and the hunt finishing `:completed`. The hunt's run phase is a +few seconds long and gave one sample at a 1 s poll, so the watcher now +samples twice a second. + +## 2026-08-22: the unplugged-exhaust-fan drill + +The gate on the real failure path, not a settings override: the operator +unplugged the exhaust fan's whole connector at the Interconnect PCB (fan +dead, tach silent), the machine idle in GRBL mode, lid closed, nothing +armed. One `M8` session from the board, `/cool/status` read once a second +(dev image `20260822135848`): + +- 1 to 14 s: every gate `grace` (the shipped 15 s), exhaust reading 0. +- 15 and 16 s: exhaust `under` at 0 rpm; intakes, air assist and purge + `ok`, up to speed inside the grace. +- 17 s: verdict **`AIRFLOW`**, `fire_ok false`, `hold true`, exhaust + `TRIPPED`, reason `AIRFLOW: exhaust 0 under the 6400 floor for 3 s - + hold, no resume this job`; the other four fans held at run duty around + the dead one. grblHAL relayed the reason on the Grbl port as a + `[MSG:Warning: ...]`, and forgectrl logged the `WARNING` line. +- `M9` ended the session into the smoke-clear phase with the fault still + named; the operator replugged the connector. +- The next `M8` session: exhaust 4112 rpm at 1 s, 6623 at 2 s (past the + floor), 11640 at 7 s (the measured time to 90 percent), `ok` with every + gate at the end of the grace, verdict `OK`, clean end. + +One observation for a decision: between the two sessions the engine sat +at idle with the verdict still `AIRFLOW`, `hold=true`, `fire_ok=false`. +The fan fault latches for the run session and clears at the next session +start, the same shape as the fire alarm; at idle the hold cancels GRBL +jogs and the cloud client's print pre-check refuses a print before a +session could re-prove the fan (a hunt clears it). Decision (operator, +the same day): the fault ends with its session, since every session +judges every fan afresh after the grace before anything can fire; +`cooling.fan-gate-trips` now checks the verdict is `OK` with no hold once +the tripped session is over. The fire alarm keeps its idle hold. + ## Superseded status notes ### Shared machine services — remaining polish, as listed 2026-08-13 diff --git a/docs/COOLING.md b/docs/COOLING.md index 07626b6..e9c7390 100644 --- a/docs/COOLING.md +++ b/docs/COOLING.md @@ -170,7 +170,10 @@ floor. session: a fan that has stopped moving air is not a condition to cut through. The fans stay at run duty (a stalled extraction fan needs every other fan around it running), and the reason names the fan, the reading - and the floor. The next job starts the gates fresh. + and the floor. The fault ends with the session: at idle the verdict is + `OK` again (a standing hold would cancel jogs and refuse the next job + before it could re-prove the fan), and the next session judges every fan + afresh after the grace. - **A floor of zero is that gate off** (§8a). It still measures: the first reading in a job that would have tripped the shipped default is logged. diff --git a/forgetest/forgetest/suite/cloud.py b/forgetest/forgetest/suite/cloud.py index 1d14ff8..ecd8aa3 100644 --- a/forgetest/forgetest/suite/cloud.py +++ b/forgetest/forgetest/suite/cloud.py @@ -372,8 +372,8 @@ def wait_action_finished(ctx, offset, action, timeout, poll=0.5): def watch_hunt_gates(ctx, offset, timeout): """Wait for the action's terminal line like wait_action_finished, - sampling /cool/status once a second meanwhile. Returns (line or - None, samples).""" + sampling /cool/status twice a second meanwhile (a hunt's run phase + can be a few seconds). Returns (line or None, samples).""" fc = ctx.forgectrl samples = [] t0 = time.time() @@ -386,7 +386,7 @@ def watch_hunt_gates(ctx, offset, timeout): i = action_finish_index(lines, "hunt") if i is not None: return lines[i], samples - time.sleep(1) + time.sleep(0.5) return None, samples diff --git a/forgetest/forgetest/suite/cooling.py b/forgetest/forgetest/suite/cooling.py index e9d4afe..20ebfc4 100644 --- a/forgetest/forgetest/suite/cooling.py +++ b/forgetest/forgetest/suite/cooling.py @@ -10,7 +10,7 @@ import time from ..catalog import test from .. import hw -_COOL_COVERS = [("forgectrl", "src/cool.*"), ("forgectrl", "src/diag.*"), +_COOL_COVERS = [("forgectrl", "src/cool.*"), ("forgectrl", "src/coolfmt.*"), ("forgectrl", "src/diag.*"), ("forgectrl", "src/gates.*"), ("forgectrl", "src/airflow.*"), ("forgectrl", "src/settings.*"), ("forgectrl", "src/status.*"), ("forgectrl", "src/ui/**"), @@ -239,6 +239,18 @@ def _run_session(ctx, g, fc, until, what, wait=None): _session_ended(ctx, fc, what) +def _after_session(ctx, fc, wait=5): + """The engine's state a few ticks after a session ended.""" + c = {} + t0 = time.time() + while time.time() - t0 < wait: + ctx.sleep(1) + c = _cool(fc) + if c.get("phase") != "run" and c.get("verdict") == "OK": + break + return c + + def _tail_has(fc, needle): st, body = fc.get("/logs/tail", params={"name": "forgectrl", "lines": GATE_LOG_LINES}) text = body.get("text", "") if st == 200 and isinstance(body, dict) else "" @@ -358,9 +370,10 @@ def _fan_gate(c, name): description="The airflow gates judge a fan commanded at the cut fan profile, which a bare M8 " "applies, armed or not. An exhaust floor no fan " "can meet must trip AIRFLOW after the grace plus three ticks (hold, fire blocked, no " - "resume while the session lasts); a purge current floor at the ADC rail must trip the " - "same way; a floor of zero must read off in gates_off and trip nothing; restored, the " - "next session runs OK with every fan reading at or above its floor.") + "resume while the session lasts) and the fault must end with the session (verdict OK, " + "no hold, once the session is over); a purge current floor at the ADC rail must trip " + "the same way; a floor of zero must read off in gates_off and trip nothing; restored, " + "the next session runs OK with every fan reading at or above its floor.") def fan_gate_trips(ctx): fc = ctx.forgectrl ev = ctx.evidence @@ -397,6 +410,13 @@ def fan_gate_trips(ctx): ctx.check(_fan_gate(c, "exhaust").get("state") == "TRIPPED", "the exhaust gate does not read TRIPPED: %s", c.get("fan_gates")) ctx.check("exhaust" in (c.get("reason") or ""), "the reason does not name the fan: %r", c.get("reason")) + # The fault is the session's: with the session over, the + # verdict is back to OK and nothing holds (jogs and the next + # job's pre-check must not see a fan fault at idle). + c = _after_session(ctx, fc) + ev["exhaust_trip_after"] = c + ctx.check(c.get("verdict") == "OK" and c.get("hold") is False, + "the fan fault outlived its run session: %s", c) # Leg 2: the purge fan by current, floor at the ADC rail. _set_gates(ctx, fc, {"cool_tach_exhaust_min_rpm": orig["cool_tach_exhaust_min_rpm"], diff --git a/forgetest/tests/test_cooling_suite.py b/forgetest/tests/test_cooling_suite.py index 4959e90..4c083f1 100644 --- a/forgetest/tests/test_cooling_suite.py +++ b/forgetest/tests/test_cooling_suite.py @@ -448,6 +448,10 @@ class FanGateTests(unittest.TestCase): def end(): time.sleep(0.3) cool["phase"] = "idle" + # the fan fault is the session's: it ends with it + cool.update(verdict="OK", fire_ok=True, hold=False, reason="") + for g in cool.get("fan_gates", {}).values(): + g["state"] = "idle" threading.Thread(target=end, daemon=True).start() return if line != "M8": diff --git a/meta-forgefirm/recipes-forgefirm/forgectrl/forgectrl-pin.inc b/meta-forgefirm/recipes-forgefirm/forgectrl/forgectrl-pin.inc index 2ecd70a..050d7fe 100644 --- a/meta-forgefirm/recipes-forgefirm/forgectrl/forgectrl-pin.inc +++ b/meta-forgefirm/recipes-forgefirm/forgectrl/forgectrl-pin.inc @@ -2,5 +2,5 @@ # only SRCREV and PV here - the image manifest leaves *-pin.inc out of the # layer content hash because the component entry already identifies the # pinned source (forgefirm-image-manifest.bbclass). -SRCREV = "47e42569e8def2ed6fe10f61e9ab9ff4f9c8807d" +SRCREV = "d51dbdbcc6b3a12d2f7963eb94514ce52493645f" PV = "0.1.0"