mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-09-27 08:41:13 -07:00
The audit's deferred findings: the checksum exclude, the resume-lead drill, the record
B-16: BB_SIGNATURE_LOCAL_DIRS_EXCLUDE in the distro conf names __pycache__ and .pytest_cache, so a workstation's bytecode caches never enter a file:// checksum (proven in the build VM: a cache under the package leaves the fetch task alone, a source change reruns it). kernel.resume-lead: two phases behind one takeover at a 1 kHz tick. E, a resume whose lead is longer than the data ends at end-of-data within a few ticks (a lost end-of-data would show as 255 ms). L, a 1000-byte lead over FIRE bits with the latch unlocked and the chain unarmed keeps the FIRE line low through the lead and drives it from the waypoint byte on. The bench proof for the module's K-4 and K-8 fixes; the catalog counts 55 tests, 0 uncovered. BRINGUP items 9 and 11 and the CAMPAIGN-LOG entry record the batch and the campaign rule: no campaign until every audit finding is on one image.
This commit is contained in:
ScottW514
+34
-32
@@ -1371,8 +1371,13 @@ feature requests, enhancements) will eventually be tracked as GitHub issues.
|
||||
factory recovery after the 60 s watchdog with a power cycle returning, as
|
||||
documented. Image 20260902230436 (release and dev) is built on the pushed
|
||||
pins, the kernel and the module rebuilt together for the watchdog sysfs
|
||||
change. Owed now: flash the dev image, take a fresh-boot baseline, and run
|
||||
the full campaign on it, the campaign the release gate asks for.
|
||||
change, and is not the image the campaign runs on: the campaign waits
|
||||
until every audit finding, the deferred ones included, is on one image
|
||||
(the operator's rule). That batch is done (item 11) and rides the next
|
||||
local image, which is the campaign's. Owed now: build it, flash the dev
|
||||
image, take a fresh-boot baseline, run the full campaign on it, the
|
||||
campaign the release gate asks for; then the push in CI order and the
|
||||
pin bumps for the batch.
|
||||
Decisions taken in the remediation that the operator confirms or reverses:
|
||||
the release image has no shell login (the install page now says so); the
|
||||
cloud client holds and resumes on the cooling verdict with a 30-minute
|
||||
@@ -1390,36 +1395,33 @@ feature requests, enhancements) will eventually be tracked as GitHub issues.
|
||||
change: it rides the next image flash, with the coolant-reading tests
|
||||
(`cooling.aa-offset-calibrate`, `cooling.flow-verify`) as its proof.
|
||||
|
||||
11. **The audit's deferred findings.** Six findings of the 2026-09-01 audit were
|
||||
deferred with a reason and are owed here. Four are one kernel batch, since
|
||||
each edits the SDMA script, the module, or a kernel patch file, and a
|
||||
kernel change rides one image flash: **K-4** (end-of-data reached before
|
||||
an armed waypoint is decoded as the waypoint; the script clears its
|
||||
scratch at `alldone` and publishes an end-of-data flag the host decodes on;
|
||||
proof: a cloud pause within the last bytes of a job, a `cloud.pause-resume`
|
||||
case), **K-8** (the FIRE restore at the resume waypoint writes the whole
|
||||
GPIO2 data register from gpiolib's shadow while the script may be
|
||||
mid-byte; move the restore into the script on a scratch flag; a scope on
|
||||
DIR at the resume point first, since the failure is plausible, not seen),
|
||||
**P-13** (the SDMA glue patch hands out a channel without claiming it from
|
||||
dmaengine and re-initializes a tasklet that may be scheduled; claim the
|
||||
channel in `sdma_get_channel()`, release it in the put, `tasklet_kill()`
|
||||
before clearing; latent today, no other client reaches the channel), and
|
||||
**P-14** (the SPI patch takes its inter-word wait states from
|
||||
`spi_transfer.delay`, which the core also applies after the transfer;
|
||||
read `word_delay` and have `pic.c` set it; batch with item 10, since the
|
||||
PIC read pattern measured on 2026-09-02 may turn on these wait states).
|
||||
The patch-file edits are regenerated with `devtool modify linux-fslc` on
|
||||
the pinned tree. Two are host-only: **B-16** (a host `__pycache__` under the
|
||||
forgetest package directory moves the recipe's task hash without a source
|
||||
change; the remediation named the package directory and drops the caches
|
||||
in `do_unpack`, but the fetch checksum is taken before that, so the
|
||||
residual stays: run the host tests with `PYTHONDONTWRITEBYTECODE=1` or
|
||||
clear the caches before a build, since the file fetcher has no exclude);
|
||||
and **FA-20** (the panel's devserver mock diverged from the daemon:
|
||||
mode values, `/cool/status`, the gate table, the settings keys, the diag
|
||||
routes; regenerate its tables from `gates.c` and the settings table), dev
|
||||
tooling with no image consequence.
|
||||
11. **The audit's deferred findings, done and waiting for the image.** All six
|
||||
are fixed and host-proven, and ride the local image item 9's campaign
|
||||
runs on. **K-4**: the SDMA script publishes end-of-data in a coherent
|
||||
mailbox word before it signals (and clears the waypoint counter), and the
|
||||
interrupt callback decodes on the mailbox, not on the host's arming, so an
|
||||
end-of-data that arrives before an armed waypoint stops the run at once.
|
||||
**K-8**: a resume's laser-off lead is the script's own inhibit mask,
|
||||
applied to every GPIO word it writes and cleared at the waypoint byte;
|
||||
run start restores the FIRE drive while the script is idle, the callback
|
||||
writes nothing to the GPIO data register, and only the deceleration parks
|
||||
the line. Both are proven on the host by the module's -Werror cross-build
|
||||
and on the bench by the new `kernel.resume-lead` drill (end-of-data before
|
||||
the waypoint at a 1 kHz tick, and a 1000-byte lead over FIRE bits with the
|
||||
latch unlocked and the chain unarmed). **P-13**: the SDMA channel is
|
||||
claimed through dmaengine (`dma_get_slave_channel`, released by
|
||||
`dma_release_channel`, whose resource hooks hold the engine's clocks), and
|
||||
the callback setter kills the tasklet before clearing and initializes it
|
||||
only when setting. **P-14**: the inter-word wait states come from
|
||||
`spi_transfer.word_delay`, which `pic.c` sets beside the post-transfer
|
||||
`delay`. The kernel rebuilt clean with both patches. **B-16**:
|
||||
`BB_SIGNATURE_LOCAL_DIRS_EXCLUDE` in the distro conf names `__pycache__`
|
||||
and `.pytest_cache`, so a workstation's bytecode caches never enter a
|
||||
file:// checksum (proven in the build VM: a cache under the package leaves
|
||||
the fetch task alone, a source change reruns it). **FA-20**: the panel's
|
||||
dev-server mock carries the daemon's tables and reply shapes, and a host
|
||||
test in the forgectrl repository reads them out of the C sources and holds
|
||||
the mock to them. The item closes with the campaign.
|
||||
|
||||
**Deliberately not gated:** an armed GRBL job after an underrun cuts at the
|
||||
stale origin unless homing is required (GRBL mode permits unhomed cutting; the
|
||||
|
||||
@@ -7346,6 +7346,68 @@ Owed: the operator flashes the dev image, takes a fresh-boot baseline, and
|
||||
runs the full campaign (all three layer hashes moved), the campaign the
|
||||
release gate asks for.
|
||||
|
||||
## 2026-09-02: the audit's deferred findings, host-proven
|
||||
|
||||
The operator's rule for the campaign set the order: no campaign until every
|
||||
audit finding, the deferred six included, is on one image for a final test.
|
||||
So the deferred batch was done at once, all of it local, all of it host-proven.
|
||||
|
||||
**K-4 and K-8, in the SDMA script and the module.** The waypoint and the
|
||||
end-of-data interrupts share one line, and the callback cannot fetch the
|
||||
channel context (a channel-0 transfer, which sleeps), so it decoded on
|
||||
host-side arming: an end-of-data before an armed waypoint read as the
|
||||
waypoint. The script now writes 1 to a coherent mailbox word (allocated
|
||||
before the dedicated pool is attached, so the pool stays the ring's alone;
|
||||
its physical address rides a reserved context word) before its end-of-data
|
||||
notify, clears the waypoint counter, and the callback decodes on the
|
||||
mailbox; run start clears it. The resume lead moved into the script too: a
|
||||
laser inhibit mask in a second reserved context word, ANDed out of every
|
||||
GPIO word beside the motor lock, set by run start for an accelerating
|
||||
forward run and cleared by the script at the waypoint byte. Run start
|
||||
restores the FIRE drive while the script is idle, the callback writes
|
||||
nothing to the GPIO data register, and only the deceleration parks the line
|
||||
(the direction register alone). A resume with no lead plays laser-less for
|
||||
the whole run, as before, now by construction. The script grew from 160 to
|
||||
173 instructions; its branches are 8-bit displacements and the growth pushed
|
||||
one past the range, which the assembler refuses, so the layout changed: the
|
||||
waypoint action, the power-level path and the first end-of-data tick sit
|
||||
past the main loop, reached by short branches; the worst displacement is
|
||||
+120 of 127. Host proof: the module cross-compiles against the pinned
|
||||
kernel with -Werror, its host tests pass. The bench proof is the new
|
||||
`kernel.resume-lead` drill, two phases behind one takeover at a 1 kHz tick:
|
||||
E, a resume whose lead is longer than the data, where a lost end-of-data
|
||||
would show as 255 ms; L, a 1000-byte lead over FIRE bits with the latch
|
||||
unlocked and the chain unarmed, the FIRE line sampled through the run. The
|
||||
catalog counts 55 tests, 0 uncovered.
|
||||
|
||||
**P-13 and P-14, in the kernel patches.** `sdma_get_channel()` claims the
|
||||
channel through `dma_get_slave_channel()`, whose resource hook holds the
|
||||
engine's ipg/ahb clocks the way the explicit enables did, and
|
||||
`sdma_put_channel()` is `dma_release_channel()`; the callback setter kills
|
||||
the tasklet before clearing and initializes it only when setting. The SPI
|
||||
patch reads `spi_transfer.word_delay` for the PERIODREG wait states, and
|
||||
`pic.c` sets `word_delay` beside `delay` (the post-transfer gap the factory
|
||||
kernel also had). The hunks were edited in place; do_patch took them and
|
||||
the kernel rebuilt clean.
|
||||
|
||||
**B-16.** The real fix is bitbake's own knob: `BB_SIGNATURE_LOCAL_DIRS_EXCLUDE`
|
||||
in the distro conf names `__pycache__` and `.pytest_cache` beside the VCS
|
||||
directories, so the file fetcher's checksum never sees a bytecode cache.
|
||||
The experiment in the build VM: a fetch on a clean tree ran the task; a
|
||||
cache injected under the package directory left 2 of 2 tasks alone; a real
|
||||
source change ran the task again. The CI's `-B` stays as a second layer.
|
||||
|
||||
**FA-20.** The panel's dev-server mock carried 10 gate rows of the daemon's
|
||||
22, lacked 18 settings keys, accepted a mode the daemon refuses, and sent
|
||||
`/cool/status`, `/diag`, `/boot`, `/update` and `/slots` shapes the daemon
|
||||
does not. It now carries the daemon's tables and reply shapes, and a host
|
||||
test (`tests/test_devserver_mock.py`, 15 cases, one CI step) parses the C
|
||||
tables and holds the mock to them.
|
||||
|
||||
Owed: one local image with the batch (the module pinned from its local
|
||||
commit, the rest from the pushed pins), the flash, the fresh-boot baseline
|
||||
and the full campaign; then the push in CI order and the pin bumps.
|
||||
|
||||
## Reference notes
|
||||
|
||||
### Head-IRQ source validation — the beam-emission hypothesis
|
||||
|
||||
@@ -679,3 +679,130 @@ def fire_line(ctx):
|
||||
except OSError:
|
||||
pass
|
||||
ctx.log("safe state restored: state=%s latch=LOCKED", rd("cnc/state"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- resume lead
|
||||
|
||||
@test("kernel.resume-lead", title="A resume's laser-off lead, and an end-of-data before the waypoint",
|
||||
subsystem="kernel", kind="auto", hardware="takeover", est_min=2,
|
||||
covers=_KERNEL_COVERS,
|
||||
requires=["kernel.k1-k2"], precheck=hv_off,
|
||||
steps=["Phase L unlocks the latch with a zero-duty stream, so the drill starts only while "
|
||||
"the HV supply does not report good (true at idle; if it is refused, open the lid - "
|
||||
"the safety chain holds HV off - and start it again)."],
|
||||
description="Two phases behind one takeover, zero duty, motors locked, a 1 kHz tick. E: a "
|
||||
"resume whose lead is longer than the data (the ring drains before the waypoint) "
|
||||
"ends the run at end-of-data within a few ticks, not one re-notify period (255 "
|
||||
"ticks) later: the script publishes end-of-data in its mailbox and the host "
|
||||
"decodes on that, not on the waypoint it was still waiting for. L: with the "
|
||||
"latch unlocked and the chain unarmed, a resume with a 1000-byte lead over a "
|
||||
"stream of FIRE bits keeps the FIRE line low through the lead and drives it from "
|
||||
"the waypoint byte on. The script masks the laser bits itself, so no host write "
|
||||
"reaches the GPIO data register while the script runs; LASER_ON stays off (the "
|
||||
"safety AND-gate holds).")
|
||||
def resume_lead(ctx):
|
||||
ev = ctx.evidence
|
||||
require_hv_off(ctx)
|
||||
tick = 1000
|
||||
with ctx.takeover():
|
||||
try:
|
||||
# ---- E: end-of-data before the waypoint
|
||||
stream = POWER0 + PAD * (tick // 2) # 0.5 s of pads
|
||||
expected = len(stream) / tick
|
||||
ctx.log("E: %d bytes = %.1f s of pads at %d Hz; resume with a %d-byte lead (never reached)",
|
||||
len(stream), expected, tick, 10 * tick)
|
||||
wr("cnc/motor_lock", 15)
|
||||
wr("cnc/laser_latch", 1)
|
||||
wr("cnc/ramp_rate", 125000)
|
||||
wr("cnc/step_freq", tick)
|
||||
snap(ctx, "E pre")
|
||||
with PulseDevice(ctx) as dev:
|
||||
dev.rewind()
|
||||
wr("cnc/enable", 1)
|
||||
ctx.sleep(0.5)
|
||||
dev.write(stream)
|
||||
underruns_before = int(rd("cnc/underruns"))
|
||||
t0 = time.time()
|
||||
wr("cnc/resume", 10 * tick)
|
||||
state = wait_state(ctx, "running", 2, poll=0.002)
|
||||
ctx.check(state == "running", "E: the resume did not start (state=%s)", state)
|
||||
state = wait_state(ctx, "idle", 5, poll=0.002)
|
||||
dt = time.time() - t0
|
||||
ev["E"] = {"run_to_idle_s": round(dt, 3), "data_s": expected, "state": state,
|
||||
"underruns_before": underruns_before,
|
||||
"underruns_after": int(rd("cnc/underruns")), "faults": rd("cnc/faults")}
|
||||
ctx.log("E: %s after %.3f s (the data is %.3f s; a lost end-of-data would add %.3f s)",
|
||||
state, dt, expected, 255.0 / tick)
|
||||
ctx.check(state == "idle", "E: state %s after the data ran out", state)
|
||||
ctx.check(dt < expected + 0.15,
|
||||
"E: the run ended %.3f s after its data: end-of-data decoded late", dt - expected)
|
||||
ctx.check(ev["E"]["underruns_after"] == underruns_before,
|
||||
"E: an underrun was counted on a normal completion")
|
||||
ctx.check(ev["E"]["faults"] == "0", "E: faults=%s", ev["E"]["faults"])
|
||||
ctx.log("E PASS: end-of-data before the waypoint ended the run on time")
|
||||
|
||||
# ---- L: the lead, latch unlocked, chain unarmed
|
||||
check_hv_off(ctx)
|
||||
lead = tick # 1.0 s of FIRE bits held off
|
||||
lead_s = lead / tick
|
||||
stream = POWER0 + FIRE * (2 * tick) + PAD * (tick // 5)
|
||||
ctx.log("L: %d bytes = %.1f s (%.1f s of FIRE bits then pads); lead %d bytes = %.1f s; "
|
||||
"latch UNLOCKED, chain unarmed", len(stream), len(stream) / tick,
|
||||
2 * tick / tick, lead, lead_s)
|
||||
wr("cnc/motor_lock", 15)
|
||||
wr("cnc/laser_latch", 1)
|
||||
wr("cnc/step_freq", tick)
|
||||
snap(ctx, "L pre")
|
||||
samples = []
|
||||
state = ""
|
||||
with PulseDevice(ctx) as dev:
|
||||
dev.rewind()
|
||||
wr("cnc/enable", 1)
|
||||
ctx.sleep(0.5)
|
||||
dev.write(stream)
|
||||
try:
|
||||
wr("cnc/laser_latch", 0)
|
||||
t0 = time.time()
|
||||
wr("cnc/resume", lead)
|
||||
n = 0
|
||||
while time.time() - t0 < 10:
|
||||
en, on = rd("cnc/laser_enable"), rd("cnc/laser_on")
|
||||
samples.append((round(time.time() - t0, 4), en, on))
|
||||
state = rd("cnc/state")
|
||||
if state != "running" and time.time() - t0 > 0.2:
|
||||
break
|
||||
n += 1
|
||||
if n % 200 == 0:
|
||||
ctx.checkpoint()
|
||||
finally:
|
||||
wr("cnc/laser_latch", 1)
|
||||
snap(ctx, "L post")
|
||||
driven = [s for s in samples if s[1] != "0"]
|
||||
first = driven[0][0] if driven else None
|
||||
last = driven[-1][0] if driven else None
|
||||
inside_lead = [s for s in driven if s[0] < lead_s - 0.02]
|
||||
ev["L"] = {"samples": len(samples), "driven_samples": len(driven),
|
||||
"first_driven_s": first, "last_driven_s": last, "end_state": state,
|
||||
"any_laser_on": any(s[2] != "0" for s in samples),
|
||||
"inside_lead": inside_lead[:5], "laser_on_sampled": rd("cnc/laser_on_sampled"),
|
||||
"faults": rd("cnc/faults")}
|
||||
ctx.log("L: %d samples, FIRE driven in %d, first at %s s, last at %s s, end state %s",
|
||||
len(samples), len(driven), first, last, state)
|
||||
ctx.check(state == "idle", "L: ended in %s", state)
|
||||
ctx.check(not inside_lead, "L: FIRE driven inside the lead: %s", inside_lead[:5])
|
||||
ctx.check(driven, "L: FIRE never driven after the lead (the inhibit never cleared)")
|
||||
ctx.check(first is not None and first < lead_s + 0.3,
|
||||
"L: FIRE first driven at %s s; the lead ends at %.1f s", first, lead_s)
|
||||
ctx.check(not ev["L"]["any_laser_on"],
|
||||
"L: LASER_ON active with the chain unarmed - the AND-gate did not hold")
|
||||
ctx.check(rd("cnc/laser_enable") == "0", "L: FIRE still driven after end-of-data")
|
||||
ctx.log("L PASS: FIRE low through the %.1f s lead, driven from the waypoint on, LASER_ON off",
|
||||
lead_s)
|
||||
finally:
|
||||
wr("cnc/laser_latch", 1)
|
||||
try:
|
||||
wr("cnc/ramp_rate", 125000)
|
||||
wr("cnc/disable", 1) # the script's safe state
|
||||
except OSError:
|
||||
pass
|
||||
ctx.log("safe state restored: state=%s latch=LOCKED", rd("cnc/state"))
|
||||
|
||||
@@ -28,3 +28,8 @@ BAD_RECOMMENDATIONS += "eudev-hwdb"
|
||||
# signing modules (rootfs must fit the 200 MiB factory slot).
|
||||
VIRTUAL-RUNTIME_base-utils-syslog = "rsyslog"
|
||||
PACKAGECONFIG:pn-rsyslog = "rsyslogd rsyslogrt klog inet regexp"
|
||||
|
||||
# Local (file://) source checksums leave out a workstation's Python bytecode
|
||||
# caches: the forgetest recipe fetches its package directory whole, and a
|
||||
# host test run must not move a task hash without a source change.
|
||||
BB_SIGNATURE_LOCAL_DIRS_EXCLUDE = "CVS .bzr .git .hg .osc .p4 .repo .svn __pycache__ .pytest_cache"
|
||||
|
||||
Reference in New Issue
Block a user