Commit Graph
17 Commits
Author SHA1 Message Date
ScottW514 2894269115 The deny rules: the machine itself is never a destination
The way through the extension sandbox's deny rules is an allowlist, and an
allowlist names addresses. The machine's own LAN address is not a fact
anybody can pin: a new DHCP lease can turn a peer's address on some
package's list into the machine's, and with it open the Grbl port or
forgectrl's listeners to that package.

ffx.nft now refuses it structurally. Everything a host sends to one of its
own addresses, the LAN one included, leaves through lo, so chain pool
refuses `oifname "lo"` before it looks at the allow map; the two refusals
(a reset for TCP, a drop for the rest, both counted) move into chain
refuse, which pool jumps to from both places. No kernel option is new:
oifname is in the nf_tables core.

scripts/sandbox-rules-test.py gets a destination that is not the machine:
a second network namespace joined by a veth pair, with listeners of its
own. A pool uid is refused on loopback, IPv6 loopback, its own LAN address,
and the peer; an allow chain opens one port of the peer to one uid and
nothing else; with loopback, IPv6 loopback, and the machine's own address
added to that list the uid is still refused at all three while the peer
still answers; a reload closes it. It needs ip and nsenter now.

exthost.platform reads its counters from chain refuse, holds the rule's
place ahead of the map, and adds the case on the machine: an allow chain
for the last pool uid that names forgectrl on loopback and on the LAN
address opens neither, and the chain is removed.

Proven. The rules test passes with nft 1.0.9, and four controls each fail
it: the range one uid short, the TCP reject turned to accept, the
delete-table line removed, and the lo rule removed (the uid then reaches
all three of the machine's addresses). On the bench reference, image
20260920211625, this rule file loaded from /tmp with nft -f and this suite
file mounted: exthost.platform PASS, uid 831 refused at 127.0.0.1:443 and
172.16.1.97:443 with both on its allowlist, the counters [0, 0] to
[12, 4]. Against the image's own rules the same test fails on the rule's
absence, which is the control. The image's rules were reloaded after. The
unit suite passes (422).

Acceptance. exthost.platform gates the rule on the machine; sandbox-ci
gates the file. The rule file is layer content, in the platform identity
of every fingerprint.
2026-09-20 19:53:05 -04:00
ScottW514 d627ee32f1 The extension sandbox platform: accounts, cgroups, and the deny rules
What the image holds ready before any extension package exists, so that
the first one starts inside it.

forgefirm-sandbox (new recipe, on both images):
  - the account pool: ffx0 to ffx31, uid and gid 800 to 831, one group
    each, /nonexistent, /bin/false, locked. Below 1000 on purpose: the
    forgefirm-users render replaces only the accounts from 1000 up, so an
    account reset leaves the pool alone and the read-only rootfs never
    needs an account made at run time. The image's dynamic system ids
    count down from 999 and stop at 997.
  - an rcS script at S30: cgroup v2 mounted at /sys/fs/cgroup, the cpu,
    memory, and pids controllers handed down to /sys/fs/cgroup/ffx, and
    ffx marked idle-class (cpu.idle; the kernel refuses a cpu.weight on
    top of it, so none is written). The firmware's processes stay in the
    root group. `status` reports both halves and exits nonzero when
    either is missing.
  - /etc/forgefirm/ffx.nft, loaded by the same script, before the network
    starts in rc5: table inet ffx, an output-hook filter with policy
    accept that sends uid 800-831 to chain pool; pool looks the uid up in
    the verdict map `allow`, then answers TCP with a reset (a drop would
    leave a connect to time out) and drops the rest (the sender sees
    EPERM), both counted. The map is the one way through: a uid mapped to
    a chain of that package's destinations. Loading the file again
    replaces the table, allowlists included: it fails closed.

nftables comes in as its runtime dependency, trimmed in the distro config
to the binary and its library with JSON output: no interactive shell, no
Python binding. gmp and jansson were on the image; libmnl and libnftnl are
new. The release rootfs goes from 34.5 to 34.1 MiB free.

scripts/sandbox-rules-test.py, and the workflow sandbox-ci that runs it:
the rule file loaded into a network namespace of its own and sent at from
real uids. Root, 799, and 832 are not touched; 800, 815, and 831 are
refused on 127.0.0.1 and ::1 at once, UDP with EPERM, and a receiver hears
nobody from the pool; an allow chain opens one port on one address to one
uid and nothing else; a reload closes it.

exthost.platform (new suite module exthost.py): the platform proven on a
probe process, not read off a config. In a probe group under ffx, as the
last pool uid: held to cpu.max, stopped by cgroup.freeze and running again
after, stopped at pids.max, killed by the group's own OOM at memory.max
while forgectrl keeps its pid. The 32 accounts as the boot's render left
them. Pool uids 800 and 831 refused TCP to forgectrl on loopback (both
ports, IPv4 and IPv6), to the LAN address, and to the Grbl port, at once,
UDP EPERM, with the rules' counters moving by at least the attempts, while
root reaches the same listeners. A root probe under landlock loses /etc
and TCP connects and keeps /usr; a seccomp filter returns EPERM for the
filtered call. The probe group is removed whatever happens.

Proven. The rules test passes with nft 1.0.9, the image's version, and
three controls each fail it: the range one uid short, the TCP reject
turned to accept, the delete-table line removed. The unit suite passes
(422) with no undefined name. Image 20260920211625 carries all of it (read
back from both rootfs images: 32 accounts in passwd, group, and shadow,
S30forgefirm-sandbox, the rule file and the script byte-identical, nft
with its libraries and no Python binding). On the bench reference, that
image: exthost.platform PASS (5.0 percent of the core under a 5 percent
cpu.max, 0 us frozen and 87358 us thawed over 1.5 s each, 5 of 12 forks
then EAGAIN, rc -9 with oom_kill 1 at a 24 MiB memory.max, the counters
[0, 0] to [12, 4], landlock ABI 6), and forgefirm-sandbox status reports
both halves in place.

Acceptance. exthost.platform gates the platform; sandbox-ci gates the rule
file. The recipe, the rules, and the distro option are layer content, in
the platform identity of every fingerprint.
2026-09-20 18:02:31 -04:00
ScottW514 147393322f CI: run the push workflows on master only, not on tag pushes
on: push had a paths filter but no branch restriction, so a tag push
(a release's v<version> tag) re-ran forgetest-ci and fixture-ci on the
commit master had already tested - a tag push bypasses the paths
filter. branches: [master] scopes the push trigger to the branch;
pull_request and workflow_dispatch are unchanged.
2026-09-15 20:21:44 -04:00
ScottW514 0107c2cad4 Remove the cold-build workflow: Yocto builds on the build host only
The yocto-cold-build workflow and its kas/ci.yml overlay built the release image on a hosted runner as a reproducibility probe. It never ran to completion, its first dispatch (2026-09-09) stopped on the runner's user-namespace rule, and a probe nobody runs is a trap. Every Yocto build, the release included, runs on the build host; the release proof is the local pipeline (release.sh) and the bench campaign. The pre-publish checklist loses its self-containment line to match.
2026-09-09 13:28:05 -04:00
ScottW514 867b1938e4 Cold build: allow unprivileged user namespaces on the noble runner
BitBake isolates the network of its tasks with a user namespace, and the ubuntu-24.04 hosted runner's AppArmor profile refuses that to an unprivileged process, so the cold build stopped before its first task (run 34381825302). The workflow lifts the restriction for the run; nothing in the layers or the image changes.
2026-09-09 13:22:12 -04:00
ScottW514 b0fa4ccaf5 A release publishes the source of the software it installs
The release build merges kas/source-bundle.yml, which turns on the Yocto
archiver: the upstream source of each recipe as upstream publishes it, the
patches with their series file, and the recipe with its includes. The
overlay adds tasks only, so the image manifest is unchanged and an
acceptance result still applies; proven on the build host, where the
archiver build and a plain rebuild of the same tree give the same
content_sha256.

scripts/source-bundle.py packs forgefirm-source-v<version>.tar.gz: the
archives, both license manifests, the license texts, the ForgeFIRM layers,
the kas configuration, the layer revisions and the build identity of the
image. What the bundle must hold comes from the image, not from a list in
the script: every recipe of license.manifest and image_license.manifest
whose license is in the include list must have an archive, or the release
stops with the recipe named. release.sh attaches the bundle and covers it
with sha256sums.txt; FORGEFIRM_SOURCE_SKIP=1 bypasses deliberately.

On the build host: 68 of the image's 111 recipes carry source, 313.7 MiB,
under the 2 GiB limit of a release asset.

No acceptance catalog consequence: the change is release tooling on the
build host and puts no file and no behavior on the machine. The host-side
proof is forgetest/tests/test_source_bundle.py, which holds the license
decision, the choice of archive and the refusal.
2026-09-07 09:49:18 -04:00
ScottW514 77c11c52e4 forgetest CI: run the python steps with -B so no bytecode enters the recipe fetch
B-16: the forgetest recipe fetches the package directory whole, and the
file fetcher's checksum is taken before do_unpack drops __pycache__, so a
host `__pycache__` written by running the tests moves the recipe's task
hash with no source change. The three python steps run with -B now;
`src-sync` to the build VM excludes the caches as well. A developer who
runs the tests locally without -B still creates them; there is no exclude
on the file fetcher, so this is the floor, not a cure (BRINGUP item 11).
2026-09-02 19:09:07 -04:00
ScottW514 970f10a9e2 forgetest: the dwell-gap latch rule judges the hardware's unlocked window, the live runs keep a trail
laser.emission-witness required the hardware button latch clear in every
sample the engine reported armed, and, after a first fix, in every sample
up to the last nonzero emission count. Both windows were drawn from
lagging signals: the engine's armed flag follows the controller's next
report, and the emission counter latches once per second and reads
nonzero about two seconds past the relock. Both reached into the tail
where the job-end relock sets the button latch by design, and the rule
refused three clean runs on image 20260902144848 (all four sides
burned; the trail shows the latch clear from the press to the relock,
emission through the fourth side, HV_ENABLE's dip in the dwell and its
return).

The rule now uses the window the hardware defines: from the first
emission, in every sample whose readback word shows the laser latch
unlocked, the button-latch bit of that same word must be clear. That
spans the kernel-run gap of the dwell and ends at the relock, and no
lagging flag can misplace it. dwell_gap() is a pure function;
tests/test_laser_dwell.py holds the relocked tail, a set inside the gap,
and a trail without emission. The recorded trail of the third run
replays to a pass (47 unlocked samples, none set).

The live runs keep a per-sample trail in the evidence (TRAIL_FIELDS: the
readback word, the switches, the lock flag, the controller's state and
messages), so a run's timeline can be read back without a rerun.

A fourth run then errored on a name the refactor had removed and one
later check still used; py_compile does not catch it and a live drill
never executes on the host, so the CI job now fails on any undefined
name in the harness (pyflakes).

Catalog consequence: the laser implementation hashes move.
2026-09-02 17:42:07 -04:00
ScottW514 ef2a8607c2 repos: the sources live in the openglow-org organization
The recipe URLs, the release and install URLs, the vendor check and the CI checkouts name openglow-org, and the grblHAL core fork is openglow-org/grblHAL-core. No catalog change is owed: the recipe edits move the meta-forgefirm content hash, which every test fingerprint folds in through the platform block, so the whole catalog re-runs on its own.
2026-09-01 19:44:47 -04:00
ScottW514 6a48cd3969 docs: the moved documents live on the documentation site
INSTALL.md, SERIAL.md, docs/COOLING.md, docs/LIGHTBURN.md,
docs/MOTION.md, docs/SAFETY.md, docs/UPDATE-SYSTEM.md, docs/VIDEO.md and
their images are pages on https://docs.forgefirm.org/ now. Every
reference in the README, BRINGUP, the kas config, the cold-build
workflow, forgetest, and the bench scripts points to the site page. The
README carries the beta banner. docs/ keeps BRINGUP.md and
CAMPAIGN-LOG.md.

No catalog consequence: the deleted files are documents, and the code
changes are comment and help-text repoints only.
2026-09-01 15:33:17 -04:00
ScottW514 533ae7e46d Move the build, release, and acceptance docs to the documentation site
The documentation site, docs.forgefirm.org, is the one home for the
developer documentation. BUILD.md, kas/README.md, and docs/ACCEPTANCE.md
are on the site under Developers, and they are deleted here. Every
reference points at the site: README.md, BRINGUP.md, UPDATE-SYSTEM.md,
the kas configuration comments, release.sh, the CI workflow comments, and
the releases, forgetest, and bench READMEs.

BRINGUP.md: "Next work" item 16, step timing under CPU contention, is
closed; the video work resolved it. The items after it are renumbered.
CAMPAIGN-LOG.md records the closure.

Documentation and comments only. No layer content changes, so the
manifest identity of the next image does not change. No catalog
consequence.
2026-08-26 10:00:42 -04:00
ScottW514 e99888753f forgetest's page moves onto Bootstrap with forgectrl's theme; forgectrl pinned at the panel overhaul
The acceptance page is assembled by page.py from forgetest/forgetest/ui/
(index.html, page.css, help.js, app.js) plus theme.css and the vendored
Bootstrap files, which are byte for byte the ones forgectrl's panel
carries, so the two pages look like one product and share the light and
dark themes (same localStorage key). A plain file is read in a checkout;
on the dev image the recipe installs ui/ gzipped and page.py reads the
.gz sibling, inflating once at first request: the rootfs is raw ext4, so
bytes in the package are bytes on the image. The explanatory prose
(campaign rules, the queues, the campaign actions, the prerequisites
switch, the bench intro) is a "?" popover with a link into the
documentation site; operator steps, prompts, notices and the live-laser
acknowledgment stay in the page, and confirmLive() stays a blocking
dialog. The page's own rules hold: rows, prompt buttons and tool entries
are built once and updated in place, and the popovers sit on static
markup only, so no rebuild orphans one. On a phone the Run pane goes to
the top for the duration of a run.

scripts/check-ui-vendor.py compares the shared files against forgectrl
at its pinned revision (or a local checkout with --forgectrl); it runs
in forgetest-ci.yml, so the copies cannot drift.

Tests: test_page.py (the gzipped install assembles to the same bytes as
a checkout, one self-contained response, the token placeholder once, a
missing marker refused); test_server asserts the served page's
invariants; test_responsiveness keeps its rules with needles pointed at
the new files, its ASCII rule applied to our own sources (Bootstrap's
CSS carries an em dash of its own), and its self-contained rule testing
asset tags rather than the presence of https:// (the documentation links
are meant to be there). forgectrl.panel-serves gains two needles for the
panel's theme attribute and save bar. Proof: the unit suite, and the
page in Chrome against a fake catalog (both themes, popovers, the bench
tab, a full operator run with its prompt, abort).

forgectrl pinned at 9d1f6f2 (the panel on Bootstrap, one save bar, help
popovers, themes, the gzipped page); PV unchanged. The pin moves only
forgectrl's fingerprint. The forgetest changes are the harness's own and
have no catalog consequence.
2026-08-24 12:13:53 -04:00
ScottW514 8ee4ee3946 forgefixture: the bench actuator's firmware
An ESP32-S3 DevKitC-1 on the bench network drives three relays at the
machine's connectors so the acceptance tool can open the lid loop, pull
the interlock loop and press the button with nobody in the room. Two
contacts are normally closed in loops the safety chain already reads,
the third is normally open across the button input and only ever
pulsed (20 to 500 ms, the end armed before the line rises); every line
is low at boot and after any reset, the task watchdog panics and
reboots, and the button channel needs a jumper. HTTP on port 80 under
a key in X-Fixture-Key; the hostname announced over DHCP and mDNS.

ESP-IDF v5.5 native, the mDNS component pinned in dependencies.lock;
fixture.env (git-ignored) is the one input: the wifi, the key, the
hostname. fixture.sh builds with idf.py or in the espressif/idf
container, flashes with esptool from pip. The decisions that need no
hardware live in policy.c with a gcc host test; CI runs it and builds
the firmware in the pinned container.
2026-08-23 14:15:53 -04:00
ScottW514 6c1d68f2c3 Judge the cloud resume on the lines the app logs; guard every needle against the pinned app
cloud.pause-resume failed a print that paused, resumed with its laser
lead, completed and parked: the test waited for the single line "button
pressed while paused; resuming", and the app has logged that as two
lines since its feeder work ("button pressed while paused", then
"resuming (laser lead N ticks)" from _resume_retraced). The replay
fixture carried the old wording, so the host test kept passing.

The pause and resume are now judged on PAUSE_LINES + RESUME_LINES
through one checker shared by the three tests that drive a pause
(cloud.pause-resume, the streamed pause, the pause-then-lid test), which
also fails on the app's "resume refused" line with the reason. The
fixture carries the app's two lines.

So the wording cannot drift silently again: tests/test_cloud_needles.py
reads every log phrase the cloud suite greps for out of cloud.py (the
left side of each `x in ln`, every wait_log needle, the mark tuples, and
the phrases it builds) and checks each against the logger calls in the
app sources at the revisions the recipes pin, read from the manifest
cache the tree manifest builds (the sibling checkouts locally),
placeholder-aware under a rule that never lets a placeholder stand for
the phrase itself. CI now builds the tree manifest before the unit
tests so the cache is there. The old needle fails that check.

Replays added: the second press seen but no retraced restart, and a
refused resume. 159 unit tests pass; coverage lint clean. No catalog
consequence beyond the suite module's own hash.
2026-08-21 14:09:51 -04:00
ScottW514 1179d5e7c1 Release acceptance gate: release.sh refuses to sign without a matching artifact
scripts/acceptance-gate.py recomputes every catalog test's domain fingerprint
from /etc/forgefirm-manifest.json inside the release rootfs and requires the
committed releases/v<version>/acceptance.json to carry a matching PASS
(inherited results not core and newer than the invalidate epoch; the artifact
self-hashed; the catalog identical to the tree). release.sh runs it after the
build and stages the artifact as a release asset; FORGEFIRM_ACCEPTANCE_SKIP=1
bypasses loudly. scripts/manifest-from-tree.py builds the same manifest from
the recipe pins with git for CI and the workstation; forgetest-ci.yml runs the
unit tests and enforces the coverage lint (every manifest path covered by some
test). docs/ACCEPTANCE.md is the contract; the coverage currency rule and the
status live in BRINGUP.
2026-08-15 15:57:12 -04:00
ScottW514 0a05b6b114 docs: present-state build and update docs; fix the ring-size arithmetic
- kas/README.md: the real-time rationale rests on the feeder's bounded
  queue depth, not ring size; the ring is 16 MiB (~84 s at 200 kHz,
  ~28 min at the 10 kHz cloud tick), a capacity for cloud-mode preload.
- BUILD.md, kas config, release checklist, cold-build workflow: only
  forgefirm and meta-openglow (branch scarthgap) are cloned as
  siblings; every ForgeFIRM source repo is fetched by pinned SRCREV.
- UPDATE-SYSTEM.md reads as the present-state design: the cloud-mode
  compatibility baseline is the cloud client's configured firmware
  version, not release metadata; decisions and open items listed
  plainly.
- README.md states what GRBL mode still needs the Glowforge service
  for (camera-referenced homing) and what runs without it.
- BRINGUP.md: generic build-host and fwup-lab references, the retained
  reproductions of the no-fire drill, the System tab.
- LIGHTBURN.md: the arm-window timeouts are machine settings.
- forgefirm-image.bb describes forgectrl as the machine-services
  daemon and points at the right backlog entry.
- American spelling throughout.
2026-08-15 06:12:51 -04:00
ScottW514 a2f1b8e9e8 CI: dispatch-only cold-build reproducibility workflow
Proves a fresh clone builds the release image on a hosted runner
(sibling checkouts for meta-openglow and the kernel-module externalsrc,
rm_work to fit the disk budget) and publishes artifact checksums for
comparison against locally built releases. Never produces release
artifacts - releases are built and signed on the maintainer's host.
2026-08-08 13:32:00 -04:00