vh/forgefirm
1
0
Fork 0
mirror of https://github.com/openglow-org/forgefirm.git synced 2026-10-15 17:19:55 -07:00
Code Issues Packages Projects Releases Wiki Activity

Catalog: the package routes, the catalog, a package's check and the camera key leave

Browse Source 
Four catalog tests leave for the host proofs that carry them; the
catalog goes from 87 to 83 (83 @test in forgetest/forgetest/suite/).
Each proof passed; none skipped.

- exthost.package-routes, for forgectrl's tests/host_ext.sh (05848ac):
  against a compiled stand-in that is a live host, GET /ext/status and
  POST /ext/package are 403 without the token with the host not run,
  the status passes the host's list through with host.running true and
  the stand-in's pid, enable runs as the host's command, a bad action
  and a bad id are 400 with nothing run, and a package that is not
  installed is 409 in the host's words. extpkg_test holds the relay's
  argv; the state file, the required-holds directory and removal are
  forgeext's install_test.py and run_test.py; the page card is
  check_ui_js.py. REQUIRED_HOLDS, which only it read, and the "the hold"
  section header over it go.
- exthost.wizard, for host_ext.sh (05848ac): the check listed, run to its
  prompt with no lease holder, answered yes, ended with its summary and
  listed done; the setup record's wizards and the gate as they were; a
  second run aborted; a check of a package that is not installed 404.
  wizpkg_test holds each step to its form. suite/extwizard.py held only
  it: the module and its import go.
- exthost.catalog: install_test.py and extpkg_test already assert its
  behavioral claims, and the one fact the bench test did not prove, that
  the image's extension key is the catalog's published key, is now a
  forgetest-ci step: the catalog is checked out at main as a sibling and
  meta-forgefirm/recipes-forgefirm/forgefirm-keys/files/ext/forgefirm-ext.pub
  is cmp -s'd against its keys/forgefirm-ext.pub, either one missing a
  failure. suite/extcat.py held only it: the module and its import go.
- camera.key-read, for forgectrl's tests/host_lan.sh (05848ac): with
  panel_open_reads=0, GET /cam/status from the LAN client over HTTP and
  HTTPS is 403 with no key or a wrong one and 200 with the key as ?key=
  or the header, the key alone writes no setting, and a rotation stops
  the old key and the new one reads. camera.py's json import, which only
  it used, goes.

The coverage lint then named forgectrl src/wizpkg.c, src/wizpkg.h,
src/camkey.c and src/camkey.h, and nothing else. Two HOST_TESTED rows
take them, to forgectrl:tests/wizpkg_test.c and
forgectrl:tests/camkey_test.c on forgectrl:.github/workflows/build.yml,
which the pin c274f19 runs (./build/wizpkg_test, ./build/camkey_test).
src/extpkg.* stays covered (logs.tree-tail-export, logs.routing), and
forgeext/** by exthost.service.

The key step's run text, taken from the workflow, passes from the tree
root with both files present; controls on a /tmp copy, a published key
that differs and one that is missing: each fails. test_manifest.py (34)
passes. The coverage lint on the tree manifest finds no uncovered path
and no host-tested row problem across the 83 tests; its one finding is
setup.check-lens's empty src/lensfind.* entry. pyflakes reports nothing
on exthost.py, camera.py or manifest.py. The full forgetest suite was
not run.

Fingerprints, measured: 10 tests move, the 5 left in exthost.py
(armed-freeze, motion-jog, motion-job, platform, service) and the 5 left
in camera.py (frame-health, h264-stream, lid-privacy, sensor-profile,
snapshot). The pending pin bumps from forgectrl c274f19 and forgeext
2d0391e already owe 9 of them; exthost.platform is owed already since
30c0a05. This adds none. releases/ and TEST-AUDIT.md are unchanged.
This commit is contained in:
ScottW514
2026-10-14 16:24:45 -04:00
parent 30c0a05b86
commit b90299acdc
7 changed files with 26 additions and 619 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/forgetest-ci.yml
+19
View File
@@ -11,6 +11,9 @@
# release with source missing # release with source missing
# - shared UI files: theme.css and the vendored Bootstrap the page # - shared UI files: theme.css and the vendored Bootstrap the page
# carries are byte-identical to forgectrl's at its pinned revision # carries are byte-identical to forgectrl's at its pinned revision
# - the extension key: the public key the image installs for the official
# tier is byte-identical to the one the extension catalog publishes
# (keys/forgefirm-ext.pub on its main branch)
# - coverage lint: every source path of every component pinned by the # - coverage lint: every source path of every component pinned by the
# recipes must be selected by some catalog test's coverage globs or by a # recipes must be selected by some catalog test's coverage globs or by a
# host-tested row naming the host test that proves it (the tree manifest # host-tested row naming the host test that proves it (the tree manifest
@@ -50,6 +53,22 @@ jobs:
ref: scarthgap ref: scarthgap
path: meta-openglow path: meta-openglow
- name: Checkout forgefirm-extensions-catalog (sibling)
uses: actions/checkout@v4
with:
repository: openglow-org/forgefirm-extensions-catalog
ref: main
path: forgefirm-extensions-catalog
- name: The image's extension key is the catalog's published key
run: |
image=forgefirm/meta-forgefirm/recipes-forgefirm/forgefirm-keys/files/ext/forgefirm-ext.pub
published=forgefirm-extensions-catalog/keys/forgefirm-ext.pub
for f in "$image" "$published"; do
test -f "$f" || { echo "missing: $f"; exit 1; }
done
cmp -s "$image" "$published" || { echo "$image is not $published"; exit 1; }
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v5 uses: actions/setup-python@v5
with: with:
forgetest/forgetest/manifest.py
+7
View File
@@ -144,6 +144,13 @@ HOST_TESTED = [
# it: the claim and its refusals, its end by the package, the operator # it: the claim and its refusals, its end by the package, the operator
# or a stale keeper (forgeext's run_test proves the writer's side). # or a stale keeper (forgeext's run_test proves the writer's side).
("forgectrl", "src/senderout.*", "forgectrl:tests/senderout_test.c", "forgectrl:.github/workflows/build.yml"), ("forgectrl", "src/senderout.*", "forgectrl:tests/senderout_test.c", "forgectrl:.github/workflows/build.yml"),
# A package's check on the Setup page: each step held to its form, the
# check run through the host, the result the package's own (the routes
# against a live host's stand-in are tests/host_ext.sh).
("forgectrl", "src/wizpkg.*", "forgectrl:tests/wizpkg_test.c", "forgectrl:.github/workflows/build.yml"),
# The camera key: made, kept, checked and rotated (the reads it opens
# from the LAN with the reads closed are tests/host_lan.sh).
("forgectrl", "src/camkey.*", "forgectrl:tests/camkey_test.c", "forgectrl:.github/workflows/build.yml"),
] ]
# This repository's root, when the package runs from a checkout of it; a # This repository's root, when the package runs from a checkout of it; a
forgetest/forgetest/suite/__init__.py
-2
View File
@@ -21,8 +21,6 @@ from . import update # noqa: F401,E402
from . import cloud # noqa: F401,E402 from . import cloud # noqa: F401,E402
# Last: it stands on the setup record's consent and on the cloud suite's dark print. # Last: it stands on the setup record's consent and on the cloud suite's dark print.
from . import exthost # noqa: F401,E402 from . import exthost # noqa: F401,E402
from . import extcat # noqa: F401,E402
from . import extwizard # noqa: F401,E402
from . import homeoff # noqa: F401,E402 from . import homeoff # noqa: F401,E402
from . import bedsize # noqa: F401,E402 from . import bedsize # noqa: F401,E402
from . import tray # noqa: F401,E402 from . import tray # noqa: F401,E402
forgetest/forgetest/suite/camera.py
-65
View File
@@ -4,8 +4,6 @@
# SPDX-License-Identifier: MIT # SPDX-License-Identifier: MIT
"""camera.* - the lid camera pipeline through forgectrl.""" """camera.* - the lid camera pipeline through forgectrl."""
import json
from ..catalog import test from ..catalog import test
from .. import hw from .. import hw
from ..baseline import LID_LAMP_ATTR from ..baseline import LID_LAMP_ATTR
@@ -496,66 +494,3 @@ def lid_privacy(ctx):
ctx.check(st == 200 and data[:2] == b"\xff\xd8", ctx.check(st == 200 and data[:2] == b"\xff\xd8",
"snapshot after closing the lid -> %s", st) "snapshot after closing the lid -> %s", st)
ctx.log("lid closed again: snapshot %d bytes", len(data)) ctx.log("lid closed again: snapshot %d bytes", len(data))
@test("camera.key-read", title="The camera key reads without a login", subsystem="camera",
kind="auto", est_min=1,
covers=[("forgectrl", "src/camkey.*"), ("forgectrl", "src/auth.c"),
("forgectrl", "src/main.c"), ("forgectrl", "src/ui/panel.js"),
("forgectrl", "src/ui/index.html"), ("forgectrl", "src/ui/help.js")],
requires=["forgectrl.auth"],
description="With the read routes closed to the network (panel_open_reads=0), a camera "
"status read from the board's LAN address is refused without the key and "
"served with it, on HTTP and on HTTPS; a wrong key is refused; the key never "
"authorizes a write; a rotation from the panel route stops the old key and "
"the new one reads. The setting is put back after.")
def key_read(ctx):
from .setup import request
from .forgectrl import lan_ip
fc = ctx.forgectrl
ev = ctx.evidence
ip = lan_ip()
ctx.check(ip, "cannot determine the board's LAN address")
http_base = "http://%s" % ip
tls_base = "https://%s" % ip
token = {"X-ForgeFIRM-Token": fc.token}
st, body, _ = request(fc.base, "GET", "/system/camera-key", headers=token)
ctx.check(st == 200, "GET /system/camera-key from loopback -> %s", st)
key = json.loads(body)["key"]
ctx.check(len(key) == 32, "the key is not 32 characters: %r", key)
ev["key_len"] = len(key)
before = fc.settings().get("panel_open_reads", "")
st, _ = fc.post("/settings", params={"panel_open_reads": "0"})
ctx.check(st == 200, "closing the reads -> %s", st)
try:
for base in (http_base, tls_base):
st, _, _ = request(base, "GET", "/cam/status")
ctx.log("GET %s/cam/status (no key) -> %s", base, st)
ctx.check(st == 403, "a closed read without the key -> %s, expected 403", st)
st, _, _ = request(base, "GET", "/cam/status?key=" + key)
ctx.log("GET %s/cam/status?key=... -> %s", base, st)
ctx.check(st == 200, "a read with the key -> %s, expected 200", st)
st, _, _ = request(base, "GET", "/cam/status", headers={"X-ForgeFIRM-Camera-Key": key})
ctx.check(st == 200, "a read with the key header -> %s, expected 200", st)
wrong = ("0" if key[0] != "0" else "1") + key[1:]
st, _, _ = request(base, "GET", "/cam/status?key=" + wrong)
ctx.check(st == 403, "a wrong key -> %s, expected 403", st)
# the key is a read credential only
st, _, _ = request(tls_base, "POST", "/settings", data={"panel_open_reads": "0"},
headers={"X-ForgeFIRM-Camera-Key": key})
ctx.log("POST /settings with the camera key -> %s", st)
ctx.check(st == 403, "the camera key authorized a write (%s)", st)
# rotation
st, body, _ = request(fc.base, "POST", "/system/camera-key?rotate=1", headers=token)
ctx.check(st == 200, "rotate -> %s", st)
new = json.loads(body)["key"]
ctx.check(new != key, "rotation kept the same key")
st, _, _ = request(http_base, "GET", "/cam/status?key=" + key)
ctx.check(st == 403, "the old key still reads after rotation (%s)", st)
st, _, _ = request(http_base, "GET", "/cam/status?key=" + new)
ctx.check(st == 200, "the new key does not read (%s)", st)
ev["rotated"] = True
finally:
fc.post("/settings", params={"panel_open_reads": before})
forgetest/forgetest/suite/extcat.py
-188
View File
@@ -1,188 +0,0 @@
# Copyright 2026 514 LLC d/b/a OpenGlow
# Written by Scott Wiederhold
# https://community.openglow.org
# SPDX-License-Identifier: MIT
"""The catalog: the signed index, on the machine.
Its own module, for the reason extcore.py gives. The endorsement, the
judgment of each listed version, the index that never goes back, and a
withdrawn version are shown on a scratch root under /tmp with a throwaway
key standing in for the OpenGlow extension key, because only OpenGlow holds
the real one; the machine's own root is where every index that key did not
sign is refused.
"""
import hashlib
import io
import json
import os
import shutil
import subprocess
import tarfile
import tempfile
from ..catalog import test
from .exthost import FWUP, _forgeext, _write
INDEX_URL = "https://github.com/openglow-org/forgefirm-extensions-catalog/releases/latest/download/index-1.ffi"
STAGE_DIR = "/data/forgefirm/tmp"
PROBE_ID, OTHER_ID = "org.example.catprobe", "org.example.catother"
def _key(work, name):
subprocess.run([FWUP, "-g", "-o", os.path.join(work, name)], check=True, capture_output=True, timeout=60, cwd=work)
return os.path.join(work, name)
def _archive(work, name, product, members, version, key):
"""An fwup archive of the extension container's form: payload.tar.gz of members {name: text}, signed with key."""
payload = os.path.join(work, name + ".tar.gz")
with tarfile.open(payload, "w:gz") as t:
for member, text in members.items():
info = tarfile.TarInfo(member)
data = text.encode()
info.size, info.mode = len(data), 0o644
t.addfile(info, io.BytesIO(data))
conf = os.path.join(work, name + ".conf")
_write(conf, 'meta-product = "%s"\nmeta-description = "%s"\nmeta-version = "%s"\nmeta-platform = "forgefirm-ext"\n'
'file-resource payload.tar.gz {\n host-path = "%s"\n}\n' % (product, name, version, payload))
raw, signed = os.path.join(work, name + ".raw"), os.path.join(work, name + ".signed")
subprocess.run([FWUP, "-c", "-f", conf, "-o", raw], check=True, capture_output=True, timeout=60, cwd=work)
subprocess.run([FWUP, "-S", "-s", key + ".priv", "-i", raw, "-o", signed], check=True, capture_output=True,
timeout=60, cwd=work)
return signed
def _package(work, id_, key):
m = {"manifest": 1, "id": id_, "name": "catalog probe", "version": "1.0.0", "author": "forgetest", "license": "MIT",
"api": "0.1", "runtime": "data", "capabilities": []}
return _archive(work, id_, "ForgeFIRM extension", {"manifest.json": json.dumps(m)}, "1.0.0", key)
def _index(work, doc, key, version):
return _archive(work, "index-" + version, "ForgeFIRM extension index", {"index.json": json.dumps(doc)}, version, key)
def _staged():
try:
return sorted(n for n in os.listdir(STAGE_DIR) if n.startswith("ext-"))
except OSError:
return []
@test("exthost.catalog", title="The catalog: the signed index verified, kept, judged, and endorsing one key for one id",
subsystem="exthost", kind="auto", hardware="api", est_min=2,
covers=[("forgectrl", "src/extpkg.*"), ("forgectrl", "src/main.c"), ("forgeext", "src/index.*"),
("forgeext", "src/pkg.*"), ("forgeext", "src/install.*"), ("forgeext", "src/main.c")],
description="GET /ext/catalog answers the index the host keeps (or null), judged against this firmware, and "
"the one address it is fetched from. On a scratch root, with a throwaway key standing in for the "
"OpenGlow extension key, the machine's own forgeext keeps an index signed with it, and the author "
"key it names for one id makes a package of that id signed with it read as community and "
"endorsed, where before it was unverified; the same key on another id counts for nothing. Read "
"back, each listed version is judged on the machine: one that asks for a capability this "
"firmware does not have is kept and not offered, and the offer is the newest it runs. An index "
"older than the one kept is refused however well signed, and one that withdraws the probe's "
"version makes the machine refuse to install it, in OpenGlow's words. On the machine's own root, "
"the stand-in's index (signed by a key that is not the OpenGlow extension key) is refused in "
"words, a package handed over as an index is refused by the product gate, and the index kept "
"is left as it was. POST /ext/catalog/get refuses an id with no such form (400), and one the "
"kept index does not list (404, or 409 with no index kept), before anything is fetched. Nothing "
"is left in the staging directory. POST /ext/catalog/refresh is not asked here: GitHub counts "
"every request of the index's address as a download, and that count is the operators'. The "
"refresh (curl, https alone and bounded, 502 in curl's words, 409 in the host's, the file "
"removed), the fetch of the offered version held to its size and SHA-256, and the tiers an "
"install takes from it, are forgectrl's extpkg_test and forgeext's install_test.")
def catalog(ctx):
fc = ctx.forgectrl
ev = ctx.evidence
ctx.check(fc.wait_idle(timeout=30, abort=ctx.aborted), "machine not idle")
staged_before = _staged()
kept = _forgeext("index")
ctx.check(kept.get("ok") is True, "forgeext index -> %s", kept.get("error"))
ev["kept_before"] = (kept.get("index") or {}).get("version")
work = tempfile.mkdtemp(prefix="ft-catalog-", dir="/tmp")
try:
st, doc = fc.get("/ext/catalog")
ctx.check(st == 200 and isinstance(doc, dict) and doc.get("url") == INDEX_URL and "index" in doc
and doc.get("index") == kept.get("index") and "core_checked" in doc,
"GET /ext/catalog -> %s %s", st, doc)
# The endorsement, on a scratch root with a stand-in for the OpenGlow extension key.
root = os.path.join(work, "root")
os.makedirs(os.path.join(root, "keys"))
og, author = _key(work, "standin"), _key(work, "author")
probe, other = _package(work, PROBE_ID, author), _package(work, OTHER_ID, author)
with open(probe, "rb") as f:
data = f.read()
with open(author + ".pub") as f:
author_pub = f.read().strip()
listed = {"version": "1.0.0", "url": "https://example.org/catprobe.ffx", "sha256": hashlib.sha256(data).hexdigest(),
"size": len(data), "capabilities": [], "api": "0.1"}
later = dict(listed, version="1.1.0", url="https://example.org/catprobe-1.1.0.ffx",
capabilities=["forgetest.not-a-capability"])
entry = {"id": PROBE_ID, "name": "catalog probe", "author": "forgetest", "key": author_pub,
"versions": [later, listed]}
idx = _index(work, {"index": 1, "packages": [entry]}, og, "2026.923.2")
def scratch(*args):
return _forgeext("--root", root, "--official-key", og + ".pub", "--no-reserve", *args)
r = scratch("inspect", probe)
ctx.check(r.get("ok") is True and r.get("tier") == "unverified", "before an index: %s %s", r.get("tier"), r.get("error"))
r = scratch("index-verify", idx)
ev["scratch_verify"] = r
ctx.check(r.get("ok") is True and r.get("packages") == 1 and r.get("version") == "2026.923.2",
"the stand-in's index is kept on the scratch root: %s", r)
read = scratch("index")
mine = ((read.get("index") or {}).get("packages") or [{}])[0]
vs = {v.get("version"): v for v in mine.get("versions", [])}
ev["judged"] = {"offer": mine.get("offer"), "1.1.0": vs.get("1.1.0", {}).get("why"),
"core_checked": read.get("core_checked")}
ctx.check(mine.get("id") == PROBE_ID and len(mine.get("key_id", "")) == 64, "and read back: %s", mine)
ctx.check(vs.get("1.1.0", {}).get("usable") is False and "not a capability" in vs.get("1.1.0", {}).get("why", "")
and vs.get("1.0.0", {}).get("usable") is True and mine.get("offer") == "1.0.0",
"judged on the machine: 1.1.0 kept and not offered, 1.0.0 offered: %s", ev["judged"])
r = scratch("inspect", probe)
ev["endorsed"] = {"tier": r.get("tier"), "endorsed": r.get("endorsed")}
ctx.check(r.get("tier") == "community" and r.get("endorsed") is True,
"signed with the key it endorses for its id: community, endorsed (%s %s)", r.get("tier"), r.get("endorsed"))
r = scratch("inspect", other)
ctx.check(r.get("tier") == "unverified" and r.get("endorsed") is False,
"the same key on another id: %s %s", r.get("tier"), r.get("endorsed"))
# Never back: an older index is refused, and the one kept stays.
r = scratch("index-verify", _index(work, {"index": 1, "packages": []}, og, "2026.923.1"))
ev["older"] = r.get("error")
ctx.check(r.get("ok") is False and "never goes back" in (r.get("error") or ""), "an older index -> %s", r.get("error"))
ctx.check((scratch("index").get("index") or {}).get("version") == "2026.923.2", "and the one kept stays")
# A withdrawn version does not install, from the catalog or from anywhere else.
gone = dict(entry, versions=[later], withdrawn=[{"version": "1.0.0", "reason": "forgetest withdrew it"}])
r = scratch("index-verify", _index(work, {"index": 1, "packages": [gone]}, og, "2026.923.3"))
ctx.check(r.get("ok") is True, "an index withdrawing the probe's version is kept: %s", r.get("error"))
r = scratch("inspect", probe)
ev["withdrawn"] = r.get("error")
ctx.check(r.get("ok") is False and "OpenGlow withdrew %s 1.0.0 from its catalog: forgetest withdrew it" % PROBE_ID
in (r.get("error") or ""), "the withdrawn version -> %s", r.get("error"))
# The machine's own root keeps no index but OpenGlow's.
refused = {}
for name, path, words in (("the stand-in's index", idx, "not signed with the OpenGlow extension key"),
("a package as an index", probe, "product")):
r = _forgeext("index-verify", path)
refused[name] = r.get("error")
ctx.check(r.get("ok") is False and words in (r.get("error") or ""), "%s -> %s", name, r.get("error"))
ev["refused"] = refused
ctx.check(_forgeext("index").get("index") == kept.get("index"), "a refused index changed the one kept")
# The relay's refusals, before anything is fetched.
st, why = fc.post("/ext/catalog/get", data={"id": "org.example;reboot"})
ctx.check(st == 400, "an id with no such form -> %s %s", st, why)
st, why = fc.post("/ext/catalog/get", data={"id": PROBE_ID})
want = (404, "not in the catalog") if kept.get("index") else (409, "fetch it first")
ev["get_unlisted"] = [st, why]
ctx.check(st == want[0] and want[1] in json.dumps(why), "an id the kept index does not list -> %s %s", st, why)
ctx.check(_forgeext("index").get("index") == kept.get("index"), "the refusals changed the index kept")
ctx.check(_staged() == staged_before, "the staging directory holds %s, and held %s before", _staged(), staged_before)
finally:
shutil.rmtree(work, ignore_errors=True)
forgetest/forgetest/suite/exthost.py
-140
View File
@@ -1306,11 +1306,6 @@ def armed_freeze(ctx):
_as_found(ctx, fc, prior, raw, dir_mode, found_tree) _as_found(ctx, fc, prior, raw, dir_mode, found_tree)
# ------------------------------------------------------------- the hold
REQUIRED_HOLDS = EXT_ROOT + "/required-holds"
@test("exthost.motion-jog", title="A package jogs the machine, bounded and dark", @test("exthost.motion-jog", title="A package jogs the machine, bounded and dark",
subsystem="exthost", kind="auto", hardware="takeover", mode="grbl", est_min=7, subsystem="exthost", kind="auto", hardware="takeover", mode="grbl", est_min=7,
covers=[("forgeext", "src/api.*"), ("forgeext", "src/machine.*"), ("forgeext", "src/run.*"), covers=[("forgeext", "src/api.*"), ("forgeext", "src/machine.*"), ("forgeext", "src/run.*"),
@@ -1570,138 +1565,3 @@ def motion_job(ctx):
_put_back(ctx, fc, work, prior, etag, raw, dir_mode) _put_back(ctx, fc, work, prior, etag, raw, dir_mode)
_as_found(ctx, fc, prior, raw, dir_mode, found_tree) _as_found(ctx, fc, prior, raw, dir_mode, found_tree)
@test("exthost.package-routes", title="The panel's package routes are the extension host's own word",
subsystem="exthost", kind="auto", est_min=2,
covers=[("forgectrl", "src/extpkg.*"), ("forgectrl", "src/main.c"), ("forgectrl", "src/ui/ext.js"),
("forgectrl", "src/ui/index.html"), ("forgectrl", "src/ui/embed.cmake"), ("forgeext", "src/main.c"),
("forgeext", "src/install.*"), ("forgeext", "src/state.*")],
requires=["exthost.service", "forgectrl.auth"],
description="Extensions stay as found (the package is installed and never runs). The reference package is "
"installed with the hold grant through the host's command line. GET /ext/status without "
"the login is refused (403); with it, it lists the package as the host does (tier "
"community, enabled, the hold advisory, and the capabilities it may use - what needs no "
"grant and what the operator granted, which is what the panel's bridge decides on and "
"is never wider than what the manifest asked for) beside enabled, safe_mode, and the host's own "
"status with running true. POST /ext/package: hold-required names the package under "
"required-holds and the list says required; disable and enable change the host's state "
"file; an action outside the closed list and an id that has not the form of one are 400 "
"and run nothing; an id that is not installed is 409 in the host's words; without the "
"login 403; remove takes the package, its data, and its name away. The panel page carries "
"the card and its script, and the script is in the page before the one that calls it "
"(the panel runs its tab's loads while it parses). The key and the work "
"directory are removed and the extension root is as found.")
def package_routes(ctx):
import shutil
import tempfile
fc = ctx.forgectrl
ev = ctx.evidence
ctx.check(len(_host_pids()) == 1, "the extension host is not one running process: %s", _host_pids())
ctx.check(REF_ID not in [x.get("id") for x in _forgeext("list").get("packages", [])], "%s is already installed", REF_ID)
found_tree = _tree(EXT_ROOT)
work = tempfile.mkdtemp(prefix="forgetest-ffx.")
owner_key = os.path.join(EXT_ROOT, "keys", REF_KEY + ".pub")
def listed():
st, doc = fc.get("/ext/status")
ctx.check(st == 200 and isinstance(doc, dict), "GET /ext/status -> %s", st)
return doc, next((x for x in doc.get("packages", []) if x.get("id") == REF_ID), None)
def act(action, pid=REF_ID):
return fc.post("/ext/package", data={"id": pid, "action": action})
def state():
return json.loads(_read(EXT_ROOT + "/state.json") or "{}").get("packages", {}).get(REF_ID, {})
try:
# The card is in the page the image serves, and its script is loaded before the one whose
# tab loads call it: panel.js runs them while it parses, so a later ext.js is a broken tab.
st, page = fc.get("/", raw=True)
text = page.decode("utf-8", "replace") if isinstance(page, bytes) else str(page)
ev["panel_page_bytes"] = len(text)
ctx.check(st == 200 and text, "GET / -> %s", st)
for want in ("extpkgs", "extswitch", "extfile", "extstaged", "extinstallphrase", "extAct", "loadExt"):
ctx.check(want in text, "the panel page carries no %s", want)
ctx.check(0 < text.index("function loadExt") < text.index("loadExt();"),
"the panel page defines loadExt after the call that runs at parse time")
archive, pub = _pack_reference(work, lan_ip(), more_caps=["hold"])
shutil.copy(pub, owner_key)
os.chmod(owner_key, 0o644)
r = _forgeext("install", archive, "--consent-community", "--grant", "hold")
ctx.check(r.get("ok") is True, "the install -> %s", r.get("error"))
st, body, hdrs = request(fc.base, "GET", "/ext/status", headers={"Host": fc.host_header()})
ev["status_without_login"] = st
ctx.check(st == 403, "GET /ext/status without the login -> %s, expected 403", st)
doc, pkg = listed()
ev["status"] = {k: doc.get(k) for k in ("enabled", "safe_mode")}
ev["host"] = {k: (doc.get("host") or {}).get(k) for k in ("running", "pid", "enabled", "off_reason")}
ev["package"] = {k: (pkg or {}).get(k) for k in ("tier", "enabled", "quarantined", "grants", "hold", "account")}
ctx.log("GET /ext/status: %s, host %s, package %s", ev["status"], ev["host"], ev["package"])
ctx.check(set(doc) >= {"enabled", "safe_mode", "host", "packages"} and doc["safe_mode"] is False,
"the status document lacks a key: %s", sorted(doc))
ctx.check((doc.get("host") or {}).get("running") is True and doc["host"].get("pid") == _host_pids()[0],
"the host's own status is not passed on as a running host's: %s", ev["host"])
ctx.check(pkg and pkg.get("tier") == "community" and pkg.get("enabled") is True and pkg.get("hold") == "advisory"
and pkg.get("grants") == ["hold"] and (pkg.get("package") or {}).get("name") == "forgetest reference",
"the package is not listed as the host lists it: %s", pkg)
# What the package may use, beside what it asked for. The panel's
# bridge decides on this list and not on the manifest's, so a
# capability the operator did not grant must not be in it.
asked = set((pkg.get("package") or {}).get("capabilities") or [])
effective = set(pkg.get("effective") or [])
granted = set(pkg.get("grants") or [])
ev["capabilities"] = {"asked": sorted(asked), "effective": sorted(effective), "granted": sorted(granted)}
ctx.log("the reference package asked for %s, may use %s", sorted(asked), sorted(effective))
ctx.check(effective and effective <= asked, "it may use what it did not ask for: %s", ev["capabilities"])
ctx.check(granted <= effective, "a capability the operator granted is not in what it may use: %s",
ev["capabilities"])
st, reply = act("hold-advisory") # no change; the grant stands either way
r = _forgeext("list", REF_ID)
host_eff = set(((r.get("packages") or [{}])[0]).get("effective") or [])
ctx.check(host_eff == effective, "the panel's list and the host's own do not agree: %s vs %s",
sorted(effective), sorted(host_eff))
st, reply = act("hold-required")
ctx.check(st == 200 and os.listdir(REQUIRED_HOLDS) == [REF_ID] and listed()[1].get("hold") == "required",
"hold-required -> %s, named %s", st, os.listdir(REQUIRED_HOLDS))
st, reply = act("hold-advisory")
ctx.check(st == 200 and not os.listdir(REQUIRED_HOLDS), "hold-advisory -> %s, still named %s", st, os.listdir(REQUIRED_HOLDS))
st, reply = act("disable")
ctx.check(st == 200 and state().get("enabled") is False and listed()[1].get("enabled") is False,
"disable -> %s, the host's state says %s", st, state().get("enabled"))
st, reply = act("enable")
ctx.check(st == 200 and state().get("enabled") is True, "enable -> %s, the host's state says %s", st, state().get("enabled"))
before = _read(EXT_ROOT + "/state.json")
for name, form, want, words in (
("an action outside the list", {"id": REF_ID, "action": "install"}, 400, "action is enable"),
("an action with a shell's words", {"id": REF_ID, "action": "remove; reboot"}, 400, "action is enable"),
("an id that has not the form of one", {"id": "reference; reboot", "action": "disable"}, 400, "id is a package id"),
("an id with a path in it", {"id": "../../etc", "action": "remove"}, 400, "id is a package id"),
("a package that is not installed", {"id": "org.forgetest.nothere", "action": "disable"}, 409, "is not installed")):
st, reply = fc.post("/ext/package", data=form)
ev[name] = st
ctx.log("POST /ext/package, %s -> %s %s", name, st, reply if isinstance(reply, str) else "")
ctx.check(st == want and isinstance(reply, str) and words in reply, "%s -> %s %r, expected %s", name, st, reply, want)
ctx.check(_read(EXT_ROOT + "/state.json") == before, "a refused request changed the host's state")
st, body, hdrs = request(fc.base, "POST", "/ext/package", data={"id": REF_ID, "action": "remove"},
headers={"Host": fc.host_header()})
ctx.check(st == 403 and REF_ID in [x.get("id") for x in _forgeext("list").get("packages", [])],
"POST /ext/package without the login -> %s, expected 403 and the package still there", st)
st, reply = act("remove")
ctx.check(st == 200 and listed()[1] is None and not os.path.isdir(os.path.join(EXT_ROOT, "pkg", REF_ID))
and not os.path.isdir(os.path.join(EXT_ROOT, "data", REF_ID)), "remove -> %s, the package or its data still there", st)
finally:
if REF_ID in [x.get("id") for x in _forgeext("list").get("packages", [])]:
_forgeext("remove", REF_ID)
if os.path.exists(owner_key):
os.remove(owner_key)
shutil.rmtree(work, ignore_errors=True)
left = _tree(EXT_ROOT)
ctx.check(left == found_tree, "the extension root is not as found: %s", sorted(set(left) ^ set(found_tree)))
ctx.check(not os.listdir(REQUIRED_HOLDS), "a required hold is still named at the end: %s", os.listdir(REQUIRED_HOLDS))
forgetest/forgetest/suite/extwizard.py
-224
View File
@@ -1,224 +0,0 @@
# Copyright 2026 514 LLC d/b/a OpenGlow
# Written by Scott Wiederhold
# https://community.openglow.org
# SPDX-License-Identifier: MIT
"""A package's own check on the Setup page, on the machine.
Its own module, for the reason extcore.py gives. The package is the
reference package's id and key with a service of its own that runs the
check on its call socket, so exthost's put-back takes it away like the
others.
"""
import json
import os
import subprocess
from ..catalog import test
from .exthost import (EXT_ROOT, FWUP, REF_ID, REF_KEY, _as_found, _forgeext, _put_back, _svc, _tree, _until, _write)
from .setup import SAFETY_PHRASE, read_file, record_path, request
# The service: the check's steps on the call socket the host hands it (fd 4), its own record of whether the
# check passed, and every request it was asked, kept in its data directory.
WZ_SERVICE = r'''
import json, os, socket
data = os.environ["FFX_DATA"]
lst = socket.socket(fileno=int(os.environ["FFX_CALL_FD"]))
state = {"done": False, "calls": []}
while True:
c, _ = lst.accept()
try:
buf = b""
while b"\r\n\r\n" not in buf:
k = c.recv(4096)
if not k:
break
buf += k
head, _, body = buf.partition(b"\r\n\r\n")
n = 0
for line in head.split(b"\r\n")[1:]:
k, _, v = line.partition(b":")
if k.strip().lower() == b"content-length":
n = int(v)
while len(body) < n:
k = c.recv(4096)
if not k:
break
body += k
req = json.loads(body[:n] or b"{}")
method, path = head.split(b" ")[0].decode(), head.split(b" ")[1].decode()
if method == "GET" and path == "/wizard":
ans = {"title": "forgetest check", "done": state["done"]}
elif path == "/wizard/start":
ans = {"log": ["the check begins"], "phase": "asking", "progress": 50,
"prompt": {"kind": "confirm", "id": "ready", "text": "Is the machine ready?"}}
elif path == "/wizard/answer":
ok = req.get("id") == "ready" and req.get("answer") == "yes"
state["done"] = ok
ans = {"log": ["answered " + str(req.get("answer"))], "result": {"ok": ok, "summary": "ready" if ok else "not ready"}}
elif path == "/wizard/abort":
ans = {}
else:
ans = {"error": "no such call"}
state["calls"].append([method, path, req])
with open(os.path.join(data, "check.json.new"), "w") as f:
json.dump(state, f)
os.rename(os.path.join(data, "check.json.new"), os.path.join(data, "check.json"))
out = json.dumps(ans).encode()
c.sendall(b"HTTP/1.1 200 X\r\nContent-Type: application/json\r\nContent-Length: %d\r\nConnection: close\r\n\r\n"
% len(out) + out)
except Exception:
pass
finally:
c.close()
'''
def _pack_check(work):
import io
import tarfile
manifest = {"manifest": 1, "id": REF_ID, "name": "forgetest check", "version": "1.0.0",
"author": "forgetest", "license": "MIT", "api": "0.1", "runtime": "python",
"service": {"exec": "bin/check.py"}, "capabilities": ["wizard"]}
payload = os.path.join(work, "payload.tar.gz")
with tarfile.open(payload, "w:gz") as t:
for name, text, mode in (("manifest.json", json.dumps(manifest), 0o644), ("bin/check.py", WZ_SERVICE, 0o755)):
info = tarfile.TarInfo(name)
data = text.encode()
info.size, info.mode = len(data), mode
t.addfile(info, io.BytesIO(data))
conf = os.path.join(work, "fwup.conf")
_write(conf, 'meta-product = "ForgeFIRM extension"\nmeta-description = "%s"\nmeta-version = "1.0.0"\n'
'meta-platform = "forgefirm-ext"\nfile-resource payload.tar.gz {\n host-path = "%s"\n}\n'
% (REF_ID, payload))
key = os.path.join(work, REF_KEY)
raw, signed = os.path.join(work, "raw.ffx"), os.path.join(work, "check.ffx")
for cmd in ([FWUP, "-g", "-o", key], [FWUP, "-c", "-f", conf, "-o", raw],
[FWUP, "-S", "-s", key + ".priv", "-i", raw, "-o", signed]):
subprocess.run(cmd, check=True, capture_output=True, timeout=60, cwd=work)
return signed, key + ".pub"
@test("exthost.wizard", title="A package's own check runs on the Setup page, and the machine's setup is untouched",
subsystem="exthost", kind="auto", est_min=3,
covers=[("forgectrl", "src/wizpkg.*"), ("forgectrl", "src/wizdark.c"), ("forgectrl", "src/wiz.c"),
("forgectrl", "src/extpkg.*"), ("forgectrl", "src/ui/wizard.js"), ("forgeext", "src/main.c"),
("forgeext", "src/run.*"), ("forgeext", "src/caps.*"), ("forgeext", "src/manifest.*")],
requires=["exthost.service"],
description="A package that asks for wizard is installed and turned on; its service runs a check on its "
"call socket and keeps whether it passed. GET /wiz lists it among the extensions as "
"pkg:<id> with the title the package gives it, not done. POST /wiz/pkg:<id>/start runs it on "
"the wizard runner: GET /wiz/dark shows the package's log line, phase, progress, and its "
"confirm prompt, and the machine lease stays free while it waits (a package's check holds the "
"machine no more than the package does). The operator's Yes reaches the service as the answer "
"yes, the run ends complete with the package's summary, and GET /wiz then lists the check as "
"done, from the package's own answer. The machine's setup record is the same before and after: "
"no pkg: wizard in it and the gate as it was. A second run is aborted from its prompt and the "
"service is told. A check of a package that is not installed is 404. Everything is put back "
"as exthost.service puts it back.")
def wizard(ctx):
import shutil
import tempfile
fc = ctx.forgectrl
ev = ctx.evidence
ctx.check(fc.wait_idle(timeout=30, abort=ctx.aborted), "machine not idle")
prior = fc.settings().get("ext_enabled") or ""
raw = read_file(record_path())
found_tree = _tree(EXT_ROOT)
dir_mode = os.stat(os.path.dirname(EXT_ROOT)).st_mode & 0o7777
st, body, hdrs = request(fc.base, "GET", "/advisories/extensions", headers={"Host": fc.host_header()})
etag = hdrs.get("etag")
work = tempfile.mkdtemp(prefix="forgetest-ffx.")
owner_key = os.path.join(EXT_ROOT, "keys", REF_KEY + ".pub")
wid = "pkg:" + REF_ID
check_file = os.path.join(EXT_ROOT, "data", REF_ID, "check.json")
def own():
try:
return json.loads(read_file(check_file) or "{}")
except ValueError:
return {}
def listed():
st_, w = fc.get("/wiz")
return next((e for e in (w.get("extensions") or []) if e.get("id") == wid), None) if isinstance(w, dict) else None
def dark():
st_, d = fc.get("/wiz/dark")
return d if isinstance(d, dict) else {}
def record_view():
try:
rec = json.loads(read_file(record_path()) or "{}")
except ValueError:
return None
return {"wizards": sorted((rec.get("wizards") or {}).keys()), "completed": rec.get("completed")}
before = record_view()
gate_before = fc.get("/wiz")[1].get("gate")
try:
archive, pub = _pack_check(work)
shutil.copy(pub, owner_key)
os.chmod(owner_key, 0o644)
r = _forgeext("install", archive, "--consent-community")
ctx.check(r.get("ok") is True, "the install -> %s", r.get("error"))
st, reply = fc.post("/settings", data={"ext_enabled": "1", "advisory": etag, "phrase": SAFETY_PHRASE})
ctx.check(st == 200, "ext_enabled=1 -> %s %r", st, reply)
x = _until(ctx, lambda: _svc(REF_ID) if _svc(REF_ID).get("state") == "running" else None, 90, poll=0.5)
ctx.check(x, "the service is not running: %s", _svc(REF_ID))
e = _until(ctx, listed, 30, poll=1.0)
ev["listed"] = e
ctx.check(e and e.get("title") == "forgetest check" and e.get("done") is False,
"GET /wiz does not list the package's check, not done: %s", e)
st, body = fc.post("/wiz/%s/start" % wid)
ctx.check(st == 200, "POST /wiz/%s/start -> %s %s", wid, st, body)
d = _until(ctx, lambda: dark() if (dark().get("prompt") or {}).get("id") == "ready" else None, 20, poll=0.3)
ev["prompt"] = d
ctx.check(d and d.get("running") and d["prompt"].get("kind") == "confirm"
and any(l.endswith("the check begins") for l in (d.get("log") or [])) and d.get("progress") == 50,
"the check's prompt, log, and progress: %s", d)
lease = (fc.status().get("lease") or {}).get("holder")
ev["lease_while_waiting"] = lease
ctx.check(not lease, "a package's check holds the machine: %s", lease)
st, body = fc.post("/wiz/%s/answer" % wid, data={"seq": str(d["prompt"]["seq"]), "value": "Yes"})
ctx.check(st == 200, "the answer -> %s %s", st, body)
end = _until(ctx, lambda: dark() if not dark().get("running") else None, 20, poll=0.3)
ev["end"] = end
ctx.log("the check ended: %s", end)
ctx.check(end and not end.get("error") and (end.get("result") or {}).get("summary") == "ready",
"the check did not end complete with the package's summary: %s", end)
mine = own()
ctx.check(mine.get("done") is True and ["POST", "/wizard/answer", {"id": "ready", "answer": "yes"}] in
(mine.get("calls") or []), "the service was not answered yes: %s", mine)
ctx.sleep(11.0) # the Setup page's list is asked again at most every 10 s
e = listed()
ev["listed_after"] = e
ctx.check(e and e.get("done") is True, "GET /wiz does not list the check as done: %s", e)
after = record_view()
ev["record"] = {"before": before, "after": after}
ctx.check(after == before and not any(w.startswith("pkg:") for w in (after or {}).get("wizards", [])),
"the machine's setup record changed: %s -> %s", before, after)
ctx.check(fc.get("/wiz")[1].get("gate") == gate_before, "the setup gate moved")
# A second run, aborted from its prompt.
st, body = fc.post("/wiz/%s/start" % wid)
d = _until(ctx, lambda: dark() if (dark().get("prompt") or {}).get("id") == "ready" else None, 20, poll=0.3)
ctx.check(st == 200 and d, "the second run did not reach its prompt: %s %s", st, body)
fc.post("/wiz/%s/abort" % wid)
end = _until(ctx, lambda: dark() if not dark().get("running") else None, 20, poll=0.3)
ctx.check(end and end.get("error") == "aborted", "the abort: %s", end)
ctx.check(["POST", "/wizard/abort", {}] in (own().get("calls") or []), "the service was not told of the abort")
st, body = fc.post("/wiz/pkg:org.forgetest.nothere/start")
ctx.check(st == 404, "a check of a package that is not installed -> %s %s", st, body)
finally:
# A check that failed mid-run leaves the package's check waiting at its prompt.
if dark().get("running") and dark().get("id") == wid:
fc.post("/wiz/%s/abort" % wid)
_until(ctx, lambda: not dark().get("running"), 20, poll=0.3)
_put_back(ctx, fc, work, prior, etag, raw, dir_mode)
_as_found(ctx, fc, prior, raw, dir_mode, found_tree)