Catalog: setup.mirror and logs.tree-tail-export leave; the catalog's address in forgetest-ci

Two catalog tests leave for the host proofs that carry them; the catalog
goes from 83 to 81 (81 @test in forgetest/forgetest/suite/).

- setup.mirror, for forgectrl's tests/host_ext.sh (0086e65): with a
  temporary account and two logins, a package's check that session A
  starts is owned and A's, owned and not B's, and the token's with no
  session; B's abort is 409 and the run goes on; B's takeover makes it
  B's; B's abort ends it aborted. Its section header, "the lifecycle",
  held only it. setup.check-sensors and setup.account-login stay.
- logs.tree-tail-export, for forgectrl's tests/host_guard.sh (0086e65):
  GET /logs names forgectrl, the tail carries a line just written, an
  unknown logger is 404, and the export carries README.txt, the
  installer's directory (a probe without its MAC or IP address, and
  install.log) and system/extensions.json, with neither the panel token
  nor the camera key in any member. _LOG_COVERS, the _PROBE_ constants,
  and the gzip, io, and tarfile imports, which only it read, go, and so
  does logs.routing's requires on it. logs.routing stays.

forgectrl.lease stays. Its proof passed (forgectrl's tests/host_lease.sh,
0086e65), but its deletion left forgectrl src/events.c and src/events.h
uncovered, a path outside the two this chunk may row, so it does not
land.

The coverage lint then named forgectrl src/sanitize.c, src/sanitize.h,
src/extpkg.c and src/extpkg.h, and nothing else. Two HOST_TESTED rows
take them, to forgectrl:tests/sanitize_test.c and
forgectrl:tests/extpkg_test.c on forgectrl:.github/workflows/build.yml,
which the pin c274f19 runs (./build/sanitize_test, ./build/extpkg_test).

forgetest-ci, after the tree-manifest step: the key comparison moves
there unchanged, and a step beside it checks the catalog's address.
From forgectrl-pin.inc's SRCREV it reads src/extpkg.h out of the
manifest cache (git show) and takes EXTPKG_INDEX_URL; from the
checked-out catalog's tools/publish.py it takes the basename of the file
passed to gh release create; the define must be exactly
https://github.com/openglow-org/forgefirm-extensions-catalog/releases/latest/download/
and that basename. A missing pin, define, or basename fails. Nothing
requests the address or fetches the asset. The step's run text, taken
from the workflow, passes from the tree root (c274f19's address and
index-1.ffi) and on a /tmp copy of the tree; controls on that copy,
publish.py passing index-2.ffi and extpkg.h naming another repository:
each fails the comparison. The key step's text is unchanged and was not
run again.

test_manifest.py (34) passes. The coverage lint on the tree manifest
finds no uncovered path and no host-tested row problem across the 81
tests; its one finding is setup.check-lens's empty src/lensfind.*
entry. pyflakes reports nothing on setup.py, logs.py or manifest.py.
The full forgetest suite was not run.

Fingerprints, measured against the parent: 23 tests move with
setup.py's and logs.py's shared text: the 5 left in setup.py, the 12 in
the modules that read setup.py (setup_dark.py's 9, setup_burns.py's 2,
bedsize.py's setup.check-envelope), the 5 left in exthost.py, which
reads it too, and logs.routing. The pending pin bumps from forgectrl
c274f19 and forgeext 2d0391e already owe 21 of them, and
exthost.platform has been owed since 30c0a05. This adds one:
setup.machine-name, which no pinned change selects. releases/ and
TEST-AUDIT.md are unchanged.

.github/workflows/forgetest-ci.yml

@@ -14,6 +14,10 @@
 #  - the extension key: the public key the image installs for the official
 #    tier is byte-identical to the one the extension catalog publishes
 #    (keys/forgefirm-ext.pub on its main branch)
+#  - the catalog's address: EXTPKG_INDEX_URL in the pinned forgectrl's
+#    src/extpkg.h is the catalog's latest-release download of the file its
+#    tools/publish.py passes to gh release create; read, never requested
+#    (every request of it is counted as a download)
 #  - coverage lint: every source path of every component pinned by the
 #    recipes must be selected by some catalog test's coverage globs or by a
 #    host-tested row naming the host test that proves it (the tree manifest
@@ -60,6 +64,15 @@ jobs:
           ref: main
           path: forgefirm-extensions-catalog
 
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Tree manifest from the recipe pins (also fetches the pinned app sources the needle check reads)
+        working-directory: forgefirm
+        run: python scripts/manifest-from-tree.py --out tree-manifest.json
+
       - name: The image's extension key is the catalog's published key
         run: |
           image=forgefirm/meta-forgefirm/recipes-forgefirm/forgefirm-keys/files/ext/forgefirm-ext.pub
@@ -69,14 +82,58 @@ jobs:
           done
           cmp -s "$image" "$published" || { echo "$image is not $published"; exit 1; }
 
-      - name: Set up Python
+      - name: The catalog's address in the pinned forgectrl is the file the catalog publishes (read, never requested)
-        uses: actions/setup-python@v5
+        run: |
-        with:
+          python3 - <<'EOF'
-          python-version: '3.12'
+          import ast, glob, os, re, subprocess, sys
 
-      - name: Tree manifest from the recipe pins (also fetches the pinned app sources the needle check reads)
+          PREFIX = "https://github.com/openglow-org/forgefirm-extensions-catalog/releases/latest/download/"
-        working-directory: forgefirm
+
-        run: python scripts/manifest-from-tree.py --out tree-manifest.json
+          def fail(why):
+              print(why)
+              sys.exit(1)
+
+          # forgectrl's pin, and its src/extpkg.h from the tree manifest's cache
+          pin = open("forgefirm/meta-forgefirm/recipes-forgefirm/forgectrl/forgectrl-pin.inc").read()
+          m = re.search(r'^SRCREV = "([0-9a-f]{40})"$', pin, re.M)
+          if not m:
+              fail("forgectrl-pin.inc names no SRCREV")
+          rev = m.group(1)
+          header = None
+          for repo in sorted(glob.glob("forgefirm/.manifest-cache/*")):
+              p = subprocess.run(["git", "-C", repo, "show", rev + ":src/extpkg.h"], capture_output=True, text=True)
+              if p.returncode == 0:
+                  header = p.stdout
+                  break
+          if header is None:
+              fail("forgectrl %s is not in forgefirm/.manifest-cache" % rev)
+          m = re.search(r'#define\s+EXTPKG_INDEX_URL\s*(?:\\\n\s*)?"([^"]*)"', header)
+          if not m:
+              fail("forgectrl %s's src/extpkg.h defines no EXTPKG_INDEX_URL" % rev[:7])
+          url = m.group(1)
+
+          # the file the catalog's publish tool passes to gh release create
+          tree = ast.parse(open("forgefirm-extensions-catalog/tools/publish.py").read())
+          assigned = {n.targets[0].id: n.value for n in ast.walk(tree)
+                      if isinstance(n, ast.Assign) and len(n.targets) == 1 and isinstance(n.targets[0], ast.Name)}
+
+          def file_name(v):
+              if isinstance(v, ast.Name):
+                  v = assigned.get(v.id)
+              if isinstance(v, ast.Call) and v.args:
+                  v = v.args[-1]
+              return os.path.basename(v.value) if isinstance(v, ast.Constant) and isinstance(v.value, str) else None
+
+          names = [file_name(n.args[3]) if len(n.args) > 3 else None for n in ast.walk(tree)
+                   if isinstance(n, ast.Call) and isinstance(n.func, ast.Name) and n.func.id == "gh"
+                   and [a.value for a in n.args[:2] if isinstance(a, ast.Constant)] == ["release", "create"]]
+          if len(names) != 1 or not names[0]:
+              fail("tools/publish.py passes no one file to gh release create: %s" % names)
+
+          print("forgectrl %s: %s; the catalog publishes %s" % (rev[:7], url, names[0]))
+          if url != PREFIX + names[0]:
+              fail("the address is not %s" % (PREFIX + names[0]))
+          EOF
 
       - name: Unit tests
         working-directory: forgefirm/forgetest

forgetest/forgetest/manifest.py

@@ -151,6 +151,14 @@ HOST_TESTED = [
     # The camera key: made, kept, checked and rotated (the reads it opens
     # from the LAN with the reads closed are tests/host_lan.sh).
     ("forgectrl", "src/camkey.*", "forgectrl:tests/camkey_test.c", "forgectrl:.github/workflows/build.yml"),
+    # The log export's sanitizer: every identifying class taken out, the
+    # ordinary log text left as it was (the export's routes and the
+    # installer's directory on a host are tests/host_guard.sh).
+    ("forgectrl", "src/sanitize.*", "forgectrl:tests/sanitize_test.c", "forgectrl:.github/workflows/build.yml"),
+    # The package routes' relay to the extension host: the argument vector,
+    # the closed list of actions, the host's words, the catalog's fetch (the
+    # routes against a live host's stand-in are tests/host_ext.sh).
+    ("forgectrl", "src/extpkg.*", "forgectrl:tests/extpkg_test.c", "forgectrl:.github/workflows/build.yml"),
 ]
 
 # This repository's root, when the package runs from a checkout of it; a

forgetest/forgetest/suite/logs.py

@@ -3,13 +3,10 @@
 # https://community.openglow.org
 # SPDX-License-Identifier:    MIT
 
-"""logs.* - unified logging: the log tree, tail, and the sanitized export;
+"""logs.* - unified logging: the routing path (emitters, relays, rendered
-the routing path (emitters, relays, rendered rules); the level settings."""
+rules); the level settings."""
-import gzip
-import io
 import os
 import re
-import tarfile
 import time
 
 from ..catalog import test
@@ -24,137 +21,7 @@ def _read(path, default=None):
         return default
 
 
-_LOG_COVERS = [("forgectrl", "src/logs.*"), ("forgectrl", "src/fflog.*"), ("forgectrl", "src/sanitize.*"),
-               ("forgectrl", "src/extpkg.*"), ("forgectrl", "src/main.c")]
-
 LOGS_ROOT = "/data/log/forgefirm"
-# What the export test plants in the installer's directory: a line to find
-# again, and two addresses (documentation ranges) the sanitizer must take.
-_PROBE_MARK = "forgetest export probe"
-_PROBE_MAC = "02:00:5e:10:20:30"
-_PROBE_IP = "192.0.2.77"
-
-
-@test("logs.tree-tail-export", title="Log tree, tail, and sanitized export", subsystem="logs",
-      kind="auto", est_min=1,
-      covers=_LOG_COVERS, requires=["forgectrl.auth"],
-      description="/logs lists the loggers with their levels and files, /logs/tail returns the "
-                  "forgectrl logger's tail, and POST /logs/export streams a sanitized tar.gz "
-                  "bundle that contains neither the panel token nor the camera key. The bundle "
-                  "also carries the installer's directory of the tree (logs/install/), which no "
-                  "logger feeds: a probe file planted there for the export comes back in the "
-                  "bundle with its addresses redacted, and an install log that the installer "
-                  "left is in the bundle too. system/extensions.json is in it as well: what is "
-                  "installed that is not the firmware, with each package's tier, whether it is "
-                  "enabled, and what the operator granted it, which is what a report about a "
-                  "machine running extensions has to be read against.")
-def tree_tail_export(ctx):
-    install_dir = os.path.join(LOGS_ROOT, "install")
-    probe = os.path.join(install_dir, "forgetest-probe.txt")
-    made_dir = not os.path.isdir(install_dir)
-    try:
-        os.makedirs(install_dir, exist_ok=True)
-        with open(probe, "w") as f:
-            f.write("%s from %s at %s\n" % (_PROBE_MARK, _PROBE_MAC, _PROBE_IP))
-        _tree_tail_export(ctx, probe)
-    finally:
-        try:
-            os.unlink(probe)
-        except OSError:
-            pass
-        if made_dir:
-            try:
-                os.rmdir(install_dir)
-            except OSError:
-                pass
-
-
-def _tree_tail_export(ctx, probe):
-    fc = ctx.forgectrl
-    ev = ctx.evidence
-    st, body = fc.get("/logs")
-    ctx.log("GET /logs -> %s", st)
-    ctx.check(st == 200 and isinstance(body, dict), "GET /logs -> %s", st)
-    loggers = body.get("loggers") or []
-    names = [l.get("name") for l in loggers] if isinstance(loggers, list) else list(loggers)
-    ev["loggers"] = names
-    ctx.log("loggers: %s", names)
-    ctx.check("forgectrl" in names, "/logs lacks the forgectrl logger: %s", names)
-
-    st, tail = fc.get("/logs/tail", params={"name": "forgectrl", "lines": "20"})
-    ctx.log("GET /logs/tail?name=forgectrl -> %s", st)
-    ctx.check(st == 200, "GET /logs/tail -> %s %s", st, tail if isinstance(tail, dict) else "")
-    st, tail = fc.get("/logs/tail", params={"name": "no-such-logger"})
-    ctx.check(st == 404, "unknown logger -> %s, expected 404", st)
-
-    # the sanitizer walks every log file; a full tree takes tens of
-    # seconds on the target, so this call gets its own client timeout
-    slow = hw.Forgectrl(token=fc.token, timeout=300.0)
-    t0 = time.time()
-    st, data = slow.post("/logs/export", raw=True)
-    ev["export_s"] = round(time.time() - t0, 1)
-    ctx.log("POST /logs/export -> %s (%d bytes, %.1f s)", st, len(data) if data else 0, ev["export_s"])
-    ctx.check(st == 200, "POST /logs/export -> %s", st)
-    ev["export_bytes"] = len(data)
-    try:
-        raw = gzip.decompress(data)
-        tf = tarfile.open(fileobj=io.BytesIO(raw))
-        members = tf.getnames()
-    except (OSError, tarfile.TarError, EOFError) as e:
-        ctx.fail("export is not a readable tar.gz: %s", e)
-    ev["members"] = len(members)
-    ctx.log("bundle: %d members, e.g. %s", len(members), members[:5])
-    ctx.check(members, "empty bundle")
-    ctx.check(any(m.endswith("README.txt") for m in members), "sanitized bundle lacks README.txt")
-    # the two secrets the sanitizer knows by value and a bundle could
-    # carry: the panel token and the camera key (a URL parameter that
-    # lands in sender logs)
-    secrets = {}
-    if fc.token:
-        secrets["panel token"] = fc.token
-    st, body = fc.get("/system/camera-key")
-    if st == 200 and isinstance(body, dict) and body.get("key"):
-        secrets["camera key"] = body["key"]
-    ctx.check("camera key" in secrets, "GET /system/camera-key -> %s: no key to check for", st)
-    contents = [(m.name, tf.extractfile(m).read()) for m in tf.getmembers() if m.isfile()]
-    for what, value in secrets.items():
-        leaked = [name for name, content in contents if value.encode() in content]
-        ev[what.replace(" ", "_") + "_leaks"] = leaked
-        ctx.check(not leaked, "the sanitized bundle contains the %s: %s", what, leaked)
-
-    # the installer's directory: no logger feeds it, the export carries it
-    by_name = dict(contents)
-    want = "logs/install/" + os.path.basename(probe)
-    got = [name for name in by_name if name.endswith(want)]
-    ev["install_members"] = sorted(name for name in by_name if "/logs/install/" in name)
-    ctx.check(got, "the bundle lacks %s: %s", want, ev["install_members"])
-    if got:
-        text = by_name[got[0]].decode("utf-8", "replace")
-        ctx.check(_PROBE_MARK in text, "the probe file came back without its line: %r", text)
-        for what, value in (("MAC address", _PROBE_MAC), ("IPv4 address", _PROBE_IP)):
-            ctx.check(value not in text, "the install directory left the sanitizer with its %s", what)
-    if os.path.isfile(os.path.join(os.path.dirname(probe), "install.log")):
-        ctx.check(any(name.endswith("logs/install/install.log") for name in by_name),
-                  "the machine has an install log and the bundle does not")
-
-    # What is installed that is not the firmware. A report that starts "it
-    # stopped working" reads differently with packages on the machine, and
-    # the bundle is where that is read.
-    import json
-
-    ext = [name for name in by_name if name.endswith("system/extensions.json")]
-    ctx.check(ext, "the bundle carries no system/extensions.json")
-    if ext:
-        doc = json.loads(by_name[ext[0]].decode("utf-8", "replace"))
-        ev["extensions"] = {"enabled": doc.get("enabled"), "safe_mode": doc.get("safe_mode"),
-                            "packages": [{k: p.get(k) for k in ("id", "version", "tier", "enabled", "grants")}
-                                         for p in doc.get("packages", [])]}
-        ctx.log("the bundle's extension listing: %s", ev["extensions"])
-        ctx.check(set(doc) >= {"enabled", "safe_mode", "host", "packages"},
-                  "the listing is not the status document: %s", sorted(doc))
-        for p in doc.get("packages", []):
-            ctx.check(set(p) >= {"id", "version", "tier", "enabled", "grants"},
-                      "a listed package does not say what it is: %s", sorted(p))
 
 
 # The routing test proves the whole path every logger takes: emitter (or
@@ -200,7 +67,7 @@ def _wait_line(path, needle, timeout=6.0):
 
 @test("logs.routing", title="Log routing, relays, and the rendered rules", subsystem="logs",
       kind="auto", est_min=1,
-      covers=_ROUTING_COVERS, requires=["logs.tree-tail-export"],
+      covers=_ROUTING_COVERS,
       description="rsyslog is the only logger (no busybox syslogd/klogd), the rules rendered from "
                   "the settings and the effective-levels record exist and agree with /logs, every "
                   "logger has its directory, the daemon's own emitter and the per-controller "

forgetest/forgetest/suite/setup.py

@@ -1068,101 +1068,3 @@ def first_run_page(ctx):
         finished_first_run(ctx)
         ctx.clear_notice()
         ctx.confirm("Did the control panel open after Start using the machine?")
-
-
-# ------------------------------------------------------- the lifecycle
-
-@test("setup.mirror", title="One browser drives a check; a second one follows",
-      subsystem="setup", kind="auto", est_min=2, hardware="takeover",
-      covers=[("forgectrl", "src/wizdark.*"), ("forgectrl", "src/wizcalc.*"), ("forgectrl", "src/wiz.*"),
-              ("forgectrl", "src/session.*"), ("forgectrl", "src/ui/wizard.js"),
-              ("forgectrl", "src/ui/wizard.html"), ("forgectrl", "src/ui/wizard.css")],
-      requires=["forgectrl.auth", "setup.account-login", "setup.check-sensors"],
-      description="With a temporary account (made and removed as the login test does) and two "
-                  "login sessions: the first session starts the sensors check; GET /wiz/dark "
-                  "reports the run as owned and its own to the first session, not to the second, "
-                  "and as its own to a tool with the token and no session; the second session's "
-                  "abort is refused (409); POST /wiz/sensors/takeover from the second session "
-                  "makes the run its own and the first session's no longer; the second session's "
-                  "abort then stops the check, which ends aborted. Both sessions are logged out; "
-                  "the real account comes back under a restart.")
-def mirror(ctx):
-    fc = ctx.forgectrl
-    ev = ctx.evidence
-    base = tls_base()
-    tok = {"X-ForgeFIRM-Token": fc.token}
-    name = "forgetest"
-    pw = "".join(random.choice("abcdefghjkmnpqrstuvwxyz23456789") for _ in range(16))
-    before = wiz(fc)
-    ev["account_before"] = (before.get("users") or {}).get("name")
-    ctx.check(before.get("acceptance_done") is True, "the advisories are not accepted: no check can start")
-    ctx.check(fc.wait_idle(timeout=30, abort=ctx.aborted), "machine not idle: the check cannot start")
-    d = decode(request(base, "GET", "/wiz/dark", headers=tok)[1])
-    ctx.check(isinstance(d, dict) and not d.get("running"), "a wizard is already running: %s", d)
-    homes = "/data/forgefirm/home"
-
-    def login():
-        st, body, hdrs = request(base, "POST", "/login", data={"name": name, "password": pw})
-        sid = session_from_cookie(hdrs.get("set-cookie") or "")
-        ctx.check(st == 200 and sid, "POST /login -> %s, no session", st)
-        return {"Cookie": "ffsid=" + (sid or "")}
-
-    def status(hd):
-        st, body, hdrs = request(base, "GET", "/wiz/dark", headers=hd)
-        return decode(body) if st == 200 else {}
-
-    with installed(ctx, {users_path(): None}):
-        st, body = fc.post("/wiz/account", data={"name": name, "password": pw})
-        ctx.check(st == 200, "the temporary account was not created: %s %s", st, body)
-        a = login()
-        b = login()
-        a.update(tok)
-        b.update(tok)
-        try:
-            st, body, hdrs = request(base, "POST", "/wiz/sensors/start", headers=a)
-            ctx.log("POST /wiz/sensors/start (session A) -> %s", st)
-            ctx.check(st == 200, "the sensors check did not start: %s %s", st, decode(body))
-            da, db, dt = status(a), status(b), status(tok)
-            ev["a_sees"] = {k: da.get(k) for k in ("running", "owned", "mine")}
-            ev["b_sees"] = {k: db.get(k) for k in ("running", "owned", "mine")}
-            ev["tool_sees"] = {k: dt.get(k) for k in ("running", "owned", "mine")}
-            ctx.check(da.get("running") and da.get("owned") and da.get("mine"),
-                      "session A does not own its run: %s", ev["a_sees"])
-            ctx.check(db.get("owned") and db.get("mine") is False, "session B is not a mirror: %s", ev["b_sees"])
-            ctx.check(dt.get("mine"), "a tool with the token reads as a mirror: %s", ev["tool_sees"])
-            st, body, hdrs = request(base, "POST", "/wiz/sensors/abort", headers=b)
-            ev["b_abort_refused"] = st
-            ctx.log("POST /wiz/sensors/abort (session B) -> %s %s", st, decode(body))
-            ctx.check(st == 409, "the mirror's abort -> %s, expected 409", st)
-            ctx.check(status(a).get("running"), "the mirror's abort stopped the run")
-            st, body, hdrs = request(base, "POST", "/wiz/sensors/takeover", headers=b)
-            ctx.log("POST /wiz/sensors/takeover (session B) -> %s", st)
-            ctx.check(st == 200, "the takeover -> %s", st)
-            da, db = status(a), status(b)
-            ev["after_takeover"] = {"a": da.get("mine"), "b": db.get("mine")}
-            ctx.check(db.get("mine") and da.get("mine") is False,
-                      "after the takeover B is not the owner or A still is: %s", ev["after_takeover"])
-            st, body, hdrs = request(base, "POST", "/wiz/sensors/abort", headers=b)
-            ctx.log("POST /wiz/sensors/abort (session B, owner) -> %s", st)
-            ctx.check(st == 200, "the owner's abort -> %s", st)
-        finally:
-            request(base, "POST", "/wiz/sensors/abort", headers=tok)
-            t0 = time.time()
-            while time.time() - t0 < 60 and status(tok).get("running"):
-                ctx.sleep(1)
-            d = status(tok)
-            ev["end"] = {"running": d.get("running"), "error": d.get("error")}
-            ctx.check(not d.get("running"), "the check did not stop within 60 s")
-            ctx.check((d.get("error") or "") == "aborted", "the check ended %r, expected aborted", d.get("error"))
-            for hd in (a, b):
-                request(base, "POST", "/logout", headers=hd)
-    rc, out = hw.initd("forgefirm-users", "reload")
-    ev["users_reload_rc"] = rc
-    if name != ev["account_before"]:
-        import shutil
-        shutil.rmtree(os.path.join(homes, name), ignore_errors=True)
-    after = wiz(fc)
-    ctx.check((after.get("users") or {}).get("name") == ev["account_before"],
-              "the account reads %r after the restore, was %r", (after.get("users") or {}).get("name"),
-              ev["account_before"])
-    ctx.log("A owned the run, B mirrored, B took over and stopped it; the account is back")