No acceptance test requests a counted release file

update.job-locks started a real download in every campaign: POST
/update/download fetched the published release's forgefirm.fw, whose
GitHub download count is how installs are tracked. ext.catalog asked
POST /ext/catalog/refresh, which fetches the extension index's fixed
address, index.ffi, counted the same way.

update.job-locks now starts the download job with probe=1, forgectrl's
probe download: the release's acceptance.json through the download's
own path, refused by the signature check and discarded. The record is
small and the job can end within a second, so the test writes the
settings at once after the 202 (the lease is taken before it) and runs
the probe again, up to three times, when the job ended first. A posted
job is refused in the job's name beside it, /status names the holder
when it is read while the job runs, the job must end refused by the
signature check, and the staged download must be as found (size and
mtime) with no probe file left. Its covers gain forgectrl's
src/relcheck.*.

ext.catalog no longer asks the refresh. The refusals before a fetch
stay, and the index kept must be unchanged after them. The refresh
(curl, https alone and bounded, 502 in curl's words, 409 in the host's,
the file removed) is proven on the host by forgectrl's extpkg_test with
a stand-in for curl, and the fetch of a listed package and the tiers an
install takes from it by forgeext's install_test, as the description
now says.

Nothing else in the tree requests a counted release file: the
installer's own request of forgefirm.fw is an install, and its unit
test puts a stand-in curl first in PATH.

Proof: forgetest's unit tests pass on the host (474 OK, 4 skipped) and
pyflakes finds no undefined name.

Acceptance: the change is the two catalog tests themselves; their
fingerprints move, and update.job-locks now covers src/relcheck.* too.

forgetest/forgetest/suite/extcat.py

@@ -22,7 +22,6 @@ import tempfile
 
 from ..catalog import test
 from .exthost import FWUP, _forgeext, _write
-from .setup import request
 
 INDEX_URL = "https://github.com/openglow-org/forgefirm-extensions/releases/latest/download/index.ffi"
 STAGE_DIR = "/data/forgefirm/tmp"
@@ -83,11 +82,12 @@ def _staged():
                   "index (signed by a key that is not the OpenGlow extension key) is refused in words, a package "
                   "handed over as an index is refused by the product gate, and the index kept is left as it was. "
                   "POST /ext/catalog/get refuses an id with no such form (400), and one the kept index does not "
-                  "list (404, or 409 with no index kept), before anything is fetched. POST /ext/catalog/refresh "
+                  "list (404, or 409 with no index kept), before anything is fetched. Nothing is left in the "
-                  "asks the fixed https:// address: it keeps OpenGlow's index when one is published there, and "
+                  "staging directory. POST /ext/catalog/refresh is not asked here: GitHub counts every request "
-                  "when none is, it is 502 in curl's words with the kept index left as it was. Nothing is left in "
+                  "of the index's address as a download, and that count is the operators'. The refresh (curl, "
-                  "the staging directory. The fetch of a listed package, held to its size and SHA-256, and the "
+                  "https alone and bounded, 502 in curl's words, 409 in the host's, the file removed), the fetch "
-                  "tiers an install takes from it, are forgectrl's extpkg_test and forgeext's install_test.")
+                  "of a listed package held to its size and SHA-256, and the tiers an install takes from it, "
+                  "are forgectrl's extpkg_test and forgeext's install_test.")
 def catalog(ctx):
     fc = ctx.forgectrl
     ev = ctx.evidence
@@ -150,21 +150,7 @@ def catalog(ctx):
         want = (404, "not in the catalog") if kept.get("index") else (409, "fetch it first")
         ev["get_unlisted"] = [st, why]
         ctx.check(st == want[0] and want[1] in json.dumps(why), "an id the kept index does not list -> %s %s", st, why)
-
+        ctx.check(_forgeext("index").get("index") == kept.get("index"), "the refusals changed the index kept")
-        # The fetch from the one address.
-        st, body, _h = request(fc.base, "POST", "/ext/catalog/refresh", data={},
-                               headers={"X-ForgeFIRM-Token": fc.token, "Host": fc.host_header()}, timeout=90)
-        text = body.decode("utf-8", "replace")
-        ev["refresh"] = [st, text[:400]]
-        ctx.log("POST /ext/catalog/refresh -> %s %s", st, text[:200])
-        if st == 200:
-            doc = json.loads(text)
-            ctx.check(isinstance(doc.get("index"), dict) and doc["index"].get("version"),
-                      "a kept index has its version: %s", text[:200])
-        else:
-            ctx.check(st == 502 and "could not be fetched" in text, "a fetch that fails is 502 in curl's words: %s %s",
-                      st, text[:200])
-            ctx.check(_forgeext("index").get("index") == kept.get("index"), "a failed fetch changed the index kept")
         ctx.check(_staged() == staged_before, "the staging directory holds %s, and held %s before", _staged(), staged_before)
     finally:
         shutil.rmtree(work, ignore_errors=True)

forgetest/forgetest/suite/updlock.py

@@ -5,10 +5,12 @@
 
 """An update job holds the machine, on the machine.
 
-Its own module, so that no other test's fingerprint moves. The job is the
+Its own module, so that no other test's fingerprint moves. The job is a
-download of the published release: it fetches the firmware archive and
+probe download of the published release: the download job itself, run on
-checks its signature, and applies nothing. The archive it leaves is
+the release's acceptance record instead of its firmware file, because
-removed when it was not there before.
+GitHub counts every request of the firmware file as an install. The
+verification refuses the record and the probe keeps nothing; a download
+the machine had staged is left as it was.
 """
 
 import os
@@ -17,19 +19,33 @@ from ..catalog import test
 from .motion import _job_post, _words
 
 DL_FW = "/data/forgefirm/download.fw"
+DL_PROBE = "/data/forgefirm/download-probe"
+HELD = "an update job (download) holds the machine"
+
+
+def _as_found(path):
+    try:
+        st = os.stat(path)
+        return [st.st_size, st.st_mtime_ns]
+    except OSError:
+        return None
 
 
 @test("update.job-locks", title="An update job holds the machine: the settings are locked while it runs",
-      subsystem="update", kind="auto", est_min=3,
+      subsystem="update", kind="auto", est_min=2,
-      covers=[("forgectrl", "src/update.*"), ("forgectrl", "src/lease.*")],
+      covers=[("forgectrl", "src/update.*"), ("forgectrl", "src/relcheck.*"), ("forgectrl", "src/lease.*")],
       requires=["update.release-check"],
-      description="With a release published on the releases API, POST /update/download starts the download "
+      description="With a release published on the releases API, POST /update/download?probe=1 starts the "
-                  "job (202), which fetches the release's firmware archive and checks its signature and applies "
+                  "download job (202) on the release's acceptance record in place of its firmware file, so "
-                  "nothing. While it runs, /status names the job (update:download, of kind system) as the "
+                  "the test adds nothing to the firmware file's download count. The machine lease is taken "
-                  "machine lease's holder, and a settings write and a posted job are each refused in its name "
+                  "before the 202: a settings write and a posted job right after it are each refused in the "
-                  "(409, the settings locked). When it ends, the lease is free and the same settings write is "
+                  "job's name (409, an update job (download) holds the machine, the settings locked), and "
-                  "taken. The downloaded archive is removed when it was not there before. That a log export "
+                  "/status names the job (update:download, of kind system) as the holder while it runs. The "
-                  "does not lock the settings while every other holder does is lease_test's.")
+                  "job fetches the record through the download's own path and ends refused by the signature "
+                  "check with the record discarded. When it ends, the lease is free and the same settings "
+                  "write is taken. A download the machine had staged is untouched and the probe leaves no "
+                  "file. That a log export does not lock the settings while every other holder does is "
+                  "lease_test's; that a probe never names the firmware file is relcheck_test's.")
 def job_locks(ctx):
     fc = ctx.forgectrl
     ev = ctx.evidence
@@ -39,9 +55,11 @@ def job_locks(ctx):
         # the kept answer does not outlive a restart of the daemon: ask now
         st, rel = fc.post("/update/check")
         ctx.log("the release was not checked since the daemon started: POST /update/check -> %s", st)
-    ev["release"] = {k: (rel or {}).get(k) for k in ("available", "version", "bytes")} if isinstance(rel, dict) else rel
+    ev["release"] = {k: (rel or {}).get(k) for k in ("available", "version")} if isinstance(rel, dict) else rel
-    ctx.check(st == 200 and isinstance(rel, dict) and rel.get("available"), "no release is published to download: %s", rel)
+    ctx.check(st == 200 and isinstance(rel, dict) and rel.get("available"), "no release is published to probe: %s", rel)
-    had = os.path.exists(DL_FW)
+    staged = _as_found(DL_FW)
+    ev["staged_download"] = staged
+    ctx.check(_as_found(DL_PROBE) is None, "a probe's file is left over from before: %s", DL_PROBE)
     units = fc.settings().get("ui_units") or "metric"
 
     def holder():
@@ -52,37 +70,52 @@ def job_locks(ctx):
         return j if isinstance(j, dict) else {}
 
     try:
-        st, body = fc.post("/update/download")
+        # The record is small, so the job can end within a second. The
-        ctx.check(st == 202, "POST /update/download -> %s %s", st, _words(body)[:200])
+        # lease is taken before the 202, so the settings write goes first
-        ctx.wait_for(lambda: holder().get("owner") == "update:download" or None, 20, poll=0.2)
+        # and at once: it is the same value, so a probe that ended before
-        held = holder()
+        # it only costs another probe. The posted job goes only while the
-        ev["holder"] = held
+        # settings are refused in the job's name.
-        ctx.log("the lease while the download runs: %s", held)
+        refused = None
-        ctx.check(held.get("owner") == "update:download" and held.get("kind") == "system",
+        for attempt in (1, 2, 3):
-                  "the download job does not hold the machine: %s", held)
+            st, body = fc.post("/update/download", params={"probe": "1"})
-        refused = {}
+            ctx.check(st == 202, "POST /update/download probe=1 -> %s %s", st, _words(body)[:200])
             st, body = fc.post("/settings", params={"ui_units": units})
-        refused["a settings write"] = [st, _words(body)[:200]]
+            if st == 200:
+                ctx.wait_for(lambda: job().get("running") is False or None, 120, poll=0.5)
+                result = job().get("result") or {}
+                ctx.log("probe %d ended before the settings write was read (%s); again", attempt, result)
+                ctx.check("signature verification failed" in str(result.get("error")),
+                          "the probe ended, and not refused by the signature check: %s", result)
+                continue
             st2, body2 = _job_post(fc, "G21\n", name="beside-an-update")
-        refused["a posted job"] = [st2, _words(body2)[:200]]
+            held = holder()
+            refused = {"a settings write": [st, _words(body)[:200]], "a posted job": [st2, _words(body2)[:200]]}
+            break
+        ctx.check(refused is not None, "three probes each ended before a settings write could be refused")
         ev["refused"] = refused
-        ctx.log("beside the download: %s", refused)
+        ev["holder"] = held
-        ctx.check(job().get("running") is True, "the download ended before the refusals were read: %s", job())
+        ctx.log("beside the probe: %s; the lease: %s", refused, held)
-        ctx.check(st == 409 and "settings are locked" in _words(body), "a settings write beside the download -> %s %s",
+        ctx.check(st == 409 and HELD in _words(body) and "settings are locked" in _words(body),
-                  st, _words(body)[:200])
+                  "a settings write beside the probe -> %s %s", st, _words(body)[:200])
-        ctx.check(st2 == 409 and "holds the machine" in _words(body2), "a job beside the download -> %s %s",
+        ctx.check(st2 == 409 and HELD in _words(body2), "a job beside the probe -> %s %s", st2, _words(body2)[:200])
-                  st2, _words(body2)[:200])
+        if held:
-        ctx.wait_for(lambda: job().get("running") is False or None, 600, poll=2.0)
+            ctx.check(held.get("owner") == "update:download" and held.get("kind") == "system",
+                      "the probe does not hold the machine as the download job: %s", held)
+        else:
+            ctx.log("the probe had ended before /status was read; its refusals name it")
+        ctx.wait_for(lambda: job().get("running") is False or None, 600, poll=1.0)
         done = job()
         ev["job"] = done
-        ctx.log("the download ended: %s", done)
+        ctx.log("the probe ended: %s", done)
-        ctx.check(done and (done.get("result") or {}).get("ok") is True, "the download did not end well: %s", done)
+        result = done.get("result") or {}
+        ctx.check(result.get("ok") is False and "signature verification failed" in str(result.get("error")),
+                  "the probe did not end refused by the signature check (the record fetched and discarded): %s",
+                  result)
         ctx.check(not holder(), "the lease was not given back: %s", holder())
         st, body = fc.post("/settings", params={"ui_units": units})
         ctx.check(st == 200, "the same settings write after the job -> %s %s", st, _words(body)[:200])
     finally:
-        # The job runs on after a check that failed: its archive lands when it ends.
+        # The job runs on after a check that failed: its end is waited for.
         ctx.wait_for(lambda: job().get("running") is False or None, 600, poll=2.0)
-        if not had and os.path.exists(DL_FW):
+    ctx.check(_as_found(DL_FW) == staged, "the staged download moved: %s, and %s before", _as_found(DL_FW), staged)
-            os.remove(DL_FW)
+    ctx.check(_as_found(DL_PROBE) is None, "the probe left its file: %s", DL_PROBE)
-            ctx.log("removed the downloaded archive")