mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-10-17 18:11:42 -07:00
forgetest tests: the confirmed trims of the tool's own unit tests
The cases that tested a mock, a timer, a wording or a table, or that
replayed a judgment a host test already holds, leave forgetest/tests.
What catches a false PASS, keeps the bench safe, or guards the gate
stays.
- test_baseline: the fake sysfs tree is a mixin with no test_ methods,
so ExtensionFreeTests and BaselineModeTests no longer re-run
BaselineTests' 25 cases (133 runs, now 45). CoolPublishTests, the
lamp-fade cases and TransientNotLeftoverTests go; PositionDeadbandTests
keeps a reading past the band and an unreadable reading, both never
excused. Both latch-relock cases stay.
- test_fixture: MdnsTests, the three timing cases, the probe's log
wording, and nine prompt-routing variants go. A live test never runs
unattended, the arm-press opt-in, the short-press edge, the presence
press handing over the arm press, and the arm notice dropping at the
press stay (35 cases, now 21).
- test_cloud_suite: three false-PASS guards stay: the latch open at a
dark print's button, a lid stop that is not edge-driven, the kernel
refusing the resume. The guards for the latch unlocking in the hold
and a paused print resuming left with their catalog tests in 6a434bb.
The rest go, with the harness pieces and the hunt-lid fixture only
they read (41 cases, now 3).
- test_cooling_suite: GateOffTests, CriticalTierTests, FanGateTests and
FansQuietTests go with their harness; the flow verdict's log reads
stay: only the text after the offset, never an older verdict before it
(the flow-under-load false-PASS guard), a rotated log read whole.
- test_queue: RunClockTests and three ordering cases go, and refusals
are judged by a reason given, not its words; both live
acknowledgment cases stay.
- test_takeover_position: the six cases that judge the head and move
nothing stay.
- test_camera_home_return: the five cases where the head must not move
stay, and so do the three that check the hand-back jog's direction and
distance (homed, unhomed with the counters not counted twice, after
cloud mode): a wrong sign or a double count drives the head into the
gantry stops (15 cases, now 8).
- test_server: the keep-alive and ETag cases, the takeover and scope
half of the bench case, and the page's self-contained asserts
(test_page.py's) go.
- test_operator: the wording cases, the fixture case (test_fixture.py's)
and JournalTests go.
- test_mode: the supervisor timeout table goes.
- test_takeover_client: the listener, the offline service back, the
online client without the hunt, and GRBL under no marker stay.
- test_responsiveness: PageTests and ImplementationCostTests go; the
log cache keeps appends, a partial line, a replaced file and a fresh
list, and the fingerprint cache a changed manifest and one with no
content hash.
- test_bench_registry: GfbenchTests and two description and argument
pins go.
- test_installer: the log-format and TLS-wording cases and
InstallerShapeTests go; the die case checks the reason is in the log,
not its format.
- test_image: KernelIdentTests and the registration pin go.
Each file's __main__ guard now ends the file, and the docstrings say what
each still proves.
Proof: each of the fifteen files passes under Linux as forgetest-ci
discovers it, and pyflakes reports nothing new in them.
Acceptance: none; test-only.
This commit is contained in:
ScottW514
@@ -3,37 +3,22 @@
|
||||
# https://community.openglow.org
|
||||
# SPDX-License-Identifier: MIT
|
||||
|
||||
"""What keeps the page answering the operator instead of the timer.
|
||||
"""The caches behind the page's state.
|
||||
|
||||
Three things went wrong on the bench and are pinned here:
|
||||
|
||||
- the state cost. Every poll re-read and re-parsed the whole result log,
|
||||
and recomputed every test's domain fingerprint. A result record carries
|
||||
its run log, so the file reaches megabytes over a campaign and the poll
|
||||
grew with it. Both are now parsed and computed once.
|
||||
- the first state's cost. Hashing the implementations parsed a test's
|
||||
module and every module it imports once per test, and each poll the
|
||||
page timed out on started the same work again beside the first. Each
|
||||
module is now parsed once, and a poll waits for the work in progress.
|
||||
- the wasted payload. An idle page polls an unchanged state; it now gets
|
||||
a 304 instead of the whole thing.
|
||||
|
||||
The click-swallowing defect these were found with lives in the page's
|
||||
JavaScript and is not reachable from here: rows, prompt buttons and tool
|
||||
entries are built once and afterwards only updated in place, because a
|
||||
poll that rebuilt them removed the button between the operator's mousedown
|
||||
and mouseup, and no click event was ever raised. `test_page_never_rebuilds`
|
||||
holds the shape of that rule.
|
||||
Every poll used to re-read and re-parse the whole result log and recompute
|
||||
every test's domain fingerprint; a result record carries its run log, so
|
||||
the file reaches megabytes over a campaign. Both are now kept between
|
||||
polls, and a stale cache would misjudge the gate: the log cache picks up
|
||||
what is appended or replaced and never counts a partial line, and the
|
||||
fingerprint cache never serves a changed manifest or one it cannot key.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import helpers
|
||||
from forgetest import catalog, manifest as manifest_mod, page
|
||||
from forgetest import manifest as manifest_mod
|
||||
from forgetest.log import Log
|
||||
|
||||
|
||||
@@ -57,9 +42,6 @@ class LogCacheTests(unittest.TestCase):
|
||||
for line in lines:
|
||||
f.write(line + "\n")
|
||||
|
||||
def test_missing_file_reads_empty(self):
|
||||
self.assertEqual(self.log.read(), [])
|
||||
|
||||
def test_appends_are_picked_up(self):
|
||||
self.log.append({"t": "campaign", "id": "c1"})
|
||||
self.assertEqual([r["id"] for r in self.log.read()], ["c1"])
|
||||
@@ -69,27 +51,6 @@ class LogCacheTests(unittest.TestCase):
|
||||
self.assertEqual([r["t"] for r in recs], ["campaign", "result", "result"])
|
||||
self.assertEqual(recs[2]["test"], "c.d")
|
||||
|
||||
def test_only_new_bytes_are_parsed(self):
|
||||
for i in range(5):
|
||||
self.log.append({"t": "result", "test": "t%d" % i})
|
||||
self.log.read()
|
||||
# a second read must not touch the file at all
|
||||
real_open = open
|
||||
opened = []
|
||||
|
||||
def counting_open(*a, **kw):
|
||||
opened.append(a[0])
|
||||
return real_open(*a, **kw)
|
||||
|
||||
import builtins
|
||||
builtins.open = counting_open
|
||||
try:
|
||||
recs = self.log.read()
|
||||
finally:
|
||||
builtins.open = real_open
|
||||
self.assertEqual(opened, [], "an unchanged log was re-opened")
|
||||
self.assertEqual(len(recs), 5)
|
||||
|
||||
def test_read_returns_a_fresh_list(self):
|
||||
"""Callers filter and sort the result; the cache must not be theirs."""
|
||||
self.log.append({"t": "result", "test": "a.b"})
|
||||
@@ -110,16 +71,6 @@ class LogCacheTests(unittest.TestCase):
|
||||
self.assertEqual([r["test"] for r in recs], ["a.b", "c.d"])
|
||||
self.assertEqual(self.log.corrupt, 0)
|
||||
|
||||
def test_corrupt_lines_counted_once_not_per_read(self):
|
||||
self.write('{"t":"result","test":"a.b"}', "not json at all", '["not","a","record"]',
|
||||
'{"t":"result","test":"c.d"}')
|
||||
recs = self.log.read()
|
||||
self.assertEqual([r["test"] for r in recs], ["a.b", "c.d"])
|
||||
self.assertEqual(self.log.corrupt, 2)
|
||||
self.log.read()
|
||||
self.log.read()
|
||||
self.assertEqual(self.log.corrupt, 2, "corrupt lines re-counted on every read")
|
||||
|
||||
def test_replaced_file_is_read_again(self):
|
||||
self.write('{"t":"result","test":"a.b"}', '{"t":"result","test":"c.d"}')
|
||||
self.assertEqual(len(self.log.read()), 2)
|
||||
@@ -128,11 +79,6 @@ class LogCacheTests(unittest.TestCase):
|
||||
recs = self.log.read()
|
||||
self.assertEqual([r.get("id") for r in recs], ["c9"])
|
||||
|
||||
def test_blank_lines_are_not_corrupt(self):
|
||||
self.write('{"t":"result","test":"a.b"}', "", " ", '{"t":"result","test":"c.d"}')
|
||||
self.assertEqual(len(self.log.read()), 2)
|
||||
self.assertEqual(self.log.corrupt, 0)
|
||||
|
||||
|
||||
class FingerprintCacheTests(unittest.TestCase):
|
||||
"""Memoized per manifest: the page recomputes every test's fingerprint
|
||||
@@ -142,11 +88,6 @@ class FingerprintCacheTests(unittest.TestCase):
|
||||
self.man = helpers.make_manifest()
|
||||
self.t = helpers.make_test("fake.fp", [("forgectrl", "src/ui.c")], fn=t_noop)
|
||||
|
||||
def test_repeat_calls_agree(self):
|
||||
first = self.t.fingerprint(self.man)
|
||||
self.assertEqual(self.t.fingerprint(self.man), first)
|
||||
self.assertEqual(self.t.fingerprint(self.man), first)
|
||||
|
||||
def test_a_changed_manifest_is_not_served_from_the_cache(self):
|
||||
first = self.t.fingerprint(self.man)
|
||||
man2 = helpers.with_file(self.man, "forgectrl", "src/ui.c", "ui v2")
|
||||
@@ -154,11 +95,6 @@ class FingerprintCacheTests(unittest.TestCase):
|
||||
# and back again: the cache must not have latched the new one either
|
||||
self.assertEqual(self.t.fingerprint(self.man), first)
|
||||
|
||||
def test_a_file_outside_the_coverage_does_not_move_it(self):
|
||||
first = self.t.fingerprint(self.man)
|
||||
man2 = helpers.with_file(self.man, "forgectrl", "src/cool.c", "cool v2")
|
||||
self.assertEqual(self.t.fingerprint(man2), first)
|
||||
|
||||
def test_manifest_without_a_content_hash_is_not_cached(self):
|
||||
"""The cache is keyed by the manifest's content hash. A manifest
|
||||
that has none must still fingerprint correctly, not collide."""
|
||||
@@ -169,150 +105,6 @@ class FingerprintCacheTests(unittest.TestCase):
|
||||
self.assertIsNone(a.content_sha)
|
||||
self.assertNotEqual(self.t.fingerprint(a), self.t.fingerprint(b))
|
||||
|
||||
def test_component_file_list_is_stable_and_read_only(self):
|
||||
files = self.man.files("forgectrl")
|
||||
self.assertEqual(files, self.man.files("forgectrl"))
|
||||
self.assertIsNone(self.man.files("no-such-component"))
|
||||
|
||||
|
||||
class ImplementationCostTests(unittest.TestCase):
|
||||
"""The first state after a start hashes every test's implementation.
|
||||
It parses each suite module once, and a poll that arrives while it
|
||||
runs waits for it: on the board a parse per test took minutes, and
|
||||
the page's timed-out polls each started the work again beside it."""
|
||||
|
||||
def test_each_suite_module_is_parsed_once(self):
|
||||
import ast
|
||||
import inspect
|
||||
reg = catalog.load_suite()
|
||||
paths = sorted({inspect.getsourcefile(t.fn) for t in catalog.all_tests(reg)})
|
||||
for p in os.listdir(catalog.suite_dir()):
|
||||
catalog.forget(os.path.join(catalog.suite_dir(), p))
|
||||
for p in paths:
|
||||
catalog.forget(p)
|
||||
real_parse, parsed = ast.parse, []
|
||||
|
||||
def counting_parse(source, *a, **kw):
|
||||
parsed.append(source[:40])
|
||||
return real_parse(source, *a, **kw)
|
||||
|
||||
ast.parse = counting_parse
|
||||
try:
|
||||
for t in catalog.all_tests(reg):
|
||||
catalog.implementation_sha(inspect.getsourcefile(t.fn), t.id)
|
||||
finally:
|
||||
ast.parse = real_parse
|
||||
modules = [p for p in os.listdir(catalog.suite_dir()) if p.endswith(".py")]
|
||||
self.assertLessEqual(len(parsed), len(modules),
|
||||
"%d parses for %d suite modules" % (len(parsed), len(modules)))
|
||||
|
||||
def test_a_concurrent_fill_waits_instead_of_repeating(self):
|
||||
import threading
|
||||
import time
|
||||
t = helpers.make_test("fake.impl", [("forgectrl", "src/ui.c")], fn=t_noop)
|
||||
real, calls = catalog.implementation_sha, []
|
||||
|
||||
def slow(path, test_id):
|
||||
calls.append(test_id)
|
||||
time.sleep(0.3)
|
||||
return real(path, test_id)
|
||||
|
||||
catalog.implementation_sha = slow
|
||||
try:
|
||||
got = []
|
||||
threads = [threading.Thread(target=lambda: got.append(t.source_sha)) for _ in range(3)]
|
||||
for th in threads:
|
||||
th.start()
|
||||
for th in threads:
|
||||
th.join(10)
|
||||
finally:
|
||||
catalog.implementation_sha = real
|
||||
self.assertEqual(calls, ["fake.impl"], "the implementation hash was computed %d times" % len(calls))
|
||||
self.assertEqual(len(set(got)), 1)
|
||||
self.assertEqual(len(got), 3)
|
||||
|
||||
|
||||
class PageTests(unittest.TestCase):
|
||||
def test_page_never_rebuilds_what_the_operator_may_be_pressing(self):
|
||||
"""A poll must update rows, prompt buttons and tool entries in
|
||||
place. Assigning innerHTML to their containers on every poll is
|
||||
what swallowed the clicks; only the one-time build may do it."""
|
||||
html = page.render("0" * 32)
|
||||
for container, builder in (("groups", "buildGroups"), ("tools", "buildBench")):
|
||||
self.assertIn("function %s(" % builder, html,
|
||||
"no one-time builder for #%s" % container)
|
||||
assigns = html.count("$('%s').innerHTML=" % container)
|
||||
self.assertEqual(assigns, 1,
|
||||
"#%s is assigned innerHTML %d times; it belongs to the "
|
||||
"builder alone" % (container, assigns))
|
||||
# the per-poll path writes through the guarded setters only
|
||||
self.assertIn("function setHtml(e,h){if(e&&e.__h!==h)", html)
|
||||
self.assertIn("function updateGroups()", html)
|
||||
self.assertIn("function updateBench()", html)
|
||||
# the prompt buttons are rebuilt only when the prompt changes
|
||||
self.assertIn("if(pk!==promptKey)", html)
|
||||
# the queue controls are static markup: a poll relabels them and
|
||||
# flips disabled, it never replaces the node
|
||||
for bid in ("q-unattended", "q-attended", "q-stop"):
|
||||
self.assertIn('id="%s"' % bid, html)
|
||||
self.assertNotIn("id='" + bid + "-", html)
|
||||
self.assertNotIn('id="' + bid + "-", html)
|
||||
self.assertIn("function renderQueue()", html)
|
||||
# the help popovers sit on static markup only: a rebuilt row or
|
||||
# tool entry would orphan one
|
||||
self.assertNotIn("data-help", html[html.index("function buildGroups("):])
|
||||
|
||||
def test_every_group_is_built_on_the_same_grid(self):
|
||||
"""The subsystems are separate tables. Left to size themselves
|
||||
from their own content no two line up, which is what made the
|
||||
page look busy, so they share one colgroup and a fixed layout."""
|
||||
html = page.render("0" * 32)
|
||||
self.assertRegex(html, r"#groups table\s*\{\s*table-layout:\s*fixed")
|
||||
# one colgroup definition, emitted into every group's table
|
||||
self.assertEqual(html.count('var COLS="<colgroup>'), 1)
|
||||
self.assertEqual(html.count('<table>"+COLS+"'), 1,
|
||||
"a group table built without the shared colgroup")
|
||||
colgroup = re.search(r'var COLS="(.*?)";', html).group(1)
|
||||
heads = ["Test", "Kind", "Status", "Last result"]
|
||||
for h in heads:
|
||||
self.assertIn("<th>%s</th>" % h, html)
|
||||
self.assertEqual(len(re.findall(r"<col(?:>| )", colgroup)), len(heads) + 1,
|
||||
"a column per header, the action column included")
|
||||
# every fixed column carries a width, or the grid is only a wish
|
||||
for cls in re.findall(r"<col class='(\w)'", colgroup):
|
||||
self.assertRegex(html, r"#groups col\.%s\s*\{\s*width:\s*\d+px" % cls)
|
||||
|
||||
def test_details_and_the_requires_note_stay_out_of_the_columns(self):
|
||||
"""Both are long enough to stretch a cell. In a column they would
|
||||
pull one group's grid out of step with the rest: the details get a
|
||||
full-width row, and the note sits under Status, which is sized for
|
||||
it."""
|
||||
html = page.render("0" * 32)
|
||||
self.assertIn("<tr class='detrow'><td colspan='5'>", html)
|
||||
# the status cell carries the status, the unmet note and the row
|
||||
# message; the action cell carries the button and nothing else
|
||||
self.assertIn("<td><div id='st-\"+d+\"'></div><div id='unmet-\"+d+"
|
||||
"\"'></div><div id='note-\"+d+\"'></div></td>", html)
|
||||
self.assertIn("<td><button class='btn btn-sm btn-primary' id='btn-\"+d+\"' "
|
||||
"onclick='startTest(\\\"\"+d+\"\\\")'>Start</button></td>", html)
|
||||
|
||||
def test_page_is_self_contained_ascii(self):
|
||||
html = page.render("0" * 32)
|
||||
self.assertNotIn("__TOKEN__", html)
|
||||
# our own sources stay ASCII (no stray typography in an embedded
|
||||
# page); the vendored Bootstrap carries its own glyphs
|
||||
for name in ("index.html",) + page.CSS_FILES + page.JS_FILES:
|
||||
if name.startswith("vendor/"):
|
||||
continue
|
||||
page.read_ui(name).encode("ascii")
|
||||
stray = sorted(set(hex(ord(c)) for c in html if ord(c) < 32 and c != "\n"))
|
||||
self.assertEqual(stray, [], "control characters in the page source")
|
||||
# nothing fetched from anywhere: the documentation links open a
|
||||
# site, they are not assets
|
||||
for remote in ("<link ", "<script src=", "@import", "url(http", "url(//",
|
||||
'src="http', "src='http", "//cdn"):
|
||||
self.assertNotIn(remote, html)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user