mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-09-27 08:41:13 -07:00
ScottW514
6c1d68f2c3
cloud.pause-resume failed a print that paused, resumed with its laser
lead, completed and parked: the test waited for the single line "button
pressed while paused; resuming", and the app has logged that as two
lines since its feeder work ("button pressed while paused", then
"resuming (laser lead N ticks)" from _resume_retraced). The replay
fixture carried the old wording, so the host test kept passing.
The pause and resume are now judged on PAUSE_LINES + RESUME_LINES
through one checker shared by the three tests that drive a pause
(cloud.pause-resume, the streamed pause, the pause-then-lid test), which
also fails on the app's "resume refused" line with the reason. The
fixture carries the app's two lines.
So the wording cannot drift silently again: tests/test_cloud_needles.py
reads every log phrase the cloud suite greps for out of cloud.py (the
left side of each `x in ln`, every wait_log needle, the mark tuples, and
the phrases it builds) and checks each against the logger calls in the
app sources at the revisions the recipes pin, read from the manifest
cache the tree manifest builds (the sibling checkouts locally),
placeholder-aware under a rule that never lets a placeholder stand for
the phrase itself. CI now builds the tree manifest before the unit
tests so the cache is there. The old needle fails that check.
Replays added: the second press seen but no retraced restart, and a
refused resume. 159 unit tests pass; coverage lint clean. No catalog
consequence beyond the suite module's own hash.
72 lines
2.5 KiB
YAML
72 lines
2.5 KiB
YAML
# forgetest: the release acceptance tool's host-side proof.
|
|
#
|
|
# - unit tests: campaign rules, fingerprints, artifact build/verify (the
|
|
# release gate's decision, including the negative fixtures), runner +
|
|
# HTTP API end to end with a fake catalog and a fake bench tool, the
|
|
# suites replayed on the machine's own log lines, and the check that
|
|
# every log phrase the cloud suite greps for is one the pinned cloud
|
|
# app can log (it reads the app sources from the manifest cache)
|
|
# - coverage lint: every source path of every component pinned by the
|
|
# recipes must be selected by some catalog test's coverage globs (the
|
|
# tree manifest is generated from the pins with git - no Yocto build);
|
|
# enforced: an uncovered path fails the job (docs/ACCEPTANCE.md).
|
|
|
|
name: forgetest-ci
|
|
|
|
on:
|
|
push:
|
|
paths:
|
|
- 'forgetest/**'
|
|
- 'scripts/acceptance-gate.py'
|
|
- 'scripts/manifest-from-tree.py'
|
|
- 'meta-forgefirm/recipes-forgefirm/**'
|
|
- '.github/workflows/forgetest-ci.yml'
|
|
pull_request:
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
steps:
|
|
- name: Checkout forgefirm
|
|
uses: actions/checkout@v4
|
|
with:
|
|
path: forgefirm
|
|
|
|
- name: Checkout meta-openglow (sibling)
|
|
uses: actions/checkout@v4
|
|
with:
|
|
repository: ScottW514/meta-openglow
|
|
ref: scarthgap
|
|
path: meta-openglow
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
python-version: '3.12'
|
|
|
|
- name: Tree manifest from the recipe pins (also fetches the pinned app sources the needle check reads)
|
|
working-directory: forgefirm
|
|
run: python scripts/manifest-from-tree.py --out tree-manifest.json
|
|
|
|
- name: Unit tests
|
|
working-directory: forgefirm/forgetest
|
|
run: python -m unittest discover -s tests -v
|
|
|
|
- name: Coverage lint (enforced)
|
|
working-directory: forgefirm/forgetest
|
|
run: python -m forgetest.coverage --manifest ../tree-manifest.json --enforce | tee "$GITHUB_STEP_SUMMARY"
|
|
|
|
- name: Gate self-check on the tree manifest
|
|
working-directory: forgefirm
|
|
run: |
|
|
# The gate must load the catalog and refuse a missing/unauthorized
|
|
# artifact cleanly (exit 1/2, never a traceback).
|
|
set +e
|
|
echo '{}' > empty.json
|
|
python scripts/acceptance-gate.py empty.json tree-manifest.json --quiet
|
|
rc=$?
|
|
set -e
|
|
test "$rc" -eq 1
|