mirror of
https://github.com/openglow-org/forgefirm.git
synced 2026-09-27 08:41:13 -07:00
installer: a download that resumes and retries, and an install log
A field install failed on "firmware download failed", and worked after a reboot. The download was one bare curl -fL: no retry, no resume, no bound on a stalled transfer, and nothing on the machine recorded what had gone wrong. The download. download_fw makes up to five tries, 5, 15, 30 and 60 seconds apart. Each try resumes the partial file (curl -C -) and is bounded: 20 s to connect, and a transfer below 1 KB/s for 30 s ends the try. The file is written as forgefirm.fw.part and takes its name only when curl finished; the signature check that follows is what vouches for its content. A full disk (curl 23) and a release that is not there (HTTP 404) end the tries at once, because waiting cannot fix them. A partial file the server will not resume (curl 33 or 36, HTTP 416) starts over. The loop is the installer's own rather than curl --retry: the factory curl on the bench reference is 7.69.1, whose --retry does not count a resolver failure or a dropped transfer as retryable, and older factory builds carry older curls. The owner sees the reason in words with each retry, and the final failure says that a re-run goes straight to the download, because the archives are kept. The log. Every run appends to /data/log/forgefirm/install/install.log, in the log tree's own line format (UTC, program "install"): the installer's md5 (which revision ran), the factory version and the slots, the owner's answers, each archive, each download try with curl's exit code, the HTTP code and the reason, the machine's clock at each try (a wrong clock breaks TLS), and after a failed try the address, the default route, the resolver and whether github.com resolves; then the signature and identity checks, the write, the boot selection, and the reason for any failure through die(). The log is appended across runs, so the run that failed is still there after the run that worked. Logging never fails the install. forgectrl's log export carries the directory (forgectrl 0dae758). Proven: tests/test_installer.py runs the installer's own functions under sh against a scripted curl - a clean download, a resolver failure and a dropped transfer that resume to the full file, the tries running out, 404 and a full disk ending them at once, a stale partial file starting over, the TLS reason naming the clock, every log line in the tree format, die() leaving its reason, and an unwritable log not failing the run. The whole host suite, 409 tests, passes under Linux and the coverage lint is clean. Bench: the same functions under the factory firmware's own shell (busybox 1.31.1 ash, the factory slot of the bench reference in a chroot) resumed, retried, ran out of tries and logged exactly as under sh. Acceptance: logs.tree-tail-export now plants a probe file in the install directory and requires it back in the export bundle, its line intact and its MAC and IPv4 address redacted, and requires an install log in the bundle when the machine has one. The installer itself is not on the image: the install page fetches it from master, so it is live with this push.
This commit is contained in:
ScottW514
@@ -27,14 +27,46 @@ def _read(path, default=None):
|
||||
_LOG_COVERS = [("forgectrl", "src/logs.*"), ("forgectrl", "src/fflog.*"), ("forgectrl", "src/sanitize.*"),
|
||||
("forgectrl", "src/main.c")]
|
||||
|
||||
LOGS_ROOT = "/data/log/forgefirm"
|
||||
# What the export test plants in the installer's directory: a line to find
|
||||
# again, and two addresses (documentation ranges) the sanitizer must take.
|
||||
_PROBE_MARK = "forgetest export probe"
|
||||
_PROBE_MAC = "02:00:5e:10:20:30"
|
||||
_PROBE_IP = "192.0.2.77"
|
||||
|
||||
|
||||
@test("logs.tree-tail-export", title="Log tree, tail, and sanitized export", subsystem="logs",
|
||||
kind="auto", est_min=1,
|
||||
covers=_LOG_COVERS, requires=["forgectrl.auth"],
|
||||
description="/logs lists the loggers with their levels and files, /logs/tail returns the "
|
||||
"forgectrl logger's tail, and POST /logs/export streams a sanitized tar.gz "
|
||||
"bundle that contains neither the panel token nor the camera key.")
|
||||
"bundle that contains neither the panel token nor the camera key. The bundle "
|
||||
"also carries the installer's directory of the tree (logs/install/), which no "
|
||||
"logger feeds: a probe file planted there for the export comes back in the "
|
||||
"bundle with its addresses redacted, and an install log that the installer "
|
||||
"left is in the bundle too.")
|
||||
def tree_tail_export(ctx):
|
||||
install_dir = os.path.join(LOGS_ROOT, "install")
|
||||
probe = os.path.join(install_dir, "forgetest-probe.txt")
|
||||
made_dir = not os.path.isdir(install_dir)
|
||||
try:
|
||||
os.makedirs(install_dir, exist_ok=True)
|
||||
with open(probe, "w") as f:
|
||||
f.write("%s from %s at %s\n" % (_PROBE_MARK, _PROBE_MAC, _PROBE_IP))
|
||||
_tree_tail_export(ctx, probe)
|
||||
finally:
|
||||
try:
|
||||
os.unlink(probe)
|
||||
except OSError:
|
||||
pass
|
||||
if made_dir:
|
||||
try:
|
||||
os.rmdir(install_dir)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _tree_tail_export(ctx, probe):
|
||||
fc = ctx.forgectrl
|
||||
ev = ctx.evidence
|
||||
st, body = fc.get("/logs")
|
||||
@@ -87,6 +119,21 @@ def tree_tail_export(ctx):
|
||||
ev[what.replace(" ", "_") + "_leaks"] = leaked
|
||||
ctx.check(not leaked, "the sanitized bundle contains the %s: %s", what, leaked)
|
||||
|
||||
# the installer's directory: no logger feeds it, the export carries it
|
||||
by_name = dict(contents)
|
||||
want = "logs/install/" + os.path.basename(probe)
|
||||
got = [name for name in by_name if name.endswith(want)]
|
||||
ev["install_members"] = sorted(name for name in by_name if "/logs/install/" in name)
|
||||
ctx.check(got, "the bundle lacks %s: %s", want, ev["install_members"])
|
||||
if got:
|
||||
text = by_name[got[0]].decode("utf-8", "replace")
|
||||
ctx.check(_PROBE_MARK in text, "the probe file came back without its line: %r", text)
|
||||
for what, value in (("MAC address", _PROBE_MAC), ("IPv4 address", _PROBE_IP)):
|
||||
ctx.check(value not in text, "the install directory left the sanitizer with its %s", what)
|
||||
if os.path.isfile(os.path.join(os.path.dirname(probe), "install.log")):
|
||||
ctx.check(any(name.endswith("logs/install/install.log") for name in by_name),
|
||||
"the machine has an install log and the bundle does not")
|
||||
|
||||
|
||||
# The routing test proves the whole path every logger takes: emitter (or
|
||||
# relay) -> /dev/log -> rsyslog rules rendered from the settings -> the
|
||||
@@ -100,8 +147,7 @@ _ROUTING_COVERS = [("forgectrl", "src/logs.*"), ("forgectrl", "src/fflog.*"), ("
|
||||
("forgefirm-app", "forgefirm-app/ffmachine.py"), ("forgefirm-app", "forgefirm-app/gfcloud.py"),
|
||||
("forgefirm-app", "forgefirm-app/gfhome.py")]
|
||||
|
||||
LOGS_ROOT = "/data/log/forgefirm"
|
||||
LOGGERS = ("forgectrl", "grblhal", "gfcloud", "gfhome", "kernel", "system")
|
||||
LOGGERS =("forgectrl", "grblhal", "gfcloud", "gfhome", "kernel", "system")
|
||||
_LINE_RE = re.compile(r"^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(\.\d+)?[+-]\d\d:\d\d (?P<prog>[A-Za-z0-9_.-]+)\[(?P<pid>[-\d]+)\] "
|
||||
r"(?P<sev>EMERG|ALERT|CRIT|ERR|WARNING|NOTICE|INFO|DEBUG) (?P<msg>.*)$")
|
||||
_SEV_RANK = {"off": -1, "error": 3, "warning": 4, "notice": 5, "info": 6, "debug": 7}
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
# Copyright 2026 514 LLC d/b/a OpenGlow
|
||||
# Written by Scott Wiederhold
|
||||
# https://community.openglow.org
|
||||
# SPDX-License-Identifier: MIT
|
||||
|
||||
"""scripts/install-forgefirm.sh - the download that retries, and the log.
|
||||
|
||||
The installer runs once per machine, on factory firmware, over whatever
|
||||
network the owner has. These tests run its own functions under sh with a
|
||||
scripted curl in front of them: a transfer that fails resumes and is
|
||||
tried again, the failures that waiting cannot fix end the tries at once,
|
||||
and every step leaves a line in the install log in the log tree's own
|
||||
format."""
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import helpers # noqa: F401 (sys.path)
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
SCRIPT = os.path.join(REPO, "scripts", "install-forgefirm.sh")
|
||||
MARK = "# The install log, before anything can fail."
|
||||
|
||||
# The log tree's line format (forgetest/suite/logs.py keeps the same one).
|
||||
LINE_RE = re.compile(r"^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(\.\d+)?[+-]\d\d:\d\d install\[\d+\] "
|
||||
r"(EMERG|ALERT|CRIT|ERR|WARNING|NOTICE|INFO|DEBUG) .+$")
|
||||
|
||||
# A curl that plays a script: each call takes the next "rc http bytes"
|
||||
# line of $CURL_PLAN, appends that many bytes to the --output file the way
|
||||
# a resumed transfer does, prints the HTTP code as -w '%{http_code}'
|
||||
# would, records its arguments, and exits rc.
|
||||
FAKE_CURL = r"""#!/bin/sh
|
||||
echo "$*" >> "$CURL_CALLS"
|
||||
OUT=""
|
||||
while [ $# -gt 0 ]; do
|
||||
[ "$1" = "--output" ] && OUT="$2"
|
||||
shift
|
||||
done
|
||||
LINE=$(sed -n 1p "$CURL_PLAN")
|
||||
sed -i 1d "$CURL_PLAN"
|
||||
set -- $LINE
|
||||
[ "${3:-0}" -gt 0 ] && dd if=/dev/zero bs=1 count="$3" 2>/dev/null | tr '\0' 'x' >> "$OUT"
|
||||
printf '%s' "$2"
|
||||
exit "$1"
|
||||
"""
|
||||
|
||||
|
||||
def functions():
|
||||
"""The installer up to its first action: constants and functions."""
|
||||
with open(SCRIPT, encoding="utf-8") as f:
|
||||
text = f.read()
|
||||
head, mark, _ = text.partition(MARK)
|
||||
assert mark, "the installer lost the line the tests cut it at"
|
||||
return head
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which("sh") and os.name == "posix", "needs a POSIX sh")
|
||||
class InstallerTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp(prefix="ffinstall-test.")
|
||||
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
|
||||
self.bin = os.path.join(self.tmp, "bin")
|
||||
os.mkdir(self.bin)
|
||||
for name, body in (("curl", FAKE_CURL), ("nslookup", "#!/bin/sh\nexit 1\n"),
|
||||
("ip", "#!/bin/sh\nexit 0\n"), ("sleep", "#!/bin/sh\nexit 0\n")):
|
||||
path = os.path.join(self.bin, name)
|
||||
with open(path, "w", newline="\n") as f:
|
||||
f.write(body)
|
||||
os.chmod(path, os.stat(path).st_mode | stat.S_IXUSR)
|
||||
self.fw = os.path.join(self.tmp, "forgefirm.fw")
|
||||
self.log = os.path.join(self.tmp, "log", "install.log")
|
||||
self.calls = os.path.join(self.tmp, "curl.calls")
|
||||
self.plan = os.path.join(self.tmp, "curl.plan")
|
||||
|
||||
def run_sh(self, body, plan=()):
|
||||
with open(self.plan, "w", newline="\n") as f:
|
||||
f.write("".join("%s\n" % line for line in plan))
|
||||
open(self.calls, "w").close()
|
||||
script = "%s\nFW_FILE='%s'\nLOG_DIR='%s'\nLOG_FILE='%s'\nmkdir -p \"$LOG_DIR\"\nLOG_OK=yes\n%s\n" % (
|
||||
functions(), self.fw, os.path.dirname(self.log), self.log, body)
|
||||
env = dict(os.environ, PATH=self.bin + os.pathsep + os.environ["PATH"],
|
||||
CURL_PLAN=self.plan, CURL_CALLS=self.calls)
|
||||
return subprocess.run(["sh", "-c", script], env=env, capture_output=True, text=True, timeout=60)
|
||||
|
||||
def download(self, plan):
|
||||
r = self.run_sh('download_fw; echo "rc=$? why=$DL_WHY"', plan)
|
||||
with open(self.calls) as f:
|
||||
calls = f.read().splitlines()
|
||||
return r.stdout, calls, self.log_lines()
|
||||
|
||||
def log_lines(self):
|
||||
try:
|
||||
with open(self.log) as f:
|
||||
return f.read().splitlines()
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
def test_a_clean_download_takes_one_try_and_its_name(self):
|
||||
out, calls, log = self.download(["0 200 5000"])
|
||||
self.assertIn("rc=0", out)
|
||||
self.assertEqual(len(calls), 1)
|
||||
self.assertEqual(os.path.getsize(self.fw), 5000)
|
||||
self.assertFalse(os.path.exists(self.fw + ".part"))
|
||||
self.assertTrue(any("complete on try 1, 5000 bytes" in line for line in log), log)
|
||||
|
||||
def test_a_failed_transfer_resumes_and_is_tried_again(self):
|
||||
out, calls, log = self.download(["6 000 0", "56 200 3000", "0 206 2000"])
|
||||
self.assertIn("rc=0", out)
|
||||
self.assertEqual(len(calls), 3)
|
||||
for c in calls: # every try resumes, bounded in time
|
||||
self.assertIn("-C -", c)
|
||||
self.assertIn("--connect-timeout", c)
|
||||
self.assertIn("--speed-time", c)
|
||||
self.assertEqual(os.path.getsize(self.fw), 5000) # 3000 kept, 2000 resumed
|
||||
warns = [line for line in log if " WARNING download: try " in line]
|
||||
self.assertEqual(len(warns), 2, log)
|
||||
self.assertIn("did not resolve", warns[0])
|
||||
self.assertIn("dropped mid-transfer (3000 bytes so far)", warns[1])
|
||||
self.assertIn("Trying again in 5s (try 2 of 5)", out)
|
||||
self.assertIn("Trying again in 15s (try 3 of 5)", out)
|
||||
|
||||
def test_the_tries_run_out(self):
|
||||
out, calls, log = self.download(["28 000 0"] * 5)
|
||||
self.assertIn("rc=1", out)
|
||||
self.assertIn("timed out or stalled", out)
|
||||
self.assertEqual(len(calls), 5)
|
||||
self.assertFalse(os.path.exists(self.fw))
|
||||
self.assertFalse(os.path.exists(self.fw + ".part"))
|
||||
# each failed try leaves what the network looked like
|
||||
self.assertEqual(sum("github.com does not resolve" in line for line in log), 5, log)
|
||||
|
||||
def test_a_missing_release_and_a_full_disk_end_the_tries_at_once(self):
|
||||
out, calls, _ = self.download(["22 404 0"])
|
||||
self.assertIn("rc=1", out)
|
||||
self.assertIn("HTTP 404", out)
|
||||
self.assertEqual(len(calls), 1)
|
||||
out, calls, _ = self.download(["23 200 100"])
|
||||
self.assertIn("rc=1", out)
|
||||
self.assertIn("could not be written", out)
|
||||
self.assertEqual(len(calls), 1)
|
||||
|
||||
def test_a_partial_file_the_server_will_not_resume_starts_over(self):
|
||||
out, calls, _ = self.download(["56 200 4000", "22 416 0", "0 200 5000"])
|
||||
self.assertIn("rc=0", out)
|
||||
self.assertEqual(len(calls), 3)
|
||||
self.assertEqual(os.path.getsize(self.fw), 5000) # not 9000: the stale part went
|
||||
|
||||
def test_a_tls_failure_names_the_clock(self):
|
||||
out, _, _ = self.download(["60 000 0"] * 5)
|
||||
self.assertRegex(out, r"TLS handshake failed \(a wrong clock does this: \d{4}-\d\d-\d\d")
|
||||
|
||||
def test_every_log_line_is_in_the_tree_format(self):
|
||||
self.download(["7 000 0", "0 200 10"])
|
||||
self.run_sh('log NOTICE "operator declined to continue; no changes made"')
|
||||
lines = self.log_lines()
|
||||
self.assertGreater(len(lines), 5)
|
||||
for line in lines:
|
||||
self.assertRegex(line, LINE_RE)
|
||||
|
||||
def test_die_leaves_the_reason_in_the_log(self):
|
||||
r = self.run_sh('die "archiving slot 1 failed"; echo not-reached')
|
||||
self.assertEqual(r.returncode, 1)
|
||||
self.assertNotIn("not-reached", r.stdout)
|
||||
self.assertTrue(any(" ERR install failed: archiving slot 1 failed" in line
|
||||
for line in self.log_lines()))
|
||||
|
||||
def test_no_writable_log_never_fails_the_install(self):
|
||||
r = self.run_sh('LOG_OK=""; log INFO "x"; LOG_OK=yes; LOG_FILE=/nonexistent/dir/x.log; '
|
||||
'log INFO "y"; echo "still-running rc=$?"')
|
||||
self.assertIn("still-running rc=0", r.stdout)
|
||||
|
||||
|
||||
class InstallerShapeTests(unittest.TestCase):
|
||||
"""What holds on any host, with no shell to run."""
|
||||
|
||||
def test_the_log_lives_in_the_tree_the_export_carries(self):
|
||||
head = functions()
|
||||
self.assertIn('LOG_DIR="/data/log/forgefirm/install"', head)
|
||||
self.assertIn('LOG_FILE="$LOG_DIR/install.log"', head)
|
||||
|
||||
def test_the_download_is_the_retrying_one(self):
|
||||
with open(SCRIPT, encoding="utf-8") as f:
|
||||
text = f.read()
|
||||
body = text.partition(MARK)[2]
|
||||
self.assertIn("download_fw", body)
|
||||
self.assertNotRegex(body, r"(?m)^\s*curl ") # no bare curl past the functions
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -14,12 +14,19 @@
|
||||
# the archive directory; the active factory slot stays bootable.
|
||||
#
|
||||
# Usage: install-forgefirm.sh [local-forgefirm.fw]
|
||||
# With no argument the latest release .fw is downloaded from GitHub.
|
||||
# With no argument the latest release .fw is downloaded from GitHub. The
|
||||
# download resumes and retries; every run appends its steps to the
|
||||
# install log in the ForgeFIRM log tree, which the panel's log export
|
||||
# carries.
|
||||
|
||||
RELEASE_FW_URL="https://github.com/openglow-org/forgefirm/releases/latest/download/forgefirm.fw"
|
||||
ARCHIVE_DIR="/data/forgefirm/archive"
|
||||
FW_FILE="/data/forgefirm/forgefirm.fw"
|
||||
MIN_DATA_FREE_KB=300000
|
||||
LOG_DIR="/data/log/forgefirm/install"
|
||||
LOG_FILE="$LOG_DIR/install.log"
|
||||
DL_ATTEMPTS=5 # download tries before the install gives up
|
||||
DL_DELAYS="5 15 30 60" # seconds to wait before tries 2, 3, 4, 5
|
||||
|
||||
# ForgeFIRM release-signing public key (raw 32-byte Ed25519, the format the
|
||||
# factory's fwup 0.14.2 expects).
|
||||
@@ -31,7 +38,19 @@ BRIGHT="\033[1;39m"
|
||||
RESET="\033[0m"
|
||||
ASTERISK="${LIGHTRED}✺${RESET}"
|
||||
|
||||
# log <SEVERITY> <message>: one line in the log tree's own format
|
||||
# (timestamp, program[pid], severity, message). The log is appended
|
||||
# across runs, so a run that failed is still on the machine after the run
|
||||
# that worked. Logging never fails the install: with no writable log the
|
||||
# lines go nowhere.
|
||||
log () {
|
||||
[ -n "$LOG_OK" ] || return 0
|
||||
{ echo "$(date -u '+%Y-%m-%dT%H:%M:%S+00:00') install[$$] $1 $2" >> "$LOG_FILE"; } 2>/dev/null
|
||||
return 0
|
||||
}
|
||||
|
||||
die () {
|
||||
log ERR "install failed: $1"
|
||||
echo
|
||||
echo -e "${LIGHTRED}!! INSTALL FAILED:${RESET} $1"
|
||||
echo -e "${LIGHTRED}!! No boot change was made unless stated otherwise. Fix and re-run.${RESET}"
|
||||
@@ -190,6 +209,82 @@ set_env () {
|
||||
env_verify "$1" "$2" "$3" "$4"
|
||||
}
|
||||
|
||||
# curl_why <exit-code>: why a download attempt failed, in words.
|
||||
curl_why () {
|
||||
case "$1" in
|
||||
5|6) echo "the host name did not resolve (DNS)" ;;
|
||||
7) echo "the connection was refused or unreachable" ;;
|
||||
18|52|55|56) echo "the connection dropped mid-transfer" ;;
|
||||
22) echo "the server answered HTTP $2" ;;
|
||||
23) echo "the file could not be written" ;;
|
||||
28) echo "the connection timed out or stalled" ;;
|
||||
35|51|60) echo "the TLS handshake failed (a wrong clock does this: $(date -u '+%Y-%m-%d %H:%M') UTC)" ;;
|
||||
*) echo "curl error $1" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# net_snapshot: what the network looked like when an attempt failed - the
|
||||
# address, the default route, the resolver, whether the release host
|
||||
# resolves - into the install log only.
|
||||
net_snapshot () {
|
||||
log INFO "net: wlan0 $(ip -4 addr show dev wlan0 2>/dev/null | sed -n 's/^ *inet \([^ ]*\).*/\1/p' | head -n 1)"
|
||||
log INFO "net: route $(ip route 2>/dev/null | grep '^default' | head -n 1)"
|
||||
log INFO "net: resolver $(grep '^nameserver' /etc/resolv.conf 2>/dev/null | tr '\n' ' ')"
|
||||
if nslookup github.com >/dev/null 2>&1; then
|
||||
log INFO "net: github.com resolves"
|
||||
else
|
||||
log WARNING "net: github.com does not resolve"
|
||||
fi
|
||||
}
|
||||
|
||||
# download_fw: fetch the release into $FW_FILE. A home network drops, a
|
||||
# resolver hiccups, a transfer stalls: each try resumes the partial file
|
||||
# where the last one stopped, and the tries are spaced out. The file
|
||||
# lands as <file>.part and takes its name only when curl finished, and
|
||||
# the signature check that follows is what vouches for its content. Two
|
||||
# failures end the tries at once, because waiting cannot fix them: a full
|
||||
# disk, and a release that is not there (HTTP 404).
|
||||
download_fw () {
|
||||
PART="$FW_FILE.part"
|
||||
rm -f "$PART"
|
||||
TRY=1
|
||||
set -- $DL_DELAYS
|
||||
while :; do
|
||||
log INFO "download: try $TRY of $DL_ATTEMPTS, clock $(date -u '+%Y-%m-%d %H:%M:%S') UTC"
|
||||
HTTP=$(curl -fL -C - --connect-timeout 20 --speed-limit 1024 --speed-time 30 \
|
||||
-w '%{http_code}' --output "$PART" "$RELEASE_FW_URL")
|
||||
RC=$?
|
||||
if [ "$RC" = "0" ]; then
|
||||
mv "$PART" "$FW_FILE" || return 1
|
||||
log INFO "download: complete on try $TRY, $(wc -c < "$FW_FILE") bytes"
|
||||
return 0
|
||||
fi
|
||||
WHY=$(curl_why "$RC" "$HTTP")
|
||||
SOFAR=0
|
||||
[ -f "$PART" ] && SOFAR=$(wc -c < "$PART")
|
||||
log WARNING "download: try $TRY failed: curl exit $RC, HTTP ${HTTP:-none}: $WHY ($SOFAR bytes so far)"
|
||||
net_snapshot
|
||||
# A partial file the server will not resume (no range support, or a
|
||||
# range past its end) starts over rather than failing every try.
|
||||
case "$RC:$HTTP" in 33:*|36:*|22:416) rm -f "$PART" ;; esac
|
||||
if [ "$RC" = "23" ] || [ "$RC:$HTTP" = "22:404" ] || [ "$TRY" -ge "$DL_ATTEMPTS" ]; then
|
||||
rm -f "$PART"
|
||||
DL_WHY="$WHY"
|
||||
return 1
|
||||
fi
|
||||
WAIT=${1:-60}; [ $# -gt 0 ] && shift
|
||||
echo -e "${YELLOW}!! Download failed: $WHY.${RESET} Trying again in ${WAIT}s (try $((TRY + 1)) of $DL_ATTEMPTS)..."
|
||||
sleep "$WAIT"
|
||||
TRY=$((TRY + 1))
|
||||
done
|
||||
}
|
||||
|
||||
# The install log, before anything can fail.
|
||||
mkdir -p "$LOG_DIR" 2>/dev/null && : >> "$LOG_FILE" 2>/dev/null && LOG_OK=yes
|
||||
INSTALLER_MD5=""
|
||||
[ -f "$0" ] && INSTALLER_MD5=$(md5sum "$0" 2>/dev/null | cut -d' ' -f1)
|
||||
log INFO "run start: installer md5=${INSTALLER_MD5:-unknown}, firmware source: ${1:-latest release}"
|
||||
|
||||
echo
|
||||
echo -e "${LIGHTRED} ✺┈┈┈┈┈┈${RESET}"
|
||||
echo -e "${BRIGHT}Open${RESET}Glow ForgeFIRM Installation Tool"
|
||||
@@ -217,6 +312,7 @@ done
|
||||
FREE_KB=$(df -k /data | tail -1 | awk '{print $4}')
|
||||
[ "$FREE_KB" -ge "$MIN_DATA_FREE_KB" ] 2>/dev/null \
|
||||
|| die "need ${MIN_DATA_FREE_KB} KB free on /data, have ${FREE_KB:-unknown}"
|
||||
log INFO "pre-flight: factory $(cat /etc/version), booted slot $ACTIVE, target slot $TARGET, ${FREE_KB} KB free on /data, $(curl -V 2>/dev/null | head -n 1)"
|
||||
|
||||
echo -e "${LIGHTRED}!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!${RESET}"
|
||||
echo -e "${YELLOW} THIS IS EXPERIMENTAL SOFTWARE!${RESET}"
|
||||
@@ -237,6 +333,7 @@ echo
|
||||
slot_probe "$TARGET"
|
||||
TARGET_DESC=$(slot_desc)
|
||||
echo -e "Slot $TARGET currently holds: ${BRIGHT}$TARGET_DESC${RESET}"
|
||||
log INFO "slot $TARGET holds: $TARGET_DESC"
|
||||
if [ "$S_TYPE" = "factory" ]; then
|
||||
echo -e "It will be archived to /data before being overwritten."
|
||||
else
|
||||
@@ -246,6 +343,7 @@ else
|
||||
read -p "Type ERASE to overwrite slot $TARGET, or anything else to abort: " erase
|
||||
echo
|
||||
if [ "$erase" != "ERASE" ]; then
|
||||
log NOTICE "operator declined to erase slot $TARGET; no changes made"
|
||||
echo "Aborting without changes."
|
||||
exit 0
|
||||
fi
|
||||
@@ -254,10 +352,12 @@ echo
|
||||
read -p "Are you sure you want to continue [N/y]? " continue
|
||||
echo
|
||||
if [ "$continue" != "y" ]; then
|
||||
log NOTICE "operator declined to continue; no changes made"
|
||||
echo "Wise choice. Exiting without changes."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
log INFO "operator confirmed; stopping the Glowforge services"
|
||||
stop_gf_services
|
||||
|
||||
# --- archive factory content --------------------------------------------------
|
||||
@@ -268,6 +368,7 @@ for N in 1 2; do
|
||||
ARC="$ARCHIVE_DIR/factory-rootfs-$S_VER.img.gz"
|
||||
if archived_ok "$ARC"; then
|
||||
echo -e "${ASTERISK}Slot $N (factory $S_VER) already archived."
|
||||
log INFO "archive: slot $N (factory $S_VER) already archived"
|
||||
continue
|
||||
fi
|
||||
[ -e "$ARC" ] && echo -e "${ASTERISK}Slot $N: the archive on disk is incomplete; archiving again."
|
||||
@@ -275,6 +376,7 @@ for N in 1 2; do
|
||||
archive_dev /dev/mmcblk2p$N "$ARC" \
|
||||
|| { rm -f "$ARC"; die "archiving slot $N failed"; }
|
||||
echo "$(date '+%Y-%m-%d %H:%M:%S') slot$N factory $S_VER ver=${S_FWVER:-unknown} $(basename $ARC) md5=$(md5sum "$ARC" | cut -d' ' -f1)" >> "$ARCHIVE_DIR/manifest"
|
||||
log INFO "archive: slot $N (factory $S_VER) -> $(basename "$ARC"), $(wc -c < "$ARC") bytes"
|
||||
done
|
||||
for B in 0 1; do
|
||||
ARC="$ARCHIVE_DIR/recovery-boot$B.img.gz"
|
||||
@@ -283,17 +385,20 @@ for B in 0 1; do
|
||||
archive_dev /dev/mmcblk2boot$B "$ARC" \
|
||||
|| { rm -f "$ARC"; die "archiving boot$B failed"; }
|
||||
echo "$(date '+%Y-%m-%d %H:%M:%S') boot$B recovery - $(basename $ARC) md5=$(md5sum "$ARC" | cut -d' ' -f1)" >> "$ARCHIVE_DIR/manifest"
|
||||
log INFO "archive: boot$B -> $(basename "$ARC"), $(wc -c < "$ARC") bytes"
|
||||
done
|
||||
|
||||
# --- acquire the ForgeFIRM .fw ------------------------------------------------
|
||||
mkdir -p /data/forgefirm
|
||||
if [ -n "$1" ]; then
|
||||
[ -s "$1" ] || die "local firmware file '$1' not found"
|
||||
cp "$1" "$FW_FILE"
|
||||
cp "$1" "$FW_FILE" || die "cannot copy '$1' to $FW_FILE"
|
||||
echo -e "${ASTERISK}Using local firmware file: $1"
|
||||
log INFO "firmware: local file $1, $(wc -c < "$FW_FILE") bytes"
|
||||
else
|
||||
echo -e "${ASTERISK}Downloading latest OpenGlow/ForgeFIRM release:"
|
||||
curl -fL "$RELEASE_FW_URL" --output "$FW_FILE" || die "firmware download failed"
|
||||
download_fw \
|
||||
|| die "firmware download failed: ${DL_WHY:-unknown}. Check the network and re-run; the archives are kept, so a re-run goes straight to the download"
|
||||
fi
|
||||
|
||||
# --- verify signature ---------------------------------------------------------
|
||||
@@ -313,6 +418,7 @@ M_VERSION=$(echo "$META" | sed -n 's/^meta-version="\(.*\)"$/\1/p')
|
||||
[ "$M_PLATFORM" = "glowforge" ] \
|
||||
|| { rm -f "$KEYFILE"; die "archive platform is '$M_PLATFORM', not glowforge - wrong archive"; }
|
||||
echo -e "${ASTERISK}Archive: $M_PRODUCT $M_VERSION ($M_PLATFORM)"
|
||||
log INFO "firmware: signature verified; $M_PRODUCT $M_VERSION ($M_PLATFORM)"
|
||||
|
||||
# A validly signed OLDER release must never install silently; downgrades
|
||||
# need an explicit yes (rollback stays possible, just deliberate).
|
||||
@@ -330,13 +436,14 @@ if [ -n "$INSTALLED" ] && ver_lt "$M_VERSION" "$INSTALLED"; then
|
||||
read -n1 -p "Install the downgrade anyway? [y/N] " YN
|
||||
echo
|
||||
case "$YN" in
|
||||
y|Y) ;;
|
||||
y|Y) log NOTICE "operator accepted the downgrade from $INSTALLED to $M_VERSION" ;;
|
||||
*) rm -f "$KEYFILE"; die "downgrade declined" ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# --- apply to the inactive slot -----------------------------------------------
|
||||
echo -e "${ASTERISK}Writing ForgeFIRM to slot $TARGET (/dev/mmcblk2p$TARGET):"
|
||||
log INFO "apply: writing $M_VERSION to slot $TARGET"
|
||||
for M in $(sed -n "s|^/dev/mmcblk2p$TARGET \([^ ]*\).*|\1|p" /proc/mounts); do
|
||||
umount "$M" 2>/dev/null
|
||||
done
|
||||
@@ -363,6 +470,7 @@ cp "$MP/usr/sbin/ffboot" /data/ffboot.new \
|
||||
umount "$MP"
|
||||
rmdir "$MP" 2>/dev/null
|
||||
echo -e "${ASTERISK}Slot $TARGET now holds ForgeFIRM $NEWVER"
|
||||
log INFO "apply: slot $TARGET holds ForgeFIRM $NEWVER; the written filesystem verified"
|
||||
|
||||
# --- ffboot for the factory side ----------------------------------------------
|
||||
mv /data/ffboot.new /data/ffboot
|
||||
@@ -372,6 +480,7 @@ chmod +x /data/ffboot
|
||||
echo -e "${ASTERISK}Setting boot to /dev/mmcblk2p$TARGET"
|
||||
set_env 1 0 "$TARGET" "/dev/mmcblk2p$TARGET" \
|
||||
|| die "environment write did not verify - boot selection unchanged; run /data/ffboot -e$TARGET manually"
|
||||
log INFO "boot selection set to slot $TARGET and read back; install complete"
|
||||
|
||||
echo
|
||||
echo -e "${BRIGHT}Installation complete.${RESET}"
|
||||
@@ -379,5 +488,7 @@ echo -e "To return to factory firmware later: ${BRIGHT}/data/ffboot -e${RESET} (
|
||||
echo
|
||||
read -n1 -p "Press any key to reboot into OpenGlow/ForgeFIRM..." continue
|
||||
echo
|
||||
log INFO "rebooting into ForgeFIRM $NEWVER"
|
||||
sync
|
||||
reboot
|
||||
exit 0
|
||||
|
||||
Reference in New Issue
Block a user