fix(manifest)!: the size cap opened a service-wide hang; close it
The diff-scoped bug-hunt panel, four arms, artifact-only. Its strongest finding is one I created two hours earlier while hardening the reader. `stat` reports size 0 for a FIFO and 0 for a symlink to /dev/zero, so both sail under the byte cap added for the RecursionError round — and then `read_text` either blocks in read() with no EOF, so the except never runs, or allocates until the kernel intervenes. `list_booths` reads every booth on every GET / and /healthz, so ONE such file stalls the front page for the whole service, with no error and no recovery short of a restart. Reproduced before believing it (timeout returned 124). S_ISREG is checked BEFORE the size in both modules now; verified against the live service with two FIFOs planted, which answered 200 in 36ms. The shape worth carrying: st_size answers a different question than "can this be read", and a bound that trusts it inherits everything it does not mean. A hardening fix opened a worse hole than the one it closed. THE UPLOAD PATH WROTE ABOVE ITS OWN CLEANUP GUARD (4/4) A failed manifest write orphaned a .uploaded half-booth with no files in it — and because the temp name now carries a random suffix, nothing ever overwrote the leak, and .booth.json.<hex>.tmp is not a .lock, so _newest_mtime counted it and kept that empty booth past every sweep. The uniqueness fix from the previous round is what made the leak permanent. Both writes moved inside the guard; the temp is removed on every exit path. DAMAGED BYTES ARE KEPT, NOT REPLACED (4/4, INV-6) Marks made this explicit in v0.2.1 and this write path contradicted it: a manifest that failed on ONE field lost the others with it, including a why the re-announcer may never have kept anywhere. It diverges from marks in HOW it honours the rule — marks refuse and answer 409 because the operator's judgment is not restatable; a manifest quarantines and proceeds, because refusing would fail `booth add` and lose the files it was copying. ONE OPENNESS PREDICATE, AS U2 SAID (2/4) `booth answer` spelled out `if m.answer is None` while `booth marks` asked `open_marks`, so a partially-answered pick read as done to one verb and open to the other — at the same instant, on the same booth. U2's INV-2 put openness in one function precisely so they could not drift. The mirror case is fixed too: a pick that hydrates broken is refused by the web route, so `answer --wait` polled an hour on a form nothing could ever land. ALSO - now_stamp was whole-second while the importer had moved to microseconds, and '-' sorts before '.', so a later mark came out ahead of an earlier import inside the same second. One format; the previous round's ordering fix had opened this one. - `_broken` was the third of three directory-name fallbacks and the one still handing a raw name into a card's sub-line. - An identical re-announce rewrote the file and reset the TTL. `booth link` does this on every post to the standing board. - The importer's return went through the bare _hydrate, not _hydrate_safe. - A marks document could be written larger than it can be read back, and then read as no marks at all. Refused at the write instead. - `choice` reached the answer builder raw while `notes` beside it did not. AND ONE FINDING DELIBERATELY NOT FULLY CLOSED The mtime-restore race is real. The clean fix — ignore a booth directory's own mtime whenever the booth holds anything — also silently retires the documented rule that releasing a kept board resets its clock, which the CLI header, the README and a deliberately-written test all pin. That is a TTL doctrine change, not a bug fix, and an existing test caught the attempt. The concrete half is fixed (a failing os.utime escaped and 500'd the route); the race is stated in the code where the next reader will meet it. 341 tests. Live service restarted, 24/24 booth pages verified.
This commit is contained in:
Vuong Hoang
@@ -285,3 +285,64 @@ def test_an_explicitly_empty_why_still_clears_it(tmp_path):
|
||||
capture_output=True, timeout=30, env=env)
|
||||
|
||||
assert _manifest(tmp_path / "b").why == ""
|
||||
|
||||
|
||||
def test_answer_and_marks_agree_about_what_open_means(tmp_path):
|
||||
"""U2 made `_is_open` THE openness predicate — "nothing else may spell this
|
||||
out" — and `booth answer`'s reader spelled it out anyway, as
|
||||
`if m.answer is None`. So a PARTIALLY answered pick read as done to
|
||||
`answer` and still-open to `marks --wait`: one verb returns the half-filled
|
||||
form and the other blocks on the same booth at the same instant.
|
||||
|
||||
Found 2/4. The two verbs are the session's whole view of the loop, and a
|
||||
session that asks both gets two answers.
|
||||
"""
|
||||
import sys
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).parent.parent))
|
||||
from booth.marks import answer_pick, declare_pick
|
||||
|
||||
b = tmp_path / "b"
|
||||
b.mkdir()
|
||||
declare_pick(b, "batch", {
|
||||
"title": "R18",
|
||||
"questions": [
|
||||
{"key": "q1", "prompt": "One?", "options": ["keep", "drop"]},
|
||||
{"key": "q2", "prompt": "Two?", "options": ["keep", "drop"]},
|
||||
],
|
||||
})
|
||||
answer_pick(b, "batch", {"q1": "keep", "q2": None}) # partial
|
||||
|
||||
env = {**os.environ, "BOOTH_DATA_DIR": str(tmp_path),
|
||||
"BOOTH_URL": "http://booth.invalid"}
|
||||
marks = subprocess.run([str(SCRIPT), "marks", "b"], capture_output=True,
|
||||
text=True, timeout=30, env=env)
|
||||
answer = subprocess.run([str(SCRIPT), "answer", "b", "batch"],
|
||||
capture_output=True, text=True, timeout=30, env=env)
|
||||
|
||||
still_open = "batch" in json.loads(marks.stdout)["open"]
|
||||
assert still_open, "a partial answer stopped counting as open"
|
||||
assert answer.returncode == UNANSWERED, (
|
||||
"`answer` called a partially-answered pick done while `marks` called it open"
|
||||
)
|
||||
|
||||
|
||||
def test_answer_does_not_poll_forever_on_a_pick_that_cannot_be_answered(tmp_path):
|
||||
"""The mirror failure. A pick whose declaration went bad hydrates with
|
||||
`error` set, which makes it NOT open — so `marks --wait` returns at once
|
||||
while `answer --wait` polled the full hour against a form the web route
|
||||
refuses with a 400. Nothing was ever going to land."""
|
||||
b = tmp_path / "b"
|
||||
b.mkdir()
|
||||
(b / ".marks.json").write_text(json.dumps({
|
||||
"version": 1,
|
||||
"marks": [{"id": "broken", "shape": "pick", "declaration": {},
|
||||
"error": "pick has no declaration",
|
||||
"created": "2026-09-21T00:00:00.000000+00:00"}],
|
||||
}))
|
||||
|
||||
r = subprocess.run([str(SCRIPT), "answer", "b", "broken", "--wait", "8"],
|
||||
capture_output=True, text=True, timeout=40,
|
||||
env={**os.environ, "BOOTH_DATA_DIR": str(tmp_path),
|
||||
"BOOTH_URL": "http://booth.invalid"})
|
||||
assert r.returncode != 0
|
||||
assert "broken" in r.stderr.lower() or "cannot" in r.stderr.lower()
|
||||
|
||||
+148
-3
@@ -519,9 +519,25 @@ def test_only_the_manifest_module_opens_the_manifest(tmp_path):
|
||||
for src in sorted((root / "booth").glob("*.py")):
|
||||
if src.name == "manifest.py":
|
||||
continue
|
||||
if ".booth.json" in src.read_text():
|
||||
offenders.append(src.name)
|
||||
assert not offenders, f"{offenders} name the manifest file directly"
|
||||
tree = ast.parse(src.read_text())
|
||||
# STRING CONSTANTS, not raw text. A comment naming the file is prose
|
||||
# about the design and harms nothing — the first version of this test
|
||||
# scanned the whole source and went red on a comment explaining why a
|
||||
# leaked `.booth.json.<hex>.tmp` keeps a booth alive. The invariant is
|
||||
# about code that knows the filename, so ask the code.
|
||||
docstrings = set()
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, (ast.Module, ast.ClassDef,
|
||||
ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
body = getattr(node, "body", None)
|
||||
if body and isinstance(body[0], ast.Expr) and \
|
||||
isinstance(body[0].value, ast.Constant):
|
||||
docstrings.add(id(body[0].value))
|
||||
for node in ast.walk(tree):
|
||||
if (isinstance(node, ast.Constant) and isinstance(node.value, str)
|
||||
and id(node) not in docstrings and ".booth.json" in node.value):
|
||||
offenders.append(f"{src.name}:{node.lineno}")
|
||||
assert not offenders, f"{offenders} name the manifest file in code"
|
||||
|
||||
|
||||
def test_announcing_is_activity_via_the_manifest_file_itself(tmp_path):
|
||||
@@ -572,3 +588,132 @@ def test_the_title_reaches_a_surface(client):
|
||||
page = c.get("/b/r18-ab/").text
|
||||
assert "R18 A/B — denoiser bakeoff" in page
|
||||
assert "r18-ab" in page, "the directory name stopped being visible"
|
||||
|
||||
|
||||
# ---- findings from the cross-frontier BUG-HUNT panel, 2026-09-22 -------------
|
||||
#
|
||||
# Heid panel (thread 01M343SXX27Z47C3STXXRC7M42). Four arms, artifact-only,
|
||||
# diff-scoped. The strongest finding is one the SIZE CAP ITSELF opened.
|
||||
|
||||
|
||||
def test_a_reader_never_blocks_on_a_file_that_is_not_a_file(tmp_path):
|
||||
"""`stat` reports size 0 for a FIFO, so it sails under the byte cap — and
|
||||
then `read_text` blocks in `read` with no EOF, so the `except` never runs
|
||||
and the call never returns. `list_booths` reads every booth on every `GET /`
|
||||
and `/healthz`, so ONE such file stalls the front page for the whole service,
|
||||
with no error and no recovery short of a restart.
|
||||
|
||||
A symlink to `/dev/zero` is the same hole with unbounded allocation instead
|
||||
of a hang: `st_size` is 0 there too.
|
||||
|
||||
Two of four arms reached it independently. The bound added an hour earlier
|
||||
is what made it reachable — `st_size` answers a different question than
|
||||
"can this be read", and a cap that trusts it inherits the difference.
|
||||
"""
|
||||
import os
|
||||
import signal
|
||||
|
||||
b = tmp_path / "b"
|
||||
b.mkdir()
|
||||
os.mkfifo(b / MANIFEST_FILE)
|
||||
|
||||
# ⚠ ALARMED. Without this the RED state of this test does not fail, it HANGS
|
||||
# — which is the defect itself, and is also useless as a signal: a suite that
|
||||
# stops is indistinguishable from a suite that is slow. Five seconds is a
|
||||
# thousand times the budget a read of a four-field file should need.
|
||||
def _timeout(signum, frame):
|
||||
raise AssertionError("read_manifest blocked on a FIFO and never returned")
|
||||
|
||||
old_handler = signal.signal(signal.SIGALRM, _timeout)
|
||||
signal.alarm(5)
|
||||
try:
|
||||
got = read_manifest(b)
|
||||
finally:
|
||||
signal.alarm(0)
|
||||
signal.signal(signal.SIGALRM, old_handler)
|
||||
|
||||
assert isinstance(got, Manifest) and got.error
|
||||
assert "regular file" in got.error
|
||||
|
||||
|
||||
def test_a_damaged_manifest_is_kept_when_it_is_replaced(tmp_path):
|
||||
"""4/4, and it contradicted this repo's own doctrine. Marks made the rule
|
||||
explicit in v0.2.1 — reads stay lenient, writes go strict, damaged bytes
|
||||
STAY ON DISK — and the manifest's write replaced them outright.
|
||||
|
||||
The sharpest leg: a file that fails on ONE field still holds the others.
|
||||
`{"handle": 7, "why": "the thing I wanted you to look at"}` reads as broken
|
||||
and used to be destroyed whole, taking a `why` the re-announcer may not have
|
||||
kept anywhere.
|
||||
|
||||
Quarantined rather than refused: refusing would fail `booth add` and lose
|
||||
the files it was copying, which is the worse trade. One fixed-name
|
||||
quarantine, so this cannot accumulate.
|
||||
"""
|
||||
from booth.manifest import QUARANTINE_FILE
|
||||
|
||||
b = tmp_path / "b"
|
||||
b.mkdir()
|
||||
damaged = json.dumps({"handle": 7, "why": "the thing I wanted you to see"})
|
||||
(b / MANIFEST_FILE).write_text(damaged)
|
||||
|
||||
write_manifest(b, "booth-dev", why="rescued")
|
||||
|
||||
assert read_manifest(b).why == "rescued"
|
||||
assert (b / QUARANTINE_FILE).read_text() == damaged, "the damaged bytes were destroyed"
|
||||
|
||||
|
||||
def test_a_broken_record_normalizes_the_directory_name_too(tmp_path):
|
||||
"""The third fallback. `write_manifest`'s and `read_manifest`'s were fixed
|
||||
in the previous round and `_broken`'s was missed — same raw `booth.name`,
|
||||
same card sub-line, same newline."""
|
||||
b = tmp_path / ("wei" + "i" * 200 + "rd\nname")
|
||||
b.mkdir()
|
||||
(b / MANIFEST_FILE).write_text("{oops")
|
||||
|
||||
got = read_manifest(b)
|
||||
assert got.error and "\n" not in got.title and len(got.title) <= 120
|
||||
|
||||
|
||||
def test_an_identical_re_announce_does_not_touch_the_booth(tmp_path):
|
||||
"""Marks learned this in v0.2.0: a write that changes nothing is not
|
||||
activity and must not reset a booth's TTL. The manifest wrote
|
||||
unconditionally, so `booth add` on an unchanged booth kept a dead one alive
|
||||
— and `booth link` does it on every single post to the standing board."""
|
||||
import os
|
||||
|
||||
b = tmp_path / "b"
|
||||
b.mkdir()
|
||||
write_manifest(b, "booth-dev", why="x")
|
||||
path = b / MANIFEST_FILE
|
||||
os.utime(path, (1_000_000_000, 1_000_000_000))
|
||||
os.utime(b, (1_000_000_000, 1_000_000_000))
|
||||
before = path.stat().st_mtime
|
||||
|
||||
write_manifest(b, "booth-dev", why="x") # identical
|
||||
|
||||
assert path.stat().st_mtime == before, "an identical re-announce rewrote the file"
|
||||
|
||||
|
||||
def test_a_failed_write_leaves_no_temp_file_behind(tmp_path):
|
||||
"""The unique temp name fixed a cross-writer hazard and created a litter
|
||||
one: a fixed name is overwritten by the next writer, a random one is not.
|
||||
And `.booth.json.<hex>.tmp` is NOT a `.lock`, so `_newest_mtime` counts it —
|
||||
an orphaned temp would keep a dead booth alive forever."""
|
||||
import os
|
||||
|
||||
b = tmp_path / "b"
|
||||
b.mkdir()
|
||||
real_replace = os.replace
|
||||
|
||||
def boom(src, dst, *a, **kw):
|
||||
raise OSError("no space left on device")
|
||||
|
||||
os.replace = boom
|
||||
try:
|
||||
with pytest.raises(OSError):
|
||||
write_manifest(b, "booth-dev", why="x")
|
||||
finally:
|
||||
os.replace = real_replace
|
||||
|
||||
assert not list(b.glob("*.tmp")), f"orphaned temp: {list(b.glob('*.tmp'))}"
|
||||
|
||||
@@ -1247,3 +1247,130 @@ def test_a_write_over_an_unparseable_marks_file_still_refuses(tmp_path):
|
||||
|
||||
with pytest.raises(MarksCorrupt):
|
||||
set_flag(booth, "a.png", True)
|
||||
|
||||
|
||||
# ---- findings from the U5 diff-scoped BUG-HUNT panel, 2026-09-22 ------------
|
||||
|
||||
|
||||
def test_the_marks_reader_never_blocks_on_a_file_that_is_not_a_file(tmp_path):
|
||||
"""Same hole the size cap opened in the manifest, in the sibling it was
|
||||
copied from. `st_size` is 0 for a FIFO, so it passes the cap, and then
|
||||
`read_text` blocks with no EOF. `list_booths` reads every booth's marks on
|
||||
every `GET /` and `/healthz`."""
|
||||
import os
|
||||
import signal
|
||||
|
||||
booth = tmp_path / "b"
|
||||
booth.mkdir()
|
||||
os.mkfifo(booth / MARKS_FILE)
|
||||
|
||||
def _timeout(signum, frame):
|
||||
raise AssertionError("marks_for blocked on a FIFO and never returned")
|
||||
|
||||
old = signal.signal(signal.SIGALRM, _timeout)
|
||||
signal.alarm(5)
|
||||
try:
|
||||
assert marks_for(booth) == []
|
||||
finally:
|
||||
signal.alarm(0)
|
||||
signal.signal(signal.SIGALRM, old)
|
||||
|
||||
|
||||
def test_new_marks_and_imported_marks_share_one_stamp_format(tmp_path):
|
||||
"""The v0.2.2 fix for the legacy-import ordering opened a NEW ordering bug,
|
||||
which is the shape worth remembering. `import_legacy_asks` moved to
|
||||
microsecond precision while `now_stamp` stayed at whole seconds, and `-` is
|
||||
0x2D against `.` at 0x2E — so `...T10:00:00-07:00` sorts BEFORE
|
||||
`...T10:00:00.500000-07:00`, putting a LATER mark ahead of an EARLIER
|
||||
import inside the same second.
|
||||
|
||||
Deterministic order is a v1 invariant precisely because the operator refers
|
||||
to things positionally. One format, or the rule cannot be stated.
|
||||
"""
|
||||
from booth.marks import now_stamp
|
||||
|
||||
stamp = now_stamp()
|
||||
assert "." in stamp.split("T")[1], f"now_stamp is not sub-second: {stamp}"
|
||||
assert len(stamp.split(".")[1].split("+")[0].split("-")[0]) == 6
|
||||
|
||||
|
||||
def test_the_importer_cannot_raise_out_of_a_poisoned_entry(tmp_path):
|
||||
"""`marks_for` routes every entry through `_hydrate_safe`; the importer's
|
||||
return still went through the bare `_hydrate`, so the one path that reads
|
||||
entries it did not write was the one without the guard."""
|
||||
booth = tmp_path / "b"
|
||||
booth.mkdir()
|
||||
(booth / MARKS_FILE).write_text(json.dumps({
|
||||
"version": 1,
|
||||
"marks": [{"id": "n1", "shape": "note", "text": {"bad": True},
|
||||
"created": "2026-09-21T00:00:00+00:00"}],
|
||||
}))
|
||||
(booth / f"q1{ASK_SUFFIX}").write_text(json.dumps(_single()))
|
||||
|
||||
from booth.marks import import_legacy_asks
|
||||
out = import_legacy_asks(booth) # must not raise
|
||||
assert isinstance(out, list)
|
||||
|
||||
|
||||
def test_a_document_that_would_not_read_back_is_refused_at_the_write(tmp_path):
|
||||
"""The read bound is on the STORED bytes and the write adds `indent=2`, so a
|
||||
document that fits in memory can land over the limit on disk and then read
|
||||
back as no marks at all — every mark in the booth gone, silently. Refuse
|
||||
loudly instead: a write that fails is recoverable.
|
||||
|
||||
Asserted against `_write_raw` directly, because no single mark can get
|
||||
there: `_clean_text` caps a note at TEXT_MAX and a flag is a fixed shape.
|
||||
The reachable path is accumulation — `_note_id` puts no ceiling on how many
|
||||
notes one booth may carry — which is thousands of writes, not one. Testing
|
||||
it through `write_note` would need a fixture nobody could justify, and
|
||||
would be testing the cap rather than the guard.
|
||||
"""
|
||||
from booth.marks import MARKS_MAX_BYTES, MarksCorrupt, _write_raw
|
||||
|
||||
booth = tmp_path / "b"
|
||||
booth.mkdir()
|
||||
bulk = [{"id": f"note-{i}", "shape": "note", "text": "x" * 500,
|
||||
"created": "2026-09-21T00:00:00.000000+00:00"}
|
||||
for i in range(MARKS_MAX_BYTES // 400)]
|
||||
|
||||
with pytest.raises(MarksCorrupt):
|
||||
_write_raw(booth, bulk)
|
||||
assert not (booth / MARKS_FILE).exists(), "a refused write still landed"
|
||||
|
||||
|
||||
def test_a_clock_restore_that_fails_does_not_take_the_route_down(tmp_path):
|
||||
"""The concrete half of the mtime-restore finding.
|
||||
|
||||
`_Locked.__enter__` puts the booth directory's clock back after creating its
|
||||
lock, and `os.utime` can fail — a read-only directory, a booth whose owner
|
||||
we are not. It used to escape into the route and answer 500 for what is
|
||||
otherwise a perfectly good request. Not putting the clock back is a cost
|
||||
this module can absorb; not answering is not.
|
||||
|
||||
The RACE half of that finding is documented in the code and deliberately not
|
||||
closed: the alternative fix would silently retire the documented behaviour
|
||||
that releasing a kept board resets its clock
|
||||
(`test_releasing_a_board_RESETS_its_ttl_clock` pins that on purpose), which
|
||||
is a TTL doctrine change rather than a bug fix.
|
||||
"""
|
||||
import os
|
||||
|
||||
from booth.marks import MARKS_LOCK, set_flag
|
||||
|
||||
booth = tmp_path / "b"
|
||||
booth.mkdir()
|
||||
real_utime = os.utime
|
||||
|
||||
def boom(path, *a, **kw):
|
||||
if str(path) == str(booth):
|
||||
raise PermissionError("read-only directory")
|
||||
return real_utime(path, *a, **kw)
|
||||
|
||||
os.utime = boom
|
||||
try:
|
||||
assert set_flag(booth, "a.png", True) is not None
|
||||
finally:
|
||||
os.utime = real_utime
|
||||
|
||||
assert (booth / MARKS_LOCK).exists()
|
||||
assert [m.target for m in marks_for(booth)] == ["a.png"]
|
||||
|
||||
Reference in New Issue
Block a user