feat(dates): creation and update times for every booth, from the filesystem
The operator: "I think I want creation and update dates on the booths now too." UPDATE was already there — `landed_at`, the newest mtime among CONTENT excluding our own machinery, which the Desk already sorts "new since you looked" by. CREATION had no honest source. `.booth.json` carries a declared `created`, but only for booths posted through the CLI since U5 — TWELVE OF THIRTY live booths had none. Every alternative was a guess wearing a fact's clothes: oldest content mtime is wrong the moment an agent copies files with timestamps preserved; directory mtime is just "last thing added", which is landed_at renamed; and stamping a first-seen marker on read is the same write-on-read shape that spent an hour of today aging the booth it cached. ext4 records a real birth time. CPython does not expose st_birthtime on Linux, so booth/birthtime.py reads it through statx(2) — a fact the disk already holds rather than one we invent. Verified against stat(1) on live booths, 6 of 6 exact, including every booth with no manifest. ONE rule for all thirty, which is what invariant 6 asks of anything statable in a line. None when the filesystem cannot say (tmpfs, NFS, an old kernel), and None renders as nothing — the honest output when nobody knows. Never raises: list_booths calls it once per booth on every index load, so a read that can raise is a service-wide outage wearing a single-booth bug's clothes. ALSO TWO REAL TEST-HARNESS DEFECTS, found chasing a flake and fixed on their merits rather than because they were proven to be the cause: - The keyboard-flag browser test fired ArrowRight and `f` back to back, assuming the first had finished — and focus() does a scrollIntoView, so under load `f` could arrive with no cursor and flag nothing. It now waits for the cursor to land. - BOTH browser fixtures did bind -> getsockname -> CLOSE -> hand uvicorn the port NUMBER, leaving a window for the kernel to give that port to somebody else. This suite runs two browser files that each start a server per test, so the competitor is right there. The bound socket is now handed over directly. ⚠ THE FLAKE IS NOT PROVEN FIXED. Two different browser tests failed once each across full-suite runs while passing 3/3 and 5/5 in isolation; since the fixes, one failure in three runs. n=3 cannot distinguish that from the prior rate and this commit does not claim it does. 770 green on a clean run.
This commit is contained in:
@@ -171,6 +171,7 @@ def set_blurred(booth: Path, rel: str, on: bool) -> set[str]:
|
|||||||
# can use it without pulling FastAPI in. Re-exported here because call sites and
|
# can use it without pulling FastAPI in. Re-exported here because call sites and
|
||||||
# tests already reference these names through app.
|
# tests already reference these names through app.
|
||||||
from booth.thumbs import THUMB_DIR, ensure_thumb
|
from booth.thumbs import THUMB_DIR, ensure_thumb
|
||||||
|
from booth.birthtime import birth_time
|
||||||
from booth.asks import ( # noqa: E402
|
from booth.asks import ( # noqa: E402
|
||||||
ANSWER_SUFFIX,
|
ANSWER_SUFFIX,
|
||||||
ASK_SUFFIX,
|
ASK_SUFFIX,
|
||||||
@@ -706,6 +707,16 @@ def list_booths(data_dir: Path, ttl_seconds: float, now: float | None = None) ->
|
|||||||
# `landed_at` is content. "New since you looked" reads only the
|
# `landed_at` is content. "New since you looked" reads only the
|
||||||
# second, so a flag or a view never makes a booth look new.
|
# second, so a flag or a view never makes a booth look new.
|
||||||
"landed_at": _content_mtime(child),
|
"landed_at": _content_mtime(child),
|
||||||
|
# WHEN THE BOOTH WAS MADE, from the filesystem's own record.
|
||||||
|
# ONE RULE for all thirty: `.booth.json`'s declared `created`
|
||||||
|
# only exists for booths posted through the CLI since U5, and
|
||||||
|
# twelve live booths had none. Every alternative was a guess
|
||||||
|
# wearing a fact's clothes — oldest content mtime is wrong the
|
||||||
|
# moment an agent copies files with timestamps preserved, and
|
||||||
|
# directory mtime just means "last thing added". ext4 records a
|
||||||
|
# real birth time; this reads it. None when the filesystem
|
||||||
|
# cannot say, and None renders as nothing.
|
||||||
|
"created_at": birth_time(child),
|
||||||
"viewed_at": _viewed_at(child),
|
"viewed_at": _viewed_at(child),
|
||||||
# The first four images in item order, as the originals shown
|
# The first four images in item order, as the originals shown
|
||||||
# small. Blurred ones stay blurred, the cover's rule.
|
# small. Blurred ones stay blurred, the cover's rule.
|
||||||
@@ -1263,6 +1274,10 @@ def create_app(
|
|||||||
# a booth URL handed to the operator lands HERE, never on the
|
# a booth URL handed to the operator lands HERE, never on the
|
||||||
# index, and job 5 is "operator, look at this".
|
# index, and job 5 is "operator, look at this".
|
||||||
"manifest": read_manifest(booth),
|
"manifest": read_manifest(booth),
|
||||||
|
# Same two clocks the Desk row carries, because a booth URL
|
||||||
|
# handed to the operator lands HERE and not on the index.
|
||||||
|
"created_at": birth_time(booth),
|
||||||
|
"landed_at": _content_mtime(booth),
|
||||||
# The lifetime line, same three states as the index card: a
|
# The lifetime line, same three states as the index card: a
|
||||||
# booth URL handed to the operator lands HERE, not on the index,
|
# booth URL handed to the operator lands HERE, not on the index,
|
||||||
# so "why is this not counting down" has to be answerable here.
|
# so "why is this not counting down" has to be answerable here.
|
||||||
|
|||||||
@@ -0,0 +1,73 @@
|
|||||||
|
"""The filesystem's own record of when a directory was created.
|
||||||
|
|
||||||
|
The operator asked for creation dates on booths. Only 18 of 30 live booths had
|
||||||
|
one: `.booth.json` carries a declared `created`, but that file only exists for
|
||||||
|
booths posted through the CLI since U5, and the twelve older ones had nothing.
|
||||||
|
|
||||||
|
The tempting answers were all guesses wearing a fact's clothes — the oldest
|
||||||
|
content mtime (wrong whenever an agent copies files with timestamps preserved),
|
||||||
|
or the directory mtime (which is just "last time something was added"). Writing
|
||||||
|
a first-seen stamp on read was worse still: this service spent an hour today
|
||||||
|
fixing a cache that aged the booth it cached.
|
||||||
|
|
||||||
|
ext4 records a real birth time. CPython 3.13 does not expose `st_birthtime` on
|
||||||
|
Linux, but `statx(2)` does and glibc has wrapped it since 2.28 — so this reads
|
||||||
|
a FACT the disk already holds rather than inventing one.
|
||||||
|
|
||||||
|
DEGRADES TO None, always: an old kernel, a filesystem that does not record
|
||||||
|
btime (tmpfs, NFS, some overlayfs), a missing glibc symbol, or anything else
|
||||||
|
unexpected. A caller that gets None shows nothing, which is the honest output
|
||||||
|
when nobody knows.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ctypes
|
||||||
|
import ctypes.util
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
_AT_FDCWD = -100
|
||||||
|
_STATX_BTIME = 0x00000800
|
||||||
|
# struct statx: stx_btime is the SECOND statx_timestamp, and the four that
|
||||||
|
# precede it occupy a fixed 64-byte head (mask, blksize, attributes, nlink,
|
||||||
|
# uid, gid, mode, spare, ino, size, blocks, attributes_mask), then atime.
|
||||||
|
_BTIME_SEC_OFFSET = 80
|
||||||
|
_STATX_BUF_SIZE = 256
|
||||||
|
|
||||||
|
|
||||||
|
def _load():
|
||||||
|
try:
|
||||||
|
libc = ctypes.CDLL(ctypes.util.find_library("c") or "libc.so.6", use_errno=True)
|
||||||
|
return libc.statx
|
||||||
|
except (OSError, AttributeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
_statx = _load()
|
||||||
|
|
||||||
|
|
||||||
|
def birth_time(path: Path) -> float | None:
|
||||||
|
"""When the filesystem says this path was created, or None if it cannot say.
|
||||||
|
|
||||||
|
NEVER RAISES. `list_booths` calls this once per booth on every index load,
|
||||||
|
so a read that can raise is a service-wide outage wearing a single-booth
|
||||||
|
bug's clothes — the posture `read_manifest` already states, applied before
|
||||||
|
the same mistake rather than after it.
|
||||||
|
"""
|
||||||
|
if _statx is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
buf = ctypes.create_string_buffer(_STATX_BUF_SIZE)
|
||||||
|
rc = _statx(ctypes.c_int(_AT_FDCWD), os.fsencode(str(path)),
|
||||||
|
ctypes.c_int(0), ctypes.c_uint(_STATX_BTIME), buf)
|
||||||
|
if rc != 0:
|
||||||
|
return None
|
||||||
|
mask = int.from_bytes(buf.raw[0:4], "little")
|
||||||
|
if not mask & _STATX_BTIME:
|
||||||
|
return None # the filesystem does not record it
|
||||||
|
sec = int.from_bytes(buf.raw[_BTIME_SEC_OFFSET:_BTIME_SEC_OFFSET + 8],
|
||||||
|
"little", signed=True)
|
||||||
|
return float(sec) if sec > 0 else None
|
||||||
|
except Exception: # noqa: BLE001 — see the docstring; nothing here is worth a 500
|
||||||
|
return None
|
||||||
@@ -1726,3 +1726,55 @@ def test_the_blurbooth_route_toggles_and_lands_back(tmp_path):
|
|||||||
|
|
||||||
c.post("/b/g/blurbooth", data={"on": "0"}, follow_redirects=False)
|
c.post("/b/g/blurbooth", data={"on": "0"}, follow_redirects=False)
|
||||||
assert not (b / ".blurbooth").exists()
|
assert not (b / ".blurbooth").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_every_booth_can_state_when_it_was_made(tmp_path):
|
||||||
|
"""The operator asked for creation dates. `.booth.json`'s declared
|
||||||
|
`created` only exists for booths posted through the CLI since U5 — twelve
|
||||||
|
of thirty live booths had none — and every alternative was a guess wearing
|
||||||
|
a fact's clothes: oldest content mtime is wrong the moment an agent copies
|
||||||
|
files with timestamps preserved, and directory mtime just means "last thing
|
||||||
|
added".
|
||||||
|
|
||||||
|
ext4 records a real birth time and `statx` reads it, so this is a FACT the
|
||||||
|
disk already holds. ONE rule for every booth, manifest or not.
|
||||||
|
|
||||||
|
Defeating change: falling back to `stat().st_mtime`, which changes every
|
||||||
|
time a file lands and would show a week-old booth as created five minutes
|
||||||
|
ago."""
|
||||||
|
import time
|
||||||
|
|
||||||
|
b = tmp_path / "g"
|
||||||
|
b.mkdir()
|
||||||
|
made = time.time()
|
||||||
|
(b / "a.png").write_bytes(b"x")
|
||||||
|
|
||||||
|
rows = {r["name"]: r for r in list_booths(tmp_path, ttl_seconds=86400)}
|
||||||
|
row = rows["g"]
|
||||||
|
assert row["created_at"] is not None, "no creation time for a fresh booth"
|
||||||
|
assert abs(row["created_at"] - made) < 10
|
||||||
|
|
||||||
|
# and it must NOT move when content lands later
|
||||||
|
time.sleep(1.1)
|
||||||
|
(b / "b.png").write_bytes(b"y")
|
||||||
|
again = {r["name"]: r for r in list_booths(tmp_path, ttl_seconds=86400)}["g"]
|
||||||
|
assert again["created_at"] == row["created_at"], \
|
||||||
|
"the creation time moved when a file was added — that is `updated`, not `created`"
|
||||||
|
assert again["landed_at"] > row["landed_at"], "`updated` did not move"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_filesystem_with_no_birth_time_shows_nothing(tmp_path, monkeypatch):
|
||||||
|
"""None renders as nothing, which is the honest output when nobody knows —
|
||||||
|
tmpfs, NFS and some overlayfs do not record a birth time, and an old kernel
|
||||||
|
has no `statx` at all.
|
||||||
|
|
||||||
|
Defeating change: substituting any mtime when birth_time returns None."""
|
||||||
|
import booth.app as app_mod
|
||||||
|
|
||||||
|
b = tmp_path / "g"
|
||||||
|
b.mkdir()
|
||||||
|
(b / "a.png").write_bytes(b"x")
|
||||||
|
monkeypatch.setattr(app_mod, "birth_time", lambda p: None)
|
||||||
|
|
||||||
|
row = {r["name"]: r for r in list_booths(tmp_path, ttl_seconds=86400)}["g"]
|
||||||
|
assert row["created_at"] is None
|
||||||
|
|||||||
@@ -50,12 +50,25 @@ def live(tmp_path):
|
|||||||
sock = socket.socket()
|
sock = socket.socket()
|
||||||
sock.bind(("127.0.0.1", 0))
|
sock.bind(("127.0.0.1", 0))
|
||||||
port = sock.getsockname()[1]
|
port = sock.getsockname()[1]
|
||||||
sock.close()
|
# ⚠ THE SOCKET IS HANDED TO UVICORN STILL BOUND, never closed and
|
||||||
|
# re-opened by port number. The old form did bind -> getsockname -> CLOSE ->
|
||||||
|
# tell uvicorn the number, which leaves a window where the kernel can give
|
||||||
|
# that port to somebody else — and this suite runs TWO browser files that
|
||||||
|
# each start a server per test, so the other one is right there competing
|
||||||
|
# for it. Passing the live socket removes the window rather than narrowing
|
||||||
|
# it.
|
||||||
|
#
|
||||||
|
# Honest about the evidence: two different browser tests failed once each
|
||||||
|
# across full-suite runs while passing 3/3 and 5/5 on their own, which is
|
||||||
|
# the signature of contention. We cannot prove from two samples that this
|
||||||
|
# race was the cause. It is a real defect either way, and it is the only
|
||||||
|
# one visible in the harness.
|
||||||
|
|
||||||
|
|
||||||
app = create_app(tmp_path, ttl_hours=24, start_sweeper=False)
|
app = create_app(tmp_path, ttl_hours=24, start_sweeper=False)
|
||||||
config = uvicorn.Config(app, host="127.0.0.1", port=port, log_level="error")
|
config = uvicorn.Config(app, host="127.0.0.1", port=port, log_level="error")
|
||||||
server = uvicorn.Server(config)
|
server = uvicorn.Server(config)
|
||||||
thread = threading.Thread(target=server.run, daemon=True)
|
thread = threading.Thread(target=lambda: server.run(sockets=[sock]), daemon=True)
|
||||||
thread.start()
|
thread.start()
|
||||||
deadline = time.time() + 10
|
deadline = time.time() + 10
|
||||||
while not server.started and time.time() < deadline:
|
while not server.started and time.time() < deadline:
|
||||||
@@ -608,7 +621,15 @@ def test_the_keyboard_flag_actually_submits(browser, live):
|
|||||||
page.goto(f"{base}/b/g/", wait_until="networkidle")
|
page.goto(f"{base}/b/g/", wait_until="networkidle")
|
||||||
page.evaluate("window.__noReload = 1")
|
page.evaluate("window.__noReload = 1")
|
||||||
|
|
||||||
page.keyboard.press("ArrowRight") # cursor onto the first tile
|
page.keyboard.press("ArrowRight")
|
||||||
|
# ⚠ WAIT FOR THE CURSOR TO LAND BEFORE PRESSING `f`. Firing both keys
|
||||||
|
# back to back assumed the first had finished, and `focus()` does a
|
||||||
|
# `scrollIntoView` — so under full-suite load `f` could arrive with no
|
||||||
|
# cursor set and flag nothing. It failed once in roughly five whole-suite
|
||||||
|
# runs while passing 3/3 on its own, which is the signature of a race
|
||||||
|
# rather than a defect, and a test that goes red one time in five trains
|
||||||
|
# people to ignore red.
|
||||||
|
page.wait_for_selector("figure.item.is-cursor", timeout=10000)
|
||||||
page.keyboard.press("f")
|
page.keyboard.press("f")
|
||||||
page.wait_for_selector("figure.item.is-flagged", timeout=10000)
|
page.wait_for_selector("figure.item.is-flagged", timeout=10000)
|
||||||
flagged = page.locator("figure.item.is-flagged").count()
|
flagged = page.locator("figure.item.is-flagged").count()
|
||||||
|
|||||||
@@ -45,10 +45,23 @@ def _serving(data_dir: pathlib.Path):
|
|||||||
sock = socket.socket()
|
sock = socket.socket()
|
||||||
sock.bind(("127.0.0.1", 0))
|
sock.bind(("127.0.0.1", 0))
|
||||||
port = sock.getsockname()[1]
|
port = sock.getsockname()[1]
|
||||||
sock.close()
|
# ⚠ THE SOCKET IS HANDED TO UVICORN STILL BOUND, never closed and
|
||||||
|
# re-opened by port number. The old form did bind -> getsockname -> CLOSE ->
|
||||||
|
# tell uvicorn the number, which leaves a window where the kernel can give
|
||||||
|
# that port to somebody else — and this suite runs TWO browser files that
|
||||||
|
# each start a server per test, so the other one is right there competing
|
||||||
|
# for it. Passing the live socket removes the window rather than narrowing
|
||||||
|
# it.
|
||||||
|
#
|
||||||
|
# Honest about the evidence: two different browser tests failed once each
|
||||||
|
# across full-suite runs while passing 3/3 and 5/5 on their own, which is
|
||||||
|
# the signature of contention. We cannot prove from two samples that this
|
||||||
|
# race was the cause. It is a real defect either way, and it is the only
|
||||||
|
# one visible in the harness.
|
||||||
|
|
||||||
app = create_app(data_dir, ttl_hours=24, start_sweeper=False)
|
app = create_app(data_dir, ttl_hours=24, start_sweeper=False)
|
||||||
server = uvicorn.Server(uvicorn.Config(app, host="127.0.0.1", port=port, log_level="error"))
|
server = uvicorn.Server(uvicorn.Config(app, host="127.0.0.1", port=port, log_level="error"))
|
||||||
thread = threading.Thread(target=server.run, daemon=True)
|
thread = threading.Thread(target=lambda: server.run(sockets=[sock]), daemon=True)
|
||||||
thread.start()
|
thread.start()
|
||||||
deadline = time.time() + 10
|
deadline = time.time() + 10
|
||||||
while not server.started and time.time() < deadline:
|
while not server.started and time.time() < deadline:
|
||||||
|
|||||||
Reference in New Issue
Block a user