fix(manifest): fold in both cross-frontier panels — and a live hole in v0.2.2
Two four-arm artifact-only rounds landed together: the contract paraphrase (against the pre-seam-review capture) and the code-vs-contract conformance review (against the amended one), correctly firewalled from each other. The conformance round found ZERO drift in the strict sense — the code is a clause-for-clause implementation of the contract — and the weight of both rounds landed one layer down, in what green tests structurally cannot report. Full triage in persistent-memory.d/. A LIVE HOLE IN RELEASED CODE, FOUND ON THE SIBLING MODULE v0.2.2 adopted the RecursionError finding from the bug-hunt round and closed half of it: `_hydrate_safe` guards hydration, but `json.loads` runs above it in `_read_raw`, whose catch list covers neither RecursionError nor MemoryError. A 400 KB file of nothing but brackets in any ONE booth therefore still returned 500 for `/` and `/healthz` across every booth on the service. Confirmed by running it before believing it. Both modules now bound the read by `stat` before touching the bytes and catch both classes anyway, so raising a bound later cannot quietly re-open the hole. The strict half of the marks asymmetry refuses everything the lenient half tolerates, or a file that reads as "no marks" gets replaced by a write that believed it. THE WHY-WIPE `booth new x --why "..."` then `booth add x out/*.png` erased the sentence the first command existed to record. Omitted flags meant empty strings and empty strings overwrote. Two arms predicted it from the contract's wording alone; every test here passed --why on both calls and so could not see it. Omitted now means unchanged and an explicit --why "" still clears — the shell carries the distinction by leaving the variable UNSET, not empty. --title WAS WRITE-ONLY Stored, flag-surfaced, rendered nowhere. 4/4, and independently top-ranked by every arm of the paraphrase round. It lands on the booth page heading with the directory name beside it, because the directory name is the identity the operator navigates by and refers to positionally. THREE TESTS THAT COULD NOT FAIL - test_the_write_is_atomic asserted no *.tmp survived, which a plain write_text passes. It asserts the inode changes now. (The first replacement was ALSO vacuous — it spied on os.open, which Path.write_text reaches through io.open in C and never touches. Recorded in the test, because writing a second vacuous test while fixing the first is exactly the failure this round is about.) - The INV-3 preservation test passed against an implementation that regenerated `created` every time, because _now() is whole-second resolution and back-to-back writes share a stamp. Seeded from 2019 now. - test_announcing_is_activity passed whether or not _newest_mtime counted the manifest, because writing it bumps the directory mtime either way. The directory's clock is put back, leaving the file as the only thing that can keep the booth alive. ALSO - The title fallback skipped the normalizer the explicit value gets; a directory name may legally carry a newline and run to 255 bytes. - Every writer derived the same .booth.json.tmp. Marks are protected from that by their flock; the manifest has none, so uniqueness stands in. - test_stdlib_only was blind to relative imports in all four modules. - INV-1 had no guard at all; INV-5 named two different promises; the negative render states were asserted on the index only. Contract amended throughout: the 4 GB case is a stat-checked bound rather than a return constraint, every field of an error-carrying record has a stated value, INV-1 no longer contradicts INV-3, repo-wide rules are named in words instead of by a colliding number, and touches admits the macro partial the implementation added. 329 tests.
This commit is contained in:
Vuong Hoang
+75
-18
@@ -26,6 +26,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
@@ -38,6 +39,14 @@ MANIFEST_FILE = ".booth.json"
|
||||
HANDLE_MAX = 64
|
||||
TITLE_MAX = 120
|
||||
WHY_MAX = 200
|
||||
CREATED_MAX = 64
|
||||
|
||||
# A manifest is four short fields. Anything near this is not one, and reading it
|
||||
# into memory to find that out is the wrong order of operations: `list_booths`
|
||||
# calls the reader once per booth on every index load, so an unbounded read is
|
||||
# the service-wide outage the lenient reader exists to prevent, arriving in a
|
||||
# different costume. Checked by `stat`, before the bytes are touched.
|
||||
MANIFEST_MAX_BYTES = 64 * 1024
|
||||
|
||||
# The handle a booth created by the service itself carries. A pickup booth and
|
||||
# the standing link board are made by the Booth, not by an agent, and saying so
|
||||
@@ -62,11 +71,27 @@ class Manifest:
|
||||
|
||||
|
||||
def _one_line(value, limit: int) -> str:
|
||||
"""One line, bounded. Collapses ALL runs of whitespace, not only newlines —
|
||||
a tab or a forty-space indent in a `why` renders as badly inside a card's
|
||||
sub-line as a newline does, and the field is one line by construction."""
|
||||
if not isinstance(value, str):
|
||||
return ""
|
||||
return " ".join(value.split())[:limit]
|
||||
|
||||
|
||||
def _temp_path(booth: Path) -> Path:
|
||||
"""A scratch name no other writer will pick.
|
||||
|
||||
Every writer used to derive the same `.booth.json.tmp`, so two `booth add`
|
||||
calls on one booth could interleave through a stale descriptor into the
|
||||
published path. Marks are protected from that by their flock; the manifest
|
||||
deliberately has none — it is written once at creation, not read-modify-
|
||||
written per click — so uniqueness is what stands in for the lock. Still a
|
||||
dotfile, so no listing, gallery or zip can see it mid-write.
|
||||
"""
|
||||
return booth / f"{MANIFEST_FILE}.{secrets.token_hex(4)}.tmp"
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
|
||||
@@ -92,18 +117,33 @@ def read_manifest(booth: Path) -> Manifest | None:
|
||||
"""
|
||||
booth = Path(booth)
|
||||
path = booth / MANIFEST_FILE
|
||||
# BOUNDED BEFORE THE READ. "Never raises" was not true of an unbounded one:
|
||||
# a 4 GB file raises MemoryError and a deeply nested document raises
|
||||
# RecursionError out of `json.loads`, and neither is an OSError or a
|
||||
# ValueError. Both escape into `list_booths`, which calls this per booth on
|
||||
# every index load — so one file returns 500 for the whole front page. Size
|
||||
# first, by `stat`; then catch the two classes anyway, because a bound that
|
||||
# is one day raised should not quietly re-open the hole.
|
||||
try:
|
||||
size = path.stat().st_size
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as exc:
|
||||
return _broken(booth, f"cannot be read: {exc}")
|
||||
if size > MANIFEST_MAX_BYTES:
|
||||
return _broken(booth, f"is too large to be a manifest ({size} bytes)")
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except (OSError, UnicodeDecodeError) as exc:
|
||||
except (OSError, UnicodeDecodeError, MemoryError) as exc:
|
||||
return _broken(booth, f"cannot be read: {exc}")
|
||||
if not text.strip():
|
||||
return _broken(booth, "is empty")
|
||||
try:
|
||||
raw = json.loads(text)
|
||||
except ValueError as exc:
|
||||
return _broken(booth, f"is not valid JSON: {exc}")
|
||||
except (ValueError, RecursionError, MemoryError) as exc:
|
||||
return _broken(booth, f"is not valid JSON: {type(exc).__name__}")
|
||||
if not isinstance(raw, dict):
|
||||
return _broken(booth, "is not a JSON object")
|
||||
|
||||
@@ -112,9 +152,12 @@ def read_manifest(booth: Path) -> Manifest | None:
|
||||
return _broken(booth, "names no handle")
|
||||
return Manifest(
|
||||
handle=handle,
|
||||
title=_one_line(raw.get("title"), TITLE_MAX) or booth.name,
|
||||
# `or booth.name` goes THROUGH the normalizer too. A directory name may
|
||||
# legally carry a newline on POSIX and may run to 255 bytes, and the
|
||||
# fallback used to hand either straight into a card's sub-line.
|
||||
title=_one_line(raw.get("title"), TITLE_MAX) or _one_line(booth.name, TITLE_MAX),
|
||||
why=_one_line(raw.get("why"), WHY_MAX),
|
||||
created=_one_line(raw.get("created"), 64),
|
||||
created=_one_line(raw.get("created"), CREATED_MAX),
|
||||
)
|
||||
|
||||
|
||||
@@ -123,35 +166,49 @@ def _broken(booth: Path, reason: str) -> Manifest:
|
||||
error=f"{MANIFEST_FILE} {reason}")
|
||||
|
||||
|
||||
def write_manifest(booth: Path, handle: str, *, title: str = "",
|
||||
why: str = "") -> Manifest:
|
||||
def write_manifest(booth: Path, handle: str, *, title: str | None = None,
|
||||
why: str | None = None) -> Manifest:
|
||||
"""Announce a booth, atomically (CLAUDE.md invariant 5).
|
||||
|
||||
Temp file + `os.replace`, because the CLI writes this in one process while
|
||||
the browser reads it in another — a reader must never see a half-written
|
||||
document. The temp file is itself a dotfile (`.booth.json.tmp`), so no
|
||||
listing, gallery or zip can see it mid-write either.
|
||||
document. The temp file is itself a dotfile, so no listing, gallery or zip
|
||||
can see it mid-write either.
|
||||
|
||||
OMITTED MEANS UNCHANGED; `""` MEANS CLEAR. `title` and `why` default to
|
||||
None, not to the empty string, because the ordinary sequence is
|
||||
`booth new x --why "..."` and then `booth add x out/*.png` — and while
|
||||
omission meant empty, that second command silently erased the sentence the
|
||||
first one existed to record. Two arms of the contract panel predicted it
|
||||
from the wording alone; every test written for this module passed `--why`
|
||||
on both calls and so could not see it.
|
||||
|
||||
RE-ANNOUNCING PRESERVES `created` (INV-3). It is when the booth APPEARED,
|
||||
and saying something more about it later is not a second appearance —
|
||||
`booth add` on an existing booth is the common case, where the poster drops
|
||||
the second batch and sharpens the why. A `created` that cannot be read back
|
||||
is replaced rather than guessed at: a stamp that is silently wrong is worse
|
||||
than one that is silently new.
|
||||
and saying something more about it later is not a second appearance. A
|
||||
`created` that cannot be read back is replaced rather than guessed at: a
|
||||
stamp that is silently wrong is worse than one that is silently new.
|
||||
|
||||
An empty `handle` becomes `SERVICE_HANDLE` rather than being refused — a
|
||||
manifest with no handle does not read back at all, and an unreadable file is
|
||||
the worse outcome. Unreachable from the CLI, whose fallback chain always
|
||||
yields something; callers of this function directly should pass a real one.
|
||||
"""
|
||||
booth = Path(booth)
|
||||
booth.mkdir(parents=True, exist_ok=True)
|
||||
prior = read_manifest(booth)
|
||||
created = prior.created if prior and not prior.error and prior.created else _now()
|
||||
usable = prior if prior and not prior.error else None
|
||||
created = usable.created if usable and usable.created else _now()
|
||||
|
||||
record = Manifest(
|
||||
handle=_one_line(handle, HANDLE_MAX) or SERVICE_HANDLE,
|
||||
title=_one_line(title, TITLE_MAX) or booth.name,
|
||||
why=_one_line(why, WHY_MAX),
|
||||
title=(_one_line(title, TITLE_MAX) if title is not None
|
||||
else (usable.title if usable else "")) or _one_line(booth.name, TITLE_MAX),
|
||||
why=(_one_line(why, WHY_MAX) if why is not None
|
||||
else (usable.why if usable else "")),
|
||||
created=created,
|
||||
)
|
||||
path = booth / MANIFEST_FILE
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
tmp = _temp_path(booth)
|
||||
tmp.write_text(
|
||||
json.dumps(
|
||||
{"handle": record.handle, "title": record.title,
|
||||
|
||||
+31
-7
@@ -73,6 +73,14 @@ class MarksCorrupt(RuntimeError):
|
||||
"""
|
||||
|
||||
|
||||
# A booth's whole judgment lives in one document, so this is generous — a
|
||||
# 270-item booth flagged throughout, with notes, is far under it. What it rules
|
||||
# out is the case that is not marks at all: an unbounded read raises MemoryError
|
||||
# and a deeply nested one raises RecursionError out of `json.loads`, neither of
|
||||
# which is an OSError or a ValueError, and `list_booths` calls the reader once
|
||||
# per booth on every index load. Bounded by `stat`, before the bytes are read.
|
||||
MARKS_MAX_BYTES = 4 * 1024 * 1024
|
||||
|
||||
MARKS_FILE = ".marks.json"
|
||||
MARKS_LOCK = ".marks.lock"
|
||||
SCHEMA_VERSION = 1
|
||||
@@ -173,9 +181,12 @@ def _read_raw(booth: Path) -> list[dict]:
|
||||
for the same reason: a review surface that will not load is worse than one
|
||||
that has lost an annotation.
|
||||
"""
|
||||
path = Path(booth) / MARKS_FILE
|
||||
try:
|
||||
raw = json.loads((Path(booth) / MARKS_FILE).read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError, UnicodeDecodeError):
|
||||
if path.stat().st_size > MARKS_MAX_BYTES:
|
||||
return []
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError, UnicodeDecodeError, RecursionError, MemoryError):
|
||||
return []
|
||||
if not isinstance(raw, dict):
|
||||
return []
|
||||
@@ -198,18 +209,29 @@ def _read_raw_strict(booth: Path) -> list[dict]:
|
||||
case where writing would destroy something.
|
||||
"""
|
||||
path = Path(booth) / MARKS_FILE
|
||||
try:
|
||||
size = path.stat().st_size
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
except OSError as exc:
|
||||
raise MarksCorrupt(f"{path} cannot be read: {exc}") from exc
|
||||
# The strict half has to refuse everything the lenient half tolerates, or a
|
||||
# file that reads as "no marks" gets replaced by a write that believed it.
|
||||
if size > MARKS_MAX_BYTES:
|
||||
raise MarksCorrupt(f"{path} is too large to be a marks document ({size} bytes)")
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
except (OSError, UnicodeDecodeError) as exc:
|
||||
except (OSError, UnicodeDecodeError, MemoryError) as exc:
|
||||
raise MarksCorrupt(f"{path} cannot be read: {exc}") from exc
|
||||
if not text.strip():
|
||||
return []
|
||||
try:
|
||||
raw = json.loads(text)
|
||||
except ValueError as exc:
|
||||
raise MarksCorrupt(f"{path} is not valid JSON: {exc}") from exc
|
||||
except (ValueError, RecursionError, MemoryError) as exc:
|
||||
raise MarksCorrupt(
|
||||
f"{path} is not valid JSON: {type(exc).__name__}") from exc
|
||||
if not isinstance(raw, dict) or not isinstance(raw.get("marks"), list):
|
||||
raise MarksCorrupt(f"{path} is not a marks document")
|
||||
entries = [e for e in raw["marks"] if isinstance(e, dict) and isinstance(e.get("id"), str)]
|
||||
@@ -671,7 +693,8 @@ def import_legacy_asks(booth: Path) -> list[Mark]:
|
||||
continue
|
||||
try:
|
||||
decl = json.loads(p.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError, UnicodeDecodeError) as exc:
|
||||
except (OSError, ValueError, UnicodeDecodeError,
|
||||
RecursionError, MemoryError) as exc:
|
||||
found.append((mtime, stem, None, f"unreadable ask: {exc}"))
|
||||
continue
|
||||
if not isinstance(decl, dict):
|
||||
@@ -693,7 +716,8 @@ def import_legacy_asks(booth: Path) -> list[Mark]:
|
||||
loaded = json.loads(ap.read_text(encoding="utf-8"))
|
||||
if isinstance(loaded, dict):
|
||||
answer = loaded
|
||||
except (OSError, ValueError, UnicodeDecodeError):
|
||||
except (OSError, ValueError, UnicodeDecodeError,
|
||||
RecursionError, MemoryError):
|
||||
pass
|
||||
|
||||
prior = by_id.get(stem)
|
||||
|
||||
@@ -427,6 +427,10 @@
|
||||
/* Its own row under the title, not another chip in the flex line — a `why`
|
||||
can run to WHY_MAX and would otherwise shove the zip link around. */
|
||||
.boothhead .prov{flex:0 0 100%;margin-top:-.35rem}
|
||||
/* The directory name beside a manifest title: quieter than the title, but
|
||||
never absent — it is what the URL says and what "the third one" refers to. */
|
||||
.h1-slug{font-family:var(--font-mono);font-size:.62em;font-weight:400;
|
||||
letter-spacing:.06em;color:var(--fg-3);margin-left:.5rem;white-space:nowrap}
|
||||
.wipe-lg{position:static}
|
||||
/* red-outline danger button — legible on the dark canvas, fills on hover */
|
||||
.wipe-lg button{width:auto;height:auto;padding:.42rem .85rem;border-radius:var(--radius-md);
|
||||
|
||||
@@ -56,7 +56,15 @@
|
||||
{% block content %}
|
||||
<div class="boothhead">
|
||||
<a class="back" href="/">‹ all booths</a>
|
||||
<h1>{{ name }}</h1>
|
||||
{# The manifest's TITLE is the display name; the directory name stays visible
|
||||
beside it because that is the identity the operator navigates by and refers
|
||||
to positionally, and losing it would be losing the thing the URL says.
|
||||
Index cards keep the directory name alone for the same reason. #}
|
||||
{% if manifest and not manifest.error and manifest.title and manifest.title != name %}
|
||||
<h1>{{ manifest.title }} <span class="h1-slug">{{ name }}</span></h1>
|
||||
{% else %}
|
||||
<h1>{{ name }}</h1>
|
||||
{% endif %}
|
||||
<span class="sub">{% if uploaded %}<span class="badge">⬆ pickup</span> {% endif %}{% if board %}{{ board|length }} link{{ '' if board|length == 1 else 's' }}{% if items %} · {{ items|length }} file{{ '' if items|length == 1 else 's' }}{% endif %}{% else %}{% if marks_open %}<span class="badge badge-mark">{{ marks_open }} open</span> · {% endif %}{{ items|length }} item{{ '' if items|length == 1 else 's' }} · expires in {{ expires_in|dur }}{% endif %}</span>
|
||||
{% if items %}<a class="dl-link" href="/b/{{ name_url }}/?download=1" title="download this booth as a zip">⬇ zip</a>{% endif %}
|
||||
{{ provenance(manifest) }}
|
||||
|
||||
Reference in New Issue
Block a user