fix(manifest)!: the size cap opened a service-wide hang; close it
The diff-scoped bug-hunt panel, four arms, artifact-only. Its strongest finding is one I created two hours earlier while hardening the reader. `stat` reports size 0 for a FIFO and 0 for a symlink to /dev/zero, so both sail under the byte cap added for the RecursionError round — and then `read_text` either blocks in read() with no EOF, so the except never runs, or allocates until the kernel intervenes. `list_booths` reads every booth on every GET / and /healthz, so ONE such file stalls the front page for the whole service, with no error and no recovery short of a restart. Reproduced before believing it (timeout returned 124). S_ISREG is checked BEFORE the size in both modules now; verified against the live service with two FIFOs planted, which answered 200 in 36ms. The shape worth carrying: st_size answers a different question than "can this be read", and a bound that trusts it inherits everything it does not mean. A hardening fix opened a worse hole than the one it closed. THE UPLOAD PATH WROTE ABOVE ITS OWN CLEANUP GUARD (4/4) A failed manifest write orphaned a .uploaded half-booth with no files in it — and because the temp name now carries a random suffix, nothing ever overwrote the leak, and .booth.json.<hex>.tmp is not a .lock, so _newest_mtime counted it and kept that empty booth past every sweep. The uniqueness fix from the previous round is what made the leak permanent. Both writes moved inside the guard; the temp is removed on every exit path. DAMAGED BYTES ARE KEPT, NOT REPLACED (4/4, INV-6) Marks made this explicit in v0.2.1 and this write path contradicted it: a manifest that failed on ONE field lost the others with it, including a why the re-announcer may never have kept anywhere. It diverges from marks in HOW it honours the rule — marks refuse and answer 409 because the operator's judgment is not restatable; a manifest quarantines and proceeds, because refusing would fail `booth add` and lose the files it was copying. ONE OPENNESS PREDICATE, AS U2 SAID (2/4) `booth answer` spelled out `if m.answer is None` while `booth marks` asked `open_marks`, so a partially-answered pick read as done to one verb and open to the other — at the same instant, on the same booth. U2's INV-2 put openness in one function precisely so they could not drift. The mirror case is fixed too: a pick that hydrates broken is refused by the web route, so `answer --wait` polled an hour on a form nothing could ever land. ALSO - now_stamp was whole-second while the importer had moved to microseconds, and '-' sorts before '.', so a later mark came out ahead of an earlier import inside the same second. One format; the previous round's ordering fix had opened this one. - `_broken` was the third of three directory-name fallbacks and the one still handing a raw name into a card's sub-line. - An identical re-announce rewrote the file and reset the TTL. `booth link` does this on every post to the standing board. - The importer's return went through the bare _hydrate, not _hydrate_safe. - A marks document could be written larger than it can be read back, and then read as no marks at all. Refused at the write instead. - `choice` reached the answer builder raw while `notes` beside it did not. AND ONE FINDING DELIBERATELY NOT FULLY CLOSED The mtime-restore race is real. The clean fix — ignore a booth directory's own mtime whenever the booth holds anything — also silently retires the documented rule that releasing a kept board resets its clock, which the CLI header, the README and a deliberately-written test all pin. That is a TTL doctrine change, not a bug fix, and an existing test caught the attempt. The concrete half is fixed (a failing os.utime escaped and 500'd the route); the race is stated in the code where the next reader will meet it. 341 tests. Live service restarted, 24/24 booth pages verified.
This commit is contained in:
+47
-11
@@ -59,6 +59,34 @@ _As of 2026-09-22:_
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-09-22]` **The U5 bug-hunt panel found a service-wide hang that the
|
||||
SIZE CAP ITSELF opened — two hours after I added the cap.** `stat` reports
|
||||
size 0 for a FIFO and 0 for a symlink to `/dev/zero`, so both sail under a
|
||||
byte cap and then `read_text` blocks with no EOF or allocates until the kernel
|
||||
intervenes. `list_booths` reads every booth on every `GET /`, so ONE such file
|
||||
stalls the front page for the whole service with no error and no recovery
|
||||
short of a restart. Reproduced (`timeout` returned 124), fixed with an
|
||||
`S_ISREG` check BEFORE the size check in both modules, verified live: the
|
||||
index answered 200 in 36 ms with two FIFOs planted. **The reusable shape:
|
||||
`st_size` answers a different question than "can this be read", and a bound
|
||||
that trusts it inherits everything it does not mean — a hardening fix opened
|
||||
a worse hole than the one it closed.** Also adopted: the upload path wrote the
|
||||
manifest ABOVE its own cleanup guard (4/4), so a failure orphaned a half-booth
|
||||
whose uniquely-named leaked temp then kept it alive forever; replace-over-
|
||||
damaged destroyed recoverable bytes (4/4, now QUARANTINED rather than refused
|
||||
— marks refuse because judgment is not restatable, a booth's description is);
|
||||
and `booth answer` spelled out its own openness test, disagreeing with
|
||||
`booth marks` about a partially-answered pick, which is a direct violation of
|
||||
U2's INV-2. Full triage in `persistent-memory.d/2026-09-22-u5-panels.md`.
|
||||
- `[2026-09-22]` **An existing test stopped me retiring documented behaviour
|
||||
while fixing a race.** The mtime-restore race is real, and the clean fix —
|
||||
ignoring a booth directory's own mtime whenever the booth holds anything —
|
||||
would also have silently retired the rule that RELEASING a kept board resets
|
||||
its clock, which the CLI header, the README and a deliberately-written test
|
||||
all pin. That is a TTL doctrine change, not a bug fix. Fixed the concrete half
|
||||
(a failing `os.utime` used to escape and 500 the route), left the race stated
|
||||
in the code. **A fix that changes a documented rule is a proposal, not a
|
||||
patch.**
|
||||
- `[2026-09-22]` **Two cross-frontier panels on U5, and a paraphrase panel reached
|
||||
a production outage two modules away.** 3-of-4 flagged the contract's "4 GB"
|
||||
case as letter-compliant but purpose-defeating; the conformance round found that
|
||||
@@ -76,17 +104,25 @@ _As of 2026-09-22:_
|
||||
`booth add` wiped the `why` on the one sequence the feature exists for, and
|
||||
`--title` was write-only. Full triage in
|
||||
`persistent-memory.d/2026-09-22-u5-panels.md`.
|
||||
- `[2026-09-22]` **U5's adoption is a stated, falsifiable prediction — RE-MEASURE
|
||||
2026-09-29.** Operator declined the fleetwide announcement and chose to let the
|
||||
convention sit, explicitly so that adoption can be told apart from design:
|
||||
**count how many booths under `~/booth-data` carry a `.booth.json` a week
|
||||
after landing.** A near-zero count means the convention was never heard of
|
||||
(an adoption failure — fix by announcing), NOT that the shape was wrong. A
|
||||
non-zero count from handles that only read the README means it propagates on
|
||||
its own. Baseline at landing: **0 of 26**. Same instrument as the `.forever`
|
||||
prediction below, and for the same reason — a diagnosis nobody re-measures is
|
||||
a belief.
|
||||
`find ~/booth-data -maxdepth 2 -name .booth.json | wc -l`
|
||||
- `[2026-09-22]` **U5's adoption prediction, SPLIT IN TWO within an hour of
|
||||
landing — and the split is the interesting part.** The baseline was recorded as
|
||||
0 of 26. Fifty minutes after the deploy, `comfy-dev` created `muse-clothed-repro`
|
||||
and it announced itself: `{handle: comfy-dev, why: "", created: ...}`. That peer
|
||||
was told nothing. **The HANDLE propagates for free** — it rides on `booth new`
|
||||
and `booth add`, so every existing CLI caller starts announcing without learning
|
||||
anything, which is the flags-on-existing-verbs decision paying off on day zero.
|
||||
**The WHY does not** — it needs someone to know the flag exists, and this first
|
||||
one is empty.
|
||||
|
||||
So re-measure BOTH on **2026-09-29**, because they answer different questions:
|
||||
|
||||
find ~/booth-data -maxdepth 2 -name .booth.json | wc -l # free
|
||||
grep -l '"why": "[^"]' ~/booth-data/*/.booth.json 2>/dev/null | wc -l # learned
|
||||
|
||||
A high first count and a near-zero second is the predicted shape of "nobody was
|
||||
told", and it is the case the operator's no-announcement decision was designed
|
||||
to be able to see. Do not read the n=1 above as a rate — it is a code-path
|
||||
observation (every CLI caller writes a handle), not a sample.
|
||||
- `[2026-09-22]` **The U2 bug-hunt panel landed and it was not ceremony —
|
||||
`v0.2.2`.** Nine adopted findings across four arms; eight were real against
|
||||
live code and one was already fixed. The headline was **4/4 convergent from
|
||||
|
||||
Reference in New Issue
Block a user