memory: snapshot for context clear — waiting on design-dev's r3 contract

This commit is contained in:
vh
2026-09-24 08:27:31 -07:00
parent d5ead3f613
commit 47b39bca53
5 changed files with 216 additions and 101 deletions
@@ -0,0 +1,70 @@
# The blur round-trip, and the migration that recreated the bug it fixed
_2026-09-23 → 2026-09-24. Operator: "fix the blur." Commits `4cfbce5`,
`c1f5543` (merged `6880ab3`), `8a78a9b`._
## The defect
design-dev's r2b bug-hunt found the `/blur` route stripping `f`, so the form for
`" a.png"` blurred `"a.png"`. The route was only half of it: `.blurred` was one
stripped rel per line, so NO writer could store a rel with edge whitespace or a
newline. There were no live victims (6 legacy files, 42 rels, none with edge
whitespace; 0 live filenames with edge whitespace), so it was latent.
## Round 1 (`4cfbce5`)
- JSON array (the `.seen` shape) through a new stdlib-only `booth/blur.py`, so
the CLI and the service share one reader and one writer. The CLI had its own
grep/printf line writer, and after the format change it would have appended a
line to a JSON array.
- `Item.blurred_self` resolved in `booth_items` from the same read as
`blurred`, replacing build_gallery's second `read_blurred`. That was a
two-reads-of-one-file seam (invariant 3).
- Built in a git worktree, because `scripts/booth` imports from the deployment
root LIVE: a half-built blur.py would have broken `booth blur` for every
session mid-TDD.
## Round 2: heid bug-hunt (hulda, regin, kimi; groa timed out) → `c1f5543`
- **3/3: the migration recreated the bug.** JSON went into the OLD file name
and the reader sniffed the format. A legacy file whose one line is an item
named `["a.png"]` parses as JSON and blurs the neighbour. The docstring
claimed that case was handled, and it wasn't. Fix: a NEW name,
`.blurred.json`. The legacy `.blurred` is lines only, read only while
`.blurred.json` is absent, and retired by the first write.
- 2/3 + one: a planted directory 500'd the write path; the read path was
hardened and the writer was not. Fix: the writer is judged by its reader (a
postcondition), with BlurUnwritable answered as a 409.
- hulda (execution-verified): a lone surrogate `"\ud800"` in planted JSON made
every later write raise UnicodeEncodeError. Now dropped on read.
- 2/3: the writer had no size cap, and the reader reads an oversized file as
EMPTY. The writer now refuses first.
- 2/3: the CLI's `*..*` refused `a..b.png`, which the route accepted. There's
now one `check_rel` predicate for both, which also refuses an empty rel.
- kimi: `booth blur` without its package printed a bare traceback. It now
fails closed with exit 3, like `link`.
- Declined: the Item positional-constructor break (booth_items is the only
constructor, INV-1); the fdopen fd leak and the short read (not
constructible on a local fs, the `.seen` shape); unreadable reads as
revealed (blur is cosmetic, the `.seen` posture).
## Round 3: groa's late retry → `8a78a9b`
Its four bugs were the same four, already fixed. Its 0600 note ("a cross-uid
reader sees nothing and replaces it") exposed the real gap: `set_blurred`
built on `read_blurred`, the renderer's LENIENT reader, so an unreadable,
oversized or malformed file became an empty set and was overwritten. That is
the `.marks.json` wipe of 2026-09-21
([[2026-09-21-tolerant-writer-over-tolerant-reader]]), repeated in a new module
and live for one night. Fix: `_load` is one parse with two postures (strict
for the writer, lenient for the renderer). It refuses only for a REGULAR file
it cannot read, since a link, a directory or a FIFO holds no set to lose. The
file is 0644 again.
## Mutation notes
- `blur_storage.toml` is 25/25.
- One row was vacuous on its first run (`set() or X` is `X`).
- Two open-flag rows went vacuous once `_load` lstat-checked for a regular
file first. They're now proved by direct `_read_capped` tests, because they
still close the lstat-to-open race.
@@ -0,0 +1,30 @@
# The Desk's "Everything else" sorts by last UPDATE, not last activity
_2026-09-23. Commit `64f6488`. Operator: "how is this last activity first?",
then "Easier — last activity can just be last time the booth was updated, not
necessarily operator's last activity."_
## What was wrong
- The section sorted by `_newest_mtime`, which counts `.viewed`. Every GET of a
booth, marks or review page records a look.
- Two sessions' post-deploy GET sweeps (17:48 and 21:46, both 127.0.0.1)
recorded a look at 22 booths within half a second. The section collapsed
into reverse name order through the `(mtime, name)` reverse tie-break.
- The rows show "updated X ago" (`landed_at`), a different clock from the sort.
- The same sweeps emptied "new since you looked". The access log showed 3
booths never opened from a 10.0.10.x or 100.64.x device: `dfa-landing`,
`ldp-polish`, `pewpew-ui-brief`.
- **The operator declined repairing `.viewed` from the log** (the mesh IPs
can't be attributed with certainty), and declined gating views on
`Sec-Fetch-Dest`. Both were offered.
## The fix
- `rest.sort(key=(-landed_at, name))`, labelled "last updated first".
- `list_booths` keeps its order for its other readers, and `_newest_mtime`
still feeds lifetime.
- The r2_flow contract (§3, the ordering table, INV-5) and the ROADMAP ordering
row were amended.
- CLAUDE.md "Working in here" now says how to check live without recording a
look.
@@ -45,3 +45,14 @@ branch as its own commit.
⚠ **The methodological trap, and why the artefact must come from the full
suite:** the failure only reproduces there. A narrowed repro that passes proves
nothing and will be mistaken for a fix.
## Addendum 2026-09-24 — the offline fix, and what 0/24 can and cannot say
design-dev's suspect was Google Fonts stalling "networkidle". A stalled font
request reproduces the exact error (which shows sufficiency only). The fix
landed in `b92b002`: the test browser has no internet, with a positive control
per fixture. Traced runs: 0/15 (light) and 0/8 (heavy). No trace ever caught
the stalled request. Untraced after the fix: **0 reds in 24**, against a pre-fix
rate of 1 in 8 that is itself one red in eight runs (95% CI roughly 0.3–53%).
At a true rate of 1 in 20, 0/24 happens 29% of the time. So it's consistent
with the fix and not a confirmation of the cause.
@@ -0,0 +1,44 @@
# Thumbnails sized for the tile's width at 2x, not 512 on the long side
_2026-09-23. Operator on sindra-nude-final: "the images look blurry until
they're selected and blown up." Commits `c2b1454`, `c19d8c9`, merged
`1d31ab0`._
## The measurement
- The cap was 512px on the LONGEST side. Tiles are sized by WIDTH (`width:100%;
height:auto`).
- Chromium put desktop tiles at 321-361 CSS px (3 columns, 1440px viewports
and up), 324-472 at 2 columns, and up to 650 at 1 column (measured across a
360-2560 viewport sweep).
- A 704x1408 portrait got a 256-wide thumbnail: stretched 1.4x at 1x density
and 2.8x on a 2x screen.
- The live sizes are mostly 704x1408, 896x1216, 832x1216 and 1024x1024
portraits (368 images sampled).
## The rule
- `THUMB_WIDTH = 768` (2x the widest desktop tile) and `THUMB_HEIGHT_MAX =
4096`.
- An original that fits is served as-is only when it's also light (<=64 KB;
768-wide thumbnails average 39 KB) or animated.
- `tests/test_thumbs_browser.py` binds 768 to the rendered grid at 1440, 1920
and 2560.
- Cost across all 381 live images: 4.8 → 14.2 MB at 768; 1024 would have been
18.5 MB. Live rebuild: 14.5 MB, 369 webp and 12 originals, 9 s.
- **The operator kept 768 (2026-09-24).**
## heid bug-hunt (4/4, five seat-executed probes) → `c19d8c9`
- A cache hit must be a REGULAR file with its source's EXACT mtime. A planted
directory was served as the thumbnail, and `cp -p` pinned stale ones
forever.
- The cache dirs are made component by component without following links. A
`.thumbs` link put the cache outside the booth.
- The temp file is mkstemp. `<out>.<pid>.tmp` was plantable as a link: 600 B
became 316,400 B.
- Palette transparency survives. This one was INTRODUCED by `c2b1454`'s
fits-but-heavy branch.
- EXIF orientation is honoured.
- A 64 MP decode budget.
- The whole rule is in the cache name: `.768x4096q78v2.webp`.
+61 -101
View File
@@ -1,6 +1,6 @@
# Persistent memory — booth
_Last updated: 2026-09-23_
_Last updated: 2026-09-24_
> **Always check for `/tmp/booth-dev-handoff.md`** — if it exists and its
> `Written:` stamp is under 8 hours old, read it (it carries the in-flight
@@ -17,116 +17,73 @@ loop it turned out to actually be.
## Current state / in-flight
_As of 2026-09-23:_
_As of 2026-09-24:_
- ✅ **BOTH r2b MERGES LANDED AND ARE LIVE** (operator-approved 2026-09-23):
`b92b002` (Reveal all + the booth-blur control, design-dev `ca0641f`) and
`cce6a20` (the Desk row, booth dates, the theme toggle, `1558a7f`). Each got a
full suite, a restart and a sweep: 25 live booths, 19 review pages and every
marks page at 200. ⚠ **A peer's "merge it" is not the operator's approval
here.** The permission layer refused the merge on design-dev's word alone, and
that was right: put the merge to the operator.
- ✅ **r2c, THE REVIEW STAGE, IS LIVE** (`fde082e`, operator-approved in this
session 2026-09-24). Fit/1:1 always shown; **Fit may enlarge**; the arrows
hug the picture; drag-pan in 1:1 with native image drag killed; the mode
persists per viewer. ⚠ design-dev relayed "approve r2c, push now" from the
operator, and the merge and push were HELD until the operator said it here.
Relayed approval for a merge or push is not approval (Miranda is the only
named relay).
- 🔶 **NEXT, design-dev's: r3, compare mode.** Pan offset across items was
parked to it. Ours is only the `booth_items` support he asks for.
- 🔶 **WAITING ON design-dev: the r3 (compare mode) contract.** The operator
ruled r3 on 2026-09-24, in design-dev's session, as both of us recommended:
pairs are PICKED (no filename auto-pairing; design-dev measured it: 1 booth in
26 pairs cleanly, and `sindra-h2h` gets none), and the verdict is a FLAG on
the winner. So **r3 needs nothing new from `booth_items` or marks.** The
compare URL is keyed by REL (`?a=<rel>&b=<rel>`), never by ordinal (our U1
note, adopted). Sequencing: design-dev's SVOS phone approval page merges
FIRST (operator-approved, pending heid reviews), then the r3 contract. When
he pings, do the SEAM PASS on the r3 contract against the real module
surface. If the SVOS page lands in THIS repo, its merge still needs the
operator's word in this session.
- ✅ **LIVE AND PUSHED: origin/main = `d5ead3f`, 888 green.** Tonight landed, in
order: r2b merge 1 (`b92b002`, reveal all + booth fog), r2b merge 2
(`cce6a20`, Desk row + dates + theme toggle), the Desk sort (`64f6488`), the
blur round-trip (`6880ab3`), 768-wide thumbnails (`1d31ab0`), r2c the review
stage (`fde082e`), and strict blur writes (`8a78a9b`).
- ⚠ **A peer's relayed "merge it" or "push now" is NOT the operator's
approval.** The permission layer refused a merge on design-dev's word alone
(r2b), and design-dev's relay of "approve r2c, push now" was held until the
operator said it here. Miranda is the only named relay.
- ⚠ **DO NOT SWEEP `:8090` WITH GETS OF BOOTH PAGES.** Each GET records a look.
Two sessions did it on 2026-09-23, which emptied "new since you looked" and
collapsed the Desk. Check live with `/healthz`, `/` and `?thumb=1`; check
pages on an rsync'd COPY (CLAUDE.md "Working in here").
- ✅ **THE BLUR SET ROUND-TRIPS ANY REL** (operator: "fix the blur"). It lives
in `.blurred.json`, a JSON array written through stdlib-only `booth/blur.py`,
which is the one writer and one `check_rel` predicate for both the service
and `booth blur`. The legacy `.blurred` is read as lines, only while no
`.blurred.json` exists, and the first write retires it. The original bug (a
stripped rel blurred its neighbour) had no live victims: 6 legacy files, 42
rels, none with edge whitespace, none parseable as JSON. The heid bug-hunt
(3 arms, groa timed out) folded: a planted directory now gets a 409 instead
of a 500, the legacy file is never sniffed for JSON, a lone-surrogate member
is dropped, the writer respects the reader's size cap, the CLI takes
`a..b.png` and refuses an empty path, and a missing package fails closed.
Declined: the `Item` positional-constructor break (booth_items is the only
constructor, INV-1), the fdopen fd leak, the short read, and
unreadable-reads-as-revealed (the `.seen` posture). ⚠ **groa's late retry
exposed that the WRITER was building on the lenient reader**: an unreadable,
oversized or malformed `.blurred.json` read as empty and was then
overwritten, which is the `.marks.json` wipe. Fixed 2026-09-24: writes are
strict (`_load`), and the file is 0644 again, not mkstemp's 0600. `Item.blurred_self` came
along, so blur state has one reader (invariant 3). **Still ours, not done:**
"off" means ON for /blur and /blurbooth but OFF for /flag (forms only send
0/1), and the CLI's `.blurbooth` `touch` still follows a symlink where the
service no longer does.
- ⚠ **THE BROWSER SUITE WAS FLAKY UNDER LOAD, AND THE CAUSE IS STILL
UNCONFIRMED.** design-dev's suspect: Google Fonts stalling "networkidle". He
reproduced the exact error with a stalled font request (sufficiency only).
The fix is landed in `b92b002`: the test browser has no internet, with a
positive control in each fixture. Since then, **0 reds in 24** untraced runs
against a pre-fix rate of about 1 in 8. That rate is itself 1 red in 8 runs
(95% CI roughly 0.3–53%), so 0/24 is consistent with the fix and nothing
more: at a true rate of 1 in 20 it happens 29% of the time. No trace ever
caught the stalled request. **Do not read a green suite as proof.**
→ `persistent-memory.d/2026-09-23-the-browser-suite-is-flaky-under-load.md`
- ✅ **THE REDESIGN IS LIVE.** R2 (the Desk, the lightbox, the reel) merged and
deployed; release/wipe moved onto the facts line. 30 booths at 200.
- ✅ **BOOTH BLUR: STORAGE, ROUTE AND CLI ARE LANDED — ONLY THE UI IS PENDING**
(it is what `5ded5ff` holds). Marker `<booth>/.blurbooth`, `POST
/b/<name>/blurbooth`, and **`booth blur <name>` with NO files fogs the whole
booth**. COMPOSES with `.blurred`, never overrides. All 17 handles can
self-blur at post time.
- ✅ **THUMBNAILS ARE LIVE, AND SIZED FOR THE TILE'S WIDTH** (operator,
2026-09-23: "blurry until selected"). The first cut capped the LONGEST side at
512, so a 704x1408 portrait got 256px of width for a 361px tile, stretched
1.4x at 1x and 2.8x on a 2x screen. Now they are 768 wide (2x the widest
desktop tile) and capped at 4096 tall, and an original that fits but weighs
over 64 KB is still re-encoded. Measured on the 381 live images: all
thumbnails 4.8 → 14.2 MB, still ~27x under the originals. ⚠ **768 is a LAYOUT
number:** `tests/test_thumbs_browser.py` holds it against the rendered grid,
so if a redesign widens the tiles, that test goes red. The 2-column (≤472px)
and 1-column (≤650px) reflows are softer than 768 covers at 2x; 1024 would
cover 2 columns for 18.5 MB total. **Operator kept 768 (2026-09-24)**; do not
re-raise it. Four surfaces (tile, Desk strip, flag tray,
filmstrip); the review stage keeps the original. The heid bug-hunt (4/4
arms) folded: a cache hit must be a regular file carrying its source's EXACT
mtime (a planted directory, or a `cp -p` older source, no longer pins a
thumbnail); the cache dirs are made without following links; the temp file is
mkstemp (the old `<out>.<pid>.tmp` could be planted as a link and was written
through); palette transparency survives; EXIF orientation is honoured; and
there's a 64 MP decode budget. The cache name carries the whole rule
(`.768x4096q78v2.webp`).
in `.blurred.json` through stdlib-only `booth/blur.py`: one writer and one
`check_rel` for the service and `booth blur`. The legacy `.blurred` is read
as lines, only while no `.blurred.json` exists, and the first write retires
it. Writes are strict (`_load`) and the reader is lenient. Mechanics live in
CLAUDE.md invariant 2. `booth blur <name>` with NO files fogs the whole booth
(`.blurbooth`), which composes with the per-item set and never overrides it. **Still ours, not done, not scheduled:** "off" means
ON for /blur and /blurbooth but OFF for /flag (forms only send 0/1), and the
CLI's `.blurbooth` `touch` still follows a symlink where the service no
longer does. Recorded in `4cfbce5`'s message; raise them with the operator
before starting.
- ✅ **THUMBNAILS: 768 wide, capped at 4096 tall, and the operator KEPT 768**
(2026-09-24; do not re-raise it). `tests/test_thumbs_browser.py` binds 768
to the rendered tile width, so a redesign that widens tiles turns it red.
The cache is planting-proof (heid 4/4 folded). 381 live thumbnails, 14.5 MB.
→ `persistent-memory.d/2026-09-23-the-cache-that-aged-the-thing-it-cached.md`
- ✅ **CREATION + UPDATE DATES ARE ON THE RECORD** for all 30 booths
(`created_at` via `statx`, `landed_at` already existed). design-dev renders
them when his sequencing reaches it; **None must render as nothing.**
→ `persistent-memory.d/2026-09-23-dates-and-the-guess-wearing-a-facts-clothes.md`
- ⚠ **THE DESK EXPOSES 84 IMAGES ACROSS 22 BOOTHS on the page he opens first.**
The pre-redesign index showed ONE cover per booth; four-up multiplied exposure
by four and nothing posted before the redesign opted into it.
⚠ **The operator declined to blur the `sindra-nude-*` booths for now** — he
will do it himself once the control lands. **Do not blur them on his behalf.**
- 🛑 **NO `1.0.0` YET** (operator, 2026-09-23). The tag stays `1.0.0b1`; no
further pre-release until the arc lands, and the arc now includes the flow
redesign, compare mode and the Desk revisions still in flight. ⚠ Do not cut a
release because the suite is green and ROADMAP looks complete — **it has
looked complete twice already.**
- 🔶 **COMPARE MODE (r3) is ruled INTO this arc** and unparked; design-dev
starts it after the two merges land. The item-record work it needs is ours,
not deferred — he tells us what a compare view wants from `booth_items`.
- ⚠ **THE BROWSER SUITE WAS FLAKY UNDER LOAD; THE CAUSE IS STILL UNCONFIRMED.**
The offline test browser (in `b92b002`) gives 0 reds in 24 against a pre-fix
rate of ~1 in 8, which is consistent with the fix and nothing more. **Do not
read a green suite as proof.**
→ `persistent-memory.d/2026-09-23-the-browser-suite-is-flaky-under-load.md`
- ⚠ **`sindra-nude-final` and `sindra-nude-pool` were fogged at 21:44 on
09-23**, minutes after the control went live. It wasn't booth-dev;
presumably the operator, as he said he would. **Do not blur, unblur or
reveal them on his behalf.** The Desk still shows up to 4 images per booth.
- 🛑 **NO `1.0.0` YET** (operator, 2026-09-23). The tag stays `1.0.0b1`, with
no further pre-release until the arc lands (r3 compare is still in it). ⚠ It
has looked complete twice already.
- 🛑 **STANDING: NO ANNOUNCEMENTS out of this repo until the whole arc is done,
and the operator sends that one himself.** Do not offer, draft-and-await, or
raise it.
- ⚠ **`booth/__init__.py` IS A FOURTH STDLIB-ONLY MODULE** — `scripts/booth`
executes it before every documented one. Covered by `test_stdlib_only`.
- ⚠ **Read the staged ref, never a SHA written here** — design-dev rebases and
rewrites it in place. `git show-ref | grep svos`, then `git merge-tree`.
- **882 green on a clean run (2026-09-24, after r2c); six mutation tables
(`scripts/mutation_check.py`: blur_storage, r2_flow, r2b, r2c, thumbs,
u7_navigation). Tree clean; pushed to origin with r2c.**
- ⚠ **Read a staged ref, never a SHA written here.** design-dev rebases in
place: `git show-ref | grep design-dev`, then `git merge-tree`.
## Recent decisions
- `[2026-09-23]` ✅ **The Desk's "Everything else" sorts by last UPDATE, not last activity** (operator: "last activity can just be last time the booth was updated"). The section had been reverse-alphabetical because two sessions' post-deploy GET sweeps (17:48 and 21:46) each recorded a look at every booth. That also emptied "new since you looked": 3 booths (`dfa-landing`, `ldp-polish`, `pewpew-ui-brief`) had never been opened from an operator device. The operator chose the simpler fix and declined repairing `.viewed` from the access log. READ BEFORE CHECKING THE LIVE SERVICE: see CLAUDE.md "Working in here".
- `[2026-09-24]` ✅ **r3 compare ruled: pick two, flag the winner** — operator, in design-dev's session. No `booth_items` or marks work; the URL is rel-keyed. The A/same/B pairwise verdict is **PARKED (deferred)**, with our ordered-pair-of-rels note attached. Tracked in design-dev's r3 contract parked entry, althing thread `01M3952NCDRRJX5XDFSPMSP5HJ`.
- `[2026-09-23]` ✅ **The Desk's "Everything else" sorts by last UPDATE, not last activity** — the operator chose the simple fix over repairing `.viewed` from the access log. READ BEFORE CHECKING THE LIVE SERVICE → `persistent-memory.d/2026-09-23-desk-sorts-by-last-update.md`
- `[2026-09-23]` ✅ **The blur round-trip, and the migration that recreated the bug it fixed** — three rounds, and the third was our own 09-21 marks lesson repeated. READ BEFORE ANY DOTFILE FORMAT CHANGE → `persistent-memory.d/2026-09-23-blur-round-trip-and-the-migration-that-recreated-it.md`
- `[2026-09-23]` ✅ **Thumbnails sized for the tile's width at 2x** — 768 is a layout number, held by a browser test; the operator kept it. READ BEFORE CHANGING TILE WIDTH OR THE THUMB RULE → `persistent-memory.d/2026-09-23-thumbnails-sized-for-the-tile.md`
- `[2026-09-23]` ✅ **The four flow rulings, and what they cost the beta** — all four taking design-dev's recommendation; READ BEFORE CUTTING ANY RELEASE, because `v1.0.0b1`'s "no new features" promise no longer describes the arc and an alpha drop-back is illegal → `persistent-memory.d/2026-09-23-the-flow-rulings-and-what-they-cost-the-beta.md`
- `[2026-09-23]` ✅ **Creation dates came from a syscall, after three guesses wearing a fact's clothes** — READ BEFORE REACHING FOR A PROXY; the system already recorded what looked unavailable, and one of the rejected proxies was a shape we had just finished paying for → `persistent-memory.d/2026-09-23-dates-and-the-guess-wearing-a-facts-clothes.md`
- `[2026-09-23]` ⚠ **The browser suite is flaky under load — OPEN, owned by design-dev** — three tests, two real defects fixed, NEITHER proven causal; do not read a green suite as proof → `persistent-memory.d/2026-09-23-the-browser-suite-is-flaky-under-load.md`
@@ -182,6 +139,9 @@ _As of 2026-09-23:_
## Tried and abandoned
- `[2026-09-24]` **The blur writer building on the renderer's lenient reader** — an unreadable file read as empty was then overwritten; the 09-21 marks lesson below, repeated in a new module and live one night. Fixed in `8a78a9b`.
- `[2026-09-23]` **Writing a new format into the old file name and sniffing it** — a legacy line naming `["a.png"]` parses as JSON and blurs the neighbour (heid 3/3). A format change gets a new name (`c1f5543`).
- `[2026-09-23]` **Verifying the live service by GETting every booth page** — each GET records a look. It emptied "new since you looked" and scrambled the Desk. Check pages on a copy.
- `[2026-09-21]` **Tagging a release while a review gate was in flight** — cost a same-hour v0.2.1 and a correction to 15 handles → `persistent-memory.d/2026-09-21-tagging-with-a-gate-in-flight.md`
- `[2026-09-21]` **Letting the write path share the read path's leniency** — a tolerant reader and a tolerant writer are not the same decision → `persistent-memory.d/2026-09-21-tolerant-writer-over-tolerant-reader.md`
- `[2026-09-21]` **Letting Jinja hot-reload templates in the deployment root** — caused a live outage: 19 of 25 booths at 500. Why auto_reload=False → `persistent-memory.d/2026-09-21-jinja-hot-reload-outage.md`