Files
booth/persistent-memory.d/2026-09-23-the-browser-suite-is-flaky-under-load.md
T
vh 37d859c0fd memory: snapshot for context clear — the arc mid-flight, and the one thing that blocks
In-flight rewritten to what is actually live: design-dev's blur merge is HELD
at 5ded5ff awaiting his explicit 'merge it' ping (both panels dispatched 17:53,
unfolded), the redesign and thumbnails and dates are shipped, and the browser
suite is flaky under load and NOT fixed.

Two detail files added. The dates one is the reusable lesson: three plausible
proxies for a creation date were considered and one was nearly built, and the
real answer was a syscall away — the system already recorded what looked
unavailable. One of the rejected proxies was write-on-read, a shape this repo
had finished paying for hours earlier.

The flake entry is written as OPEN with its limits stated: three tests, two
real defects fixed, neither proven causal, and n=3 cannot show an improvement.

Recent decisions and Tried and abandoned preserved intact (49->51 by addition,
7 unchanged); the index is back under the soft cap at 141 lines from 285, all
of the reduction from settled history leaving the volatile section.
2026-09-23 17:56:50 -07:00

2.2 KiB

The browser suite is flaky under load — UNRESOLVED, and owned by design-dev

2026-09-23 · booth

⚠ OPEN. Not fixed. Do not read a green suite as proof of anything without re-running it.

What is observed, with its limits

Three different browser tests have each failed once under full-suite load while passing repeatedly in isolation:

test owner isolation full suite
test_the_keyboard_flag_actually_submits booth-dev 5/5 pass 1 failure
test_a_failed_save_says_so_reloads_and_never_re_posts design-dev 3/3 pass 1 failure
test_the_review_keys_judge_in_place_and_stay_out_of_the_note design-dev 10/10 + 5/5 pass 1 in ~9

Three different tests points at the environment under contention, not at any one test. That is a direction, not a finding.

Two real defects found chasing it — NEITHER PROVEN TO BE THE CAUSE

  1. A keypress race. The flag test fired ArrowRight and f back to back, assuming the first had finished — but focus() does a scrollIntoView, so under load f could arrive with no cursor set. Now waits for figure.item.is-cursor.
  2. A port TOCTOU in BOTH browser fixtures. Each did bind → getsockname → CLOSE → hand uvicorn the port NUMBER, leaving a window for the kernel to give that port away — and this suite runs two browser files that each start a server per test, so the competitor is the other file. The bound socket is now passed to server.run(sockets=[sock]).

Since those fixes: one failure in three full runs. n=3 CANNOT distinguish that from the prior rate, and no claim of improvement is made.

Who owns it and what the method is

The operator ruled: "let him diagnose it properly." design-dev owns it. His method: a trace hook keeping a Playwright trace (screenshots + DOM snapshots) for every browser test that fails, captured from the run that fails, then full-suite runs until red and the artefact decides. Test-only infra, on his branch as its own commit.

⚠ The methodological trap, and why the artefact must come from the full suite: the failure only reproduces there. A narrowed repro that passes proves nothing and will be mistaken for a fix.