vh
c1f5543b77
The heid bug-hunt panel on 4cfbce5 (hulda, regin, kimi; groa timed out) found
four real defects in the round-trip fix, and three of its arms converged on the
worst: it re-created the bug it existed to fix.
- Two names, never a sniffed file (3/3). JSON went into the OLD `.blurred`, and
the reader guessed the format from the bytes, so a legacy file whose one line
is an item named `["a.png"]` read as {"a.png"} and blurred the neighbour. The
set now lives in `.blurred.json`, JSON only. The legacy `.blurred` is read as
lines only, and only while `.blurred.json` is absent; the first write retires
it, after the new file is in place.
- A planted directory is a 409, not a 500 (2/3 plus a third angle, executed by
the seat). The reader was hardened against it and the writer was not:
os.replace and unlink raised IsADirectoryError through the route. Now the
writer is judged by its reader: set_blurred re-reads after writing and raises
BlurUnwritable unless the set on disk is the set asked for. That one check
covers a directory at either name, a permission and a race.
- A lone surrogate is dropped on read (hulda, executed). `"\ud800"` is a valid
JSON string that no filename can produce, and the UTF-8 encode raised on it
at every later write.
- The writer respects the reader's size cap (2/3). Nothing capped the write,
and the reader reads an oversized file as EMPTY, which reveals everything.
- One predicate, check_rel, for the route and the CLI (2/3). The CLI's `*..*`
substring guard refused `a..b.png`, which the route accepts. It also refuses
an empty path now (regin, kimi), and every item is checked before any is
written.
- `booth blur` fails closed, with a message and exit 3, when its package is
missing (kimi), as `link` already does.
Declined, with reasons: the Item positional-constructor break (booth_items is
the only constructor, INV-1), the fdopen fd leak and the short read (not
constructible on a local filesystem, and the `.seen` shape), and
unreadable-reads-as-revealed (blur is cosmetic; the `.seen` posture).
blur_storage.toml: 20/20 proved. One row came back VACUOUS on its first run,
because `set() or X` is X, and was rewritten before counting.
286 lines
15 KiB
Markdown
286 lines
15 KiB
Markdown
# CLAUDE.md — the Booth
|
|
|
|
Mechanical conventions for this repo: **how the code here is written.**
|
|
Timeless by design — temporal state and dated decisions live in
|
|
`persistent-memory.md`, the v1 gate lives in `ROADMAP.md`, and the model the
|
|
code is converging on lives in `docs/design/information-architecture.md`.
|
|
|
|
House-wide rules (SemVer cadence, attribution, roadmap gate, the House Code
|
|
Discipline) come from `~/.claude/CLAUDE.md` and are **not** restated here.
|
|
|
|
## What this is
|
|
|
|
A standing FastAPI service on `:8090` that scans `~/booth-data` for
|
|
drop-folders and renders each as a review surface. Agents post work by making
|
|
a folder; the operator looks at it, judges it, and the judgment gets back to
|
|
the agent.
|
|
|
|
Read `docs/design/information-architecture.md` before any structural change.
|
|
It supersedes `README.md`, which still describes the accreted file-shuttle
|
|
model and is rewritten as the v1 units land.
|
|
|
|
## Persistent memory
|
|
|
|
`persistent-memory.md` at the repo root captures durable intent and supporting
|
|
evidence (goals, decisions, foot-gun warnings, in-flight state) across context
|
|
resets. Read it at session start; treat it as one input alongside this file and
|
|
the auto-memory system, not as the single source of truth.
|
|
|
|
It is a lean **index**: the dated log sections keep each over-threshold entry's
|
|
full body in `persistent-memory.d/<slug>.md`. Pull a detail file only when its
|
|
index line is relevant to your work — never bulk-read `persistent-memory.d/`.
|
|
|
|
When you commit, include any pending `persistent-memory.md` and
|
|
`persistent-memory.d/` updates in the same commit. Never leave them as a
|
|
floating uncommitted change while shipping other work — durable memory that
|
|
lags the code defeats its own purpose.
|
|
|
|
## The six invariants
|
|
|
|
These are the ones a casual change breaks silently. Each has a test.
|
|
|
|
### 1. The modules `scripts/booth` imports are stdlib-only, on purpose
|
|
|
|
`scripts/booth` — the CLI every fleet session uses — imports them directly:
|
|
|
|
```sh
|
|
BOOTH_SRC=… python3 -c 'import sys; sys.path.insert(0, …); from booth.marks import declare_pick'
|
|
```
|
|
|
|
It runs under the system `python3` with **no venv**. A single third-party
|
|
import in any of them breaks `booth ask` / `booth marks` / `booth answer` /
|
|
`booth unlink` / `booth blur` on every host, and the failure surfaces in an
|
|
agent's session, not in ours.
|
|
|
|
The set is `marks`, `asks`, `links`, `manifest`, `benches`, `blur` and
|
|
`__init__` (which runs before every one of them). **The list of record is
|
|
`test_stdlib_only`'s parametrize in `tests/test_marks.py`**, not this
|
|
paragraph. `items.py` and `app.py` are free to import what they like; those are
|
|
not. `test_stdlib_only` walks each module's AST imports and asserts it — the
|
|
CLI imports through a `python3 -c` heredoc that no AST extractor can see, so
|
|
that test is the only thing standing here. A new module the CLI imports goes on
|
|
that list in the same commit.
|
|
|
|
### 2. The filesystem is the state
|
|
|
|
No database. `ls ~/booth-data` tells you everything the service knows.
|
|
|
|
Per-booth operator state is a **dotfile inside the booth**: `.forever` (keep),
|
|
`.viewed` (last deliberate look — U4's "viewing is activity"), `.blurred.json`
|
|
(the per-item blur set, a JSON ARRAY — see below; the legacy `.blurred` is
|
|
read-only), `.seen` (R2: rels looked at full size, a JSON ARRAY), `.blurbooth` (the whole booth fogged — a MARKER like `.forever`, not
|
|
JSON, because a boolean has no rels to round-trip), `.marks.json` + `.marks.lock` (judgment), `.pins` (link-board pin
|
|
ids), `.uploaded` (upload-booth marker). `booth_items()` skips `name.startswith(".")`, so a new
|
|
dotfile costs nothing in item counts, galleries or zips. That skip is why the
|
|
dotfile is the right shape for new operator state — use it rather than
|
|
inventing a sidecar-per-item.
|
|
|
|
⚠ **A dotfile that holds rels is a JSON array, opened `O_NOFOLLOW |
|
|
O_NONBLOCK` with an `S_ISREG` check and a size cap.** A rel may carry a leading
|
|
space or a newline, and line-stripped storage does not round-trip it: `.blurred`
|
|
was one stripped rel per line, and blurring `" a.png"` blurred `a.png` instead.
|
|
`.seen` was written as JSON for exactly that reason (design-dev, R2), and the
|
|
blur set now matches it in `.blurred.json` (`booth/blur.py`). The open flags
|
|
mean a planted symlink is refused and a FIFO cannot hang the read, which is the
|
|
outage in `persistent-memory.d/2026-09-22-size-cap-opened-a-hang.md`. **Any new
|
|
dotfile inherits that shape.**
|
|
|
|
⚠ **A format change gets a NEW NAME, never a sniffed file.** The first cut of
|
|
the blur fix wrote JSON into `.blurred` and guessed the format from the bytes;
|
|
a legacy file whose one line is an item literally named `["a.png"]` parses as
|
|
JSON and blurs the neighbour, the bug being fixed (heid bug-hunt, 3 of 3). So
|
|
`.blurred.json` is JSON only, the legacy `.blurred` is lines only and read only
|
|
while `.blurred.json` is absent, and the first write retires it. Do not remove
|
|
that legacy read while a line-format file can still exist.
|
|
|
|
**A writer is judged by its reader.** `set_blurred` re-reads after writing and
|
|
raises `BlurUnwritable` unless the reader returns exactly the set asked for.
|
|
One postcondition covers a planted directory, a permission and a race without
|
|
a branch per way the disk can be wrong; the route answers it 409, never 500.
|
|
|
|
**A dotfile with two writers has ONE implementation of the writer, and one
|
|
predicate for its keys.** The blur set is written by the service and by `booth
|
|
blur`; both call `booth.blur.set_blurred`, and both ask `check_rel` what an
|
|
item path is. The CLI used to keep a grep/printf writer and a `*..*` guard of
|
|
its own, which refused `a..b.png` where the route accepted it.
|
|
|
|
### 3. One resolver for item facts
|
|
|
|
`booth.items.booth_items(booth)` is the only place a file is classified, a
|
|
caption resolved, a section derived or blur state read. **No route body
|
|
derives an item fact.** Every surface — gallery, zoom, doc view, index card,
|
|
zip manifest — reads the `Item` record.
|
|
|
|
Contract: `docs/contracts/u1_item_record.contract.md`, INV-1. Falsifiable and
|
|
tested: no call to `classify` / `doc_kind` / `read_blurred` / `render_doc`
|
|
survives inside `create_app`.
|
|
|
|
This exists because the zoom view once re-derived the item from scratch and, in
|
|
doing so, never resolved the caption — the operator's "zoomed images lose
|
|
their annotations" bug. It was not a rendering bug; it was three readers of one
|
|
truth.
|
|
|
|
**U3 extended this to the verbatim path.** A booth's own `index.html` now gets
|
|
its chrome from `/_booth/embed.js`, which *places* server-rendered fragments and
|
|
never builds one. The fragments come from the same `_ask_inline.html` macros the
|
|
gallery page uses, handed over `/b/<name>/embed.json`. A second renderer in
|
|
JavaScript would be the same bug in a new language.
|
|
|
|
### 4. Re-export, don't move-and-break
|
|
|
|
Names that moved from `app.py` to `items.py` (`classify`, `doc_kind`,
|
|
`render_doc`, `CAPTION_MAX`, the extension sets) stay importable from
|
|
`booth.app`. 20+ test sites import them by name from there. The re-export is
|
|
load-bearing and **asserted by a test**, not left to convention — a silent drop
|
|
would be found by a consumer, not by us.
|
|
|
|
### 5. Sidecar writes are atomic; text bodies stay raw
|
|
|
|
Anything a session may read while the browser writes it goes through temp file
|
|
+ `os.replace` (see `marks._write_raw`). A reader never sees a partial file, and
|
|
a crash mid-write cannot truncate a file into a shorter — and therefore quieter
|
|
— set of marks or a more revealing blur set.
|
|
|
|
`render_doc` returns **raw** text for the non-markdown case on purpose: the
|
|
template escapes it inside `<pre>`, and pre-escaping here double-encodes under
|
|
Jinja autoescape.
|
|
|
|
### 6. Every ordered collection has a stated, deterministic order
|
|
|
|
Operator directive, 2026-09-21. Not "usually stable" and not "whatever `rglob`
|
|
yields" — a rule you can write down in one line, producing the same sequence on
|
|
every render of the same state. Any defensible rule qualifies: byte order over a
|
|
path, creation time, an explicit number, an arbitrary-but-recorded sequence. No
|
|
rule at all does not.
|
|
|
|
The Booth's job is comparison, which makes this load-bearing rather than tidy.
|
|
The operator judges tile 47 of `pancake-v3-full` against tile 47 of
|
|
`pancake-v4-full`, and refers to artifacts positionally — "the third one", "the
|
|
one after the banded one". If the order moves between renders, or differs
|
|
between the gallery, the zoom ring, the zip and the `marks` read, a flag or a
|
|
note lands on the wrong artifact. It never shows up as a crash; it shows up as
|
|
the operator's judgment being quietly misfiled.
|
|
|
|
Current rules: items `sorted(rel)`; the zoom ring is that order filtered to
|
|
images; captions resolve over a sorted scan; marks `(created, id)`; legacy
|
|
import `(mtime, name)`; link rows pinned-then-newest; a verbatim report's embed
|
|
anchors in document order, its tail in payload order, its questions in
|
|
declaration order. `ROADMAP.md` carries the table and the two places still
|
|
undecided (U7 sections and compare pairing, U6 bench listing).
|
|
|
|
U3's rows are the first that bind **across a language boundary** — decided in
|
|
Python, honoured in JavaScript. A string assertion cannot see that, which is
|
|
why `tests/test_embed_browser.py` exists.
|
|
|
|
When you add an ordered surface, state its rule in the docstring. If you cannot
|
|
state it in one line, it does not have one.
|
|
|
|
## Multi-writer vs single-writer — don't inherit the wrong shape
|
|
|
|
`links.md` is a **multi-writer** append log: 17 agent handles post to it
|
|
concurrently, so it is an `O_APPEND` write with content-hash row identity and
|
|
an `fcntl` lock only on the rewrite path. Index-based removal would delete a
|
|
neighbour's row when another session appends mid-operation.
|
|
|
|
Marks, blur and keep state are **single-writer** — the operator, in one
|
|
browser — with many readers (sessions polling). That is a different problem and
|
|
must not inherit the append-log design. Per-booth file, atomic replace, lock
|
|
the read-modify-write.
|
|
|
|
Ask which one you have before choosing a storage shape.
|
|
|
|
## Non-goals, standing
|
|
|
|
- **No auth.** LAN/mesh-internal. Blur is cosmetic and the UI says so. Anything
|
|
that must not be seen by whoever can reach `:8090` must not be in a booth.
|
|
- **No database.** See invariant 2.
|
|
- **No upload API for booths.** A session makes a folder. That is the whole
|
|
API, and it is why every agent family can use this without a client.
|
|
- **Nothing deletes the operator's data on a timer** beyond the documented 24h
|
|
TTL. Liveness is *flagged*, not enforced.
|
|
|
|
## ⚠ The repo IS the deployment root
|
|
|
|
`booth.service` runs uvicorn with `WorkingDirectory=/home/lkraven/development/booth`.
|
|
There is no build step, no staging copy, and no separate deploy artifact: the
|
|
live service on `:8090` imports **these files**. Two consequences, and the second
|
|
one caused an outage.
|
|
|
|
1. **A Python edit does nothing until you restart.** Expected, and documented
|
|
below.
|
|
2. **A template edit used to take effect INSTANTLY.** Jinja's `FileSystemLoader`
|
|
re-reads a template from disk on every render. So the two halves of the
|
|
service had different staleness rules, and editing `booth.html` deployed it
|
|
immediately against Python that had never heard of the context it wanted.
|
|
|
|
On 2026-09-21 that put **19 of 25 live booths at 500** —
|
|
`UndefinedError: 'item_marks' is undefined` — with the Python from 22:03 and
|
|
the templates from 23:40. Neither version was broken; the service was running
|
|
both. The operator found it, not the suite, because no test can see a skew
|
|
that only exists between a process and the disk under it.
|
|
|
|
Fixed at the source: the template `Environment` is now built with
|
|
`auto_reload=False`, so templates are cached at startup exactly like the
|
|
Python. **One rule now — nothing takes effect until you restart.** The price
|
|
is that template work needs a restart to see, and that price is the point.
|
|
`test_templates_do_not_hot_reload_from_disk` holds the line.
|
|
|
|
3. **`booth/static/embed.js` is the third thing that would have hot-reloaded,
|
|
and it does not.** U3 gave the service a static asset living in the
|
|
deployment root; it is read ONCE in `create_app` and served from memory with
|
|
an ETag over its content, for exactly the reason above. Same rule, same test
|
|
shape (`test_embed_js_does_not_hot_reload_from_disk`). Anything else this
|
|
repo learns to serve from disk inherits the rule — read it at startup.
|
|
|
|
**So: after ANY edit here — Python, template or static asset — the live service
|
|
is stale until you restart it.** If you are touching this repo while the operator may be using
|
|
the service, either restart promptly or expect him to be looking at the old
|
|
version. Never leave the tree in a state where a restart would 500.
|
|
|
|
⚠ The `Environment` is hand-built now, which means `autoescape` is explicit
|
|
rather than inherited from the `Jinja2Templates` constructor. It is on
|
|
(`select_autoescape(["html", "xml"])`) and it is load-bearing: booth names, item
|
|
names and mark text are all agent- or operator-authored and land in HTML.
|
|
|
|
## Working in here
|
|
|
|
```sh
|
|
.venv/bin/python -m pytest -q # the suite; keep it green
|
|
curl -s localhost:8090/healthz # the live service (systemd --user)
|
|
systemctl --user restart booth.service # after a code change, to see it live
|
|
```
|
|
|
|
```sh
|
|
.venv/bin/python scripts/mutation_check.py # prove the falsifiers still falsify
|
|
```
|
|
|
|
**A green test is not evidence.** A test that has never seen its own defeating
|
|
change may pass under it too — forbidding nothing while reading as though it
|
|
forbids something. This repo shipped that three times before the tool existed
|
|
(twice in one session, once an hour after writing the entry about it). Tables
|
|
live in `tests/mutations/*.toml`, one per unit, committed so a unit's proofs are
|
|
an artifact rather than scrollback; adding a unit means adding a file, never
|
|
editing the script. `tests/test_mutation_check.py` holds the tool's own positive
|
|
and negative controls, because an instrument that only ever sees unknowns cannot
|
|
tell "nothing wrong" from "I am blind".
|
|
|
|
When you add a `*Falsifiable:*` line to a contract, add its row to the table and
|
|
run it. A falsifier nobody has run is a claim, not a test.
|
|
|
|
`tests/test_embed_browser.py` drives a real Chromium against a real uvicorn on
|
|
an ephemeral port — the only place U3's placement and `form=` binding can be
|
|
observed at all. Browsers are NOT downloaded per project; they live box-wide in
|
|
`/opt/ms-playwright`. The file **skips rather than fails** when playwright or a
|
|
usable browser is missing, so the suite stays green anywhere. If those tests
|
|
start skipping on this box, the pinned `playwright>=1.60,<1.63` in
|
|
`pyproject.toml` has drifted past the shared store — read the comment there
|
|
before raising the bound.
|
|
|
|
`booth.service` is a user unit installed to `~/.config/systemd/user/`. The repo
|
|
copy is the source; edits there need a `daemon-reload`.
|
|
|
|
Tests are the spec for behaviour the contracts don't cover yet — `test_booth.py`
|
|
carries the accreted service's behaviour and is the regression net for the v1
|
|
rewrite. Don't edit an existing assertion to make a change pass; if the
|
|
behaviour genuinely changes, the contract says so first.
|