docs: the conventions a fresh session can't infer, and the state it can't reconstruct

The repo came out of eshpfi one day ago with neither piece of its house
furniture, so everything non-obvious about it lived in docstrings inside a
998-line app.py — which is a bad place to keep an invariant that breaks
silently on every fleet host when someone violates it.

CLAUDE.md carries the five that do exactly that:

  * links.py and asks.py are stdlib-only because scripts/booth imports them
    under the system python3 with no venv, so one third-party import breaks
    `booth ask` everywhere and fails in an agent's session, not in ours
  * the filesystem is the state, and booth_items()'s dotfile skip is why a
    per-booth dotfile is the right shape for new operator state
  * booth_items() is the only thing that classifies a file or resolves a
    caption (U1's INV-1) — the zoom-loses-the-annotation bug was three
    readers of one truth, not a rendering bug
  * moved names stay importable from booth.app, asserted by a test
  * sidecar writes are atomic; render_doc returns raw text on purpose

Plus the distinction that decided this session's storage call: links.md is
an append log because 17 handles write it concurrently, and marks have one
writer. Different problem, different shape — ask which you have first.

persistent-memory.md carries what CLAUDE.md is structurally unable to: the
dated decisions, the `.forever` prediction and its re-measure date, and the
foot-gun log. Two entries are load-bearing for the next unit — the settled
mark storage shape with the reasoning that picked it, and a measured
correction to U7's premise: every booth that actually needs navigation is
flat, so subfolder sections are worth shipping but are not the nav fix.

No version bump — docs and memory, both on the SemVer skip list.
This commit is contained in:
vh
2026-09-21 22:58:00 -07:00
parent ce598b3cf6
commit 9272c9872e
2 changed files with 271 additions and 0 deletions
+138
View File
@@ -0,0 +1,138 @@
# CLAUDE.md — the Booth
Mechanical conventions for this repo: **how the code here is written.**
Timeless by design — temporal state and dated decisions live in
`persistent-memory.md`, the v1 gate lives in `ROADMAP.md`, and the model the
code is converging on lives in `docs/design/information-architecture.md`.
House-wide rules (SemVer cadence, attribution, roadmap gate, the House Code
Discipline) come from `~/.claude/CLAUDE.md` and are **not** restated here.
## What this is
A standing FastAPI service on `:8090` that scans `~/booth-data` for
drop-folders and renders each as a review surface. Agents post work by making
a folder; the operator looks at it, judges it, and the judgment gets back to
the agent.
Read `docs/design/information-architecture.md` before any structural change.
It supersedes `README.md`, which still describes the accreted file-shuttle
model and is rewritten as the v1 units land.
## Persistent memory
`persistent-memory.md` at the repo root captures durable intent and supporting
evidence (goals, decisions, foot-gun warnings, in-flight state) across context
resets. Read it at session start; treat it as one input alongside this file and
the auto-memory system, not as the single source of truth.
It is a lean **index**: the dated log sections keep each over-threshold entry's
full body in `persistent-memory.d/<slug>.md`. Pull a detail file only when its
index line is relevant to your work — never bulk-read `persistent-memory.d/`.
When you commit, include any pending `persistent-memory.md` and
`persistent-memory.d/` updates in the same commit. Never leave them as a
floating uncommitted change while shipping other work — durable memory that
lags the code defeats its own purpose.
## The five invariants
These are the ones a casual change breaks silently. Each has a test.
### 1. `links.py` and `asks.py` are stdlib-only, on purpose
`scripts/booth` — the CLI every fleet session uses — imports them directly:
```sh
BOOTH_SRC=… python3 -c 'import sys; sys.path.insert(0, …); from booth.asks import write_ask'
```
It runs under the system `python3` with **no venv**. A single third-party
import in either module breaks `booth ask` / `booth answer` / `booth unlink`
on every host, and the failure surfaces in an agent's session, not in ours.
`items.py` and `app.py` are free to import what they like. Those two are not.
### 2. The filesystem is the state
No database. `ls ~/booth-data` tells you everything the service knows.
Per-booth operator state is a **dotfile inside the booth**: `.forever` (keep),
`.blurred` (one rel per line), `.pins` (link-board pin ids), `.uploaded`
(upload-booth marker). `booth_items()` skips `name.startswith(".")`, so a new
dotfile costs nothing in item counts, galleries or zips. That skip is why the
dotfile is the right shape for new operator state — use it rather than
inventing a sidecar-per-item.
### 3. One resolver for item facts
`booth.items.booth_items(booth)` is the only place a file is classified, a
caption resolved, a section derived or blur state read. **No route body
derives an item fact.** Every surface — gallery, zoom, doc view, index card,
zip manifest — reads the `Item` record.
Contract: `docs/contracts/u1_item_record.contract.md`, INV-1. Falsifiable and
tested: no call to `classify` / `doc_kind` / `read_blurred` / `render_doc`
survives inside `create_app`.
This exists because the zoom view once re-derived the item from scratch and, in
doing so, never resolved the caption — the operator's "zoomed images lose
their annotations" bug. It was not a rendering bug; it was three readers of one
truth.
### 4. Re-export, don't move-and-break
Names that moved from `app.py` to `items.py` (`classify`, `doc_kind`,
`render_doc`, `CAPTION_MAX`, the extension sets) stay importable from
`booth.app`. 20+ test sites import them by name from there. The re-export is
load-bearing and **asserted by a test**, not left to convention — a silent drop
would be found by a consumer, not by us.
### 5. Sidecar writes are atomic; text bodies stay raw
Anything a session may read while the browser writes it goes through temp file
+ `os.replace` (see `asks.write_answer`). A reader never sees a partial file.
`render_doc` returns **raw** text for the non-markdown case on purpose: the
template escapes it inside `<pre>`, and pre-escaping here double-encodes under
Jinja autoescape.
## Multi-writer vs single-writer — don't inherit the wrong shape
`links.md` is a **multi-writer** append log: 17 agent handles post to it
concurrently, so it is an `O_APPEND` write with content-hash row identity and
an `fcntl` lock only on the rewrite path. Index-based removal would delete a
neighbour's row when another session appends mid-operation.
Marks, blur and keep state are **single-writer** — the operator, in one
browser — with many readers (sessions polling). That is a different problem and
must not inherit the append-log design. Per-booth file, atomic replace, lock
the read-modify-write.
Ask which one you have before choosing a storage shape.
## Non-goals, standing
- **No auth.** LAN/mesh-internal. Blur is cosmetic and the UI says so. Anything
that must not be seen by whoever can reach `:8090` must not be in a booth.
- **No database.** See invariant 2.
- **No upload API for booths.** A session makes a folder. That is the whole
API, and it is why every agent family can use this without a client.
- **Nothing deletes the operator's data on a timer** beyond the documented 24h
TTL. Liveness is *flagged*, not enforced.
## Working in here
```sh
.venv/bin/python -m pytest -q # the suite; keep it green
curl -s localhost:8090/healthz # the live service (systemd --user)
systemctl --user restart booth.service # after a code change, to see it live
```
`booth.service` is a user unit installed to `~/.config/systemd/user/`. The repo
copy is the source; edits there need a `daemon-reload`.
Tests are the spec for behaviour the contracts don't cover yet — `test_booth.py`
carries the accreted service's behaviour and is the regression net for the v1
rewrite. Don't edit an existing assertion to make a change pass; if the
behaviour genuinely changes, the contract says so first.
+133
View File
@@ -0,0 +1,133 @@
# Persistent memory — booth
_Last updated: 2026-09-21_
> **Always check for `/tmp/booth-dev-handoff.md`** — if it exists and its
> `Written:` stamp is under 8 hours old, read it (it carries the in-flight
> handoff from the previous session), then delete it. Older than 8 hours:
> stale — delete it unread.
## Repo purpose
The Booth is the fleet's **operator-review surface**: agents post work by
making a folder under `~/booth-data`, the operator looks at it and judges it in
the browser, and the judgment gets back to the agent that posted it. It was
built as a file-shuttle and is being converged, unit by unit, onto the review
loop it turned out to actually be.
## Current state / in-flight
_As of 2026-09-21:_
- **v1 is gated on seven units** in `ROADMAP.md`, ordered by dependency:
**U1 → U2 → {U3, U4, U5} → U7**, with **U6 independent** of all of them.
- **U1 (one item record) has landed** at `ce598b3` and is verified against its
own invariants, not just its commit message: INV-1 holds (no `classify` /
`doc_kind` / `read_blurred` / `render_doc` call survives in a route body),
the zoom and doc templates render the caption they now receive, the
re-exports are asserted by a test. 192 tests green, `0.1.15`.
- **U2 (marks) is next**, and its storage shape is settled (see the
2026-09-21 decision below). Not yet started: no contract written, no
blast-radius pass run.
- **Next concrete step:** graphify + grep blast-radius pass over `booth.asks`'s
surface and every `ask` / `answer` call-site (including `scripts/booth`),
then author `docs/contracts/u2_marks.contract.md`. Full House Code Discipline
— U2 is a new primitive with a CLI surface and 17 consuming handles, nowhere
near surgical. The **seam review** is the load-bearing gate: U2's whole risk
is that `pick` claims to preserve ask semantics and then drifts from them.
- **Open, operator's call:** whether U6 (benches) runs in parallel with U2 or
strictly after it. Nothing blocks on the answer; U6 touches different storage
and a different surface, so it cannot be broken by U2.
- Live service is `active` on `:8090` (systemd `--user`), 25 booths.
## Recent decisions
- `[2026-09-21]` **Marks are stored as one `.marks.json` per booth**, atomic
temp-file + `os.replace`, `fcntl` lock on the read-modify-write — operator
decision, this session. Two alternatives were weighed and lost: a sidecar
per item (`<rel>.marks.json`) and extending the existing `<stem>.ask.json`
shape. Rationale, and the reason it is not `links.md`-shaped: **(a)** U4
makes *"does this booth owe an answer?"* a hot question — the sweep asks it
per booth per tick and the index asks it per card per page load, so per-item
sidecars turn it into a full walk of all 25 booths, one of which holds 270
files; **(b)** `links.md` is an `O_APPEND` content-hash log because **17
agent handles write it concurrently**, whereas marks have exactly one writer
(the operator, in one browser) and many readers — a different problem that
must not inherit the append-log design; **(c)** `.blurred` / `.pins` /
`.forever` already establish the per-booth dotfile as the house shape for
operator state, and `booth_items()`'s dotfile skip means it costs nothing in
counts, galleries or zips. Accepted cost: a corrupt `.marks.json` loses that
booth's marks rather than one item's. Implementation deferred to U2 —
tracked at `ROADMAP.md` U2 and by this entry.
- `[2026-09-21]` **U7's section premise is half wrong, and it is the half that
matters** — found by re-measuring `~/booth-data` rather than trusting the IA
doc. The IA says sections come from subfolders that already exist on disk;
true, but **every booth that actually needs navigation is flat**:
`pancake-v3-full` (270 items, 0 subfolders), `pancake-v4-full` (270, 0),
`sindra20-engines` (98 items + 99 caption sidecars, 0), `sindra-finalists`
(86 + 87, 0). Subfolders exist on exactly two booths — `pewpew-ui-brief` (7,
nested to `_ds/powerpellet-design-system-<uuid>/preview`) and `dfa-concepts`
(1) — and **both are reports**, the job where grid navigation matters least.
So sections stay worth shipping and `Item.section` stays right, but they are
**not** "most of the navigation fix": the rail, the filters and grid keyboard
are all of it. Worth noting for whoever writes U7: `sindra20-engines` encodes
its structure in the **filename prefix** (`b2-s1-<subject>-<seed>`), which is
where a grouping heuristic would actually pay. The IA doc's claim about what
sections buy needs a line struck — not yet edited.
- `[2026-09-21]` **`sindra-finalists` is U2's `flag` motivation caught in the
act** — 86 items, every one captioned, and the booth's entire name is "the
ones the operator picked." That loop currently runs through chat, which is
the defect `flag` closes. Evidence, not argument.
- `[2026-09-21]` **The information architecture and the v1 gate landed**
(`726822b`): `docs/design/information-architecture.md` names the single
defect — *one lifetime (24h from last touch) and one shape (a folder),
serving five jobs with different lifetimes and different shapes* — and
`ROADMAP.md` gates v1 on seven units, each closing a **measured** defect
rather than a wish. Both were written after a measurement pass over the live
service, and the measurements are the load-bearing part.
- `[2026-09-21]` **The `.forever` diagnosis is a stated, falsifiable
prediction.** U4 (derived lifetime) predicts the kept-rate falls to the
genuinely-durable booths. Re-measured today: **14 of 25 booths kept (56%)**,
against the 54% the IA doc recorded. **Re-count a fortnight after U4 lands.**
If it does not move, the diagnosis was wrong and the boolean was doing
something else. Tracked in the IA doc's Booth section and by this entry.
- `[2026-09-21]` **Extracted from `eshpfi` into its own repo.** The accreted
service came over whole, tests included, so `tests/test_booth.py` (1581 lines)
is the regression net the v1 rewrite is checked against.
## Tried and abandoned
- `[2026-09-21]` **Five separate mechanisms to get one question next to one
artifact** — `.forever`, the link board, `inline.py`'s placeholder DSL,
`wrap_verbatim_html`'s six regexes, and the floating amber asks chip plus
`/b/<n>/asks`. Every one is a *correct local fix* to the same global
mismatch, which is exactly why they accumulated without anyone making a bad
call. **The foot-gun is the sixth one:** the next "just add a small thing for
this case" reads as reasonable and is the pattern. The git log carries the
signature — every feature ships, then takes 2–5 patches for cases the single
shape did not anticipate. Check the ROADMAP gate before adding a mechanism.
- `[2026-09-21]` **Regex-injecting chrome into arbitrary author HTML**
(`wrap_verbatim_html` + `_HEAD_CLOSE_RE`, `_HTML_OPEN_RE`, `_DOCTYPE_RE`,
`_BODY_CLOSE_RE`, `_HTML_CLOSE_RE`, `_ICON_RE`, and the doctype/charset
ordering constraints they thread). It works today and is **still live** —
but it is the single most fragile thing in the service and it is load-bearing
for the operator's most important workflow. Slated for deletion at U3 in
favour of a declared seam (`/_booth/embed.js`, mounted through a real DOM
API), which costs an author one line and removes the whole class. Do not
extend the regex set in the meantime; if a verbatim page breaks, that is an
argument for U3, not for a seventh pattern.
- `[2026-09-21]` **A boolean escape hatch as the lifetime mechanism.**
`.forever` was added because a 24h TTL genuinely did not fit some booths —
and then 56% of live booths ended up on it, which means it is not "ephemeral
with an exception", it is two lifetimes wearing one lifetime's clothes, with
the operator doing the sorting by hand. Replaced at U4 by lifetime derived
from state (an open mark pins; viewing is activity; `keep` survives as an
explicit reasoned pin rather than the only way to say "not yet").
- `[2026-09-21]` **Letting the link board absorb the announce job.** `booth
link` is an `O_APPEND` write with no identity and no stated rule, so
re-announcing a bench appends a row instead of updating one, and a booth URL
rots the moment its booth is swept — **145 of 211 rows (69%) pointed at
nothing**, and 22 were the same target re-posted (talk 5×, peedlar 4×). The
rot is **structural, not drift**. The lesson that cost the most: enforcing
the link rule without first giving the announce job a home (`.booth.json`
provenance on the index, U5) just makes it homeless.