fix(booth): templates were hot-reloading into a live service running older Python

19 of 25 live booths returned 500 with `UndefinedError: 'item_marks' is
undefined`. Neither the old code nor the new code was broken — the service was
running both at once.

`booth.service` sets WorkingDirectory to this repo, so the repo IS the
deployment root: no build step, no staging copy, the live service imports these
files. Python is read once when the process starts. Jinja's FileSystemLoader
re-reads a template on EVERY render. So the two halves of the service had
different staleness rules, and editing booth.html deployed it instantly against
Python from 22:03 that had never heard of the context the new markup wanted.

The failure mode is worth naming precisely, because it is invisible to the
suite by construction: the skew exists between a running process and the disk
underneath it, so every test can pass against a tree that is simultaneously
serving 500s. No amount of green catches this. The operator found it.

Fixed at the source rather than with a reminder to restart. The template
Environment is built here with auto_reload=False, so templates are cached at
startup exactly like the Python, and there is ONE rule: nothing takes effect
until you restart. The price is that template work needs a restart to see —
that price is the entire point, and it is cheaper than a page of 500s while
someone is reviewing.

Building the Environment by hand means autoescape no longer comes from the
Jinja2Templates constructor, so it is explicit and load-bearing: booth names,
item names and mark text are all agent- or operator-authored strings that land
in HTML. Verified escaped, not merely configured.

Two tests hold the line — one on the snapshot property, one on the `dur` filter
that is no longer incidental to the constructor. The environment is reachable at
app.state.templates because a promise about the deployed service needs an
assertion, and an assertion needs the env the app actually renders with.

Also records the foot-gun in CLAUDE.md and persistent-memory: anyone editing
this repo while the operator may be using the service is editing production.

244 tests. No version bump — the release tier for U2 is still the operator's
call, and this rides with it.
This commit is contained in:
vh
2026-09-21 23:44:37 -07:00
parent c7f9437a64
commit bb1e3cfcd7
4 changed files with 141 additions and 15 deletions
+43
View File
@@ -1579,3 +1579,46 @@ def test_kept_lane_offers_a_direct_wipe_beside_release(client):
# ...and it actually wipes.
c.post("/b/bo/delete", follow_redirects=False)
assert not d.exists()
# ---- the running service is a coherent snapshot ------------------------------
#
# Outage, 2026-09-21: 19 of 25 live booths returned 500 with
# `UndefinedError: 'item_marks' is undefined`. Nothing was wrong with either the
# old code or the new code — the service was running BOTH. `booth.service` sets
# WorkingDirectory to the repo, so the repo IS the deployment root, and Jinja's
# FileSystemLoader re-reads a template from disk on every render while the Python
# stays as it was at process start. Editing a template therefore deployed it
# INSTANTLY, against Python that had never heard of the context it wanted.
#
# The fix is not "remember to restart" — it is to make the two halves fail the
# same way, so the running process is always the code as of its start time.
def test_templates_do_not_hot_reload_from_disk(tmp_path):
"""Templates must be cached at startup, exactly like the Python is.
With auto_reload on, the two halves of the service have DIFFERENT staleness
rules — Python needs a restart, templates do not — and any edit to a
template puts a live service into a state that was never tested: new markup
against old context. One consistent rule ("nothing takes effect until you
restart") turns a silent 500 storm into a change that simply has not
happened yet.
"""
from booth.app import create_app
app = create_app(tmp_path, ttl_hours=24, start_sweeper=False)
env = app.state.templates.env
assert env.auto_reload is False, (
"templates hot-reload from disk while the Python does not — "
"editing one deploys it to the live service instantly"
)
def test_the_dur_filter_survives_the_custom_environment(tmp_path):
"""The env is hand-built now, so the filter registration is no longer
incidental to the constructor."""
from booth.app import create_app
app = create_app(tmp_path, ttl_hours=24, start_sweeper=False)
assert app.state.templates.env.filters["dur"](3600) == "1h"