esh-pfi-infrastructure/stacks/homepage/README.md

# homepage — the fleet dashboard

`ghcr.io/gethomepage/homepage` on **esh-docker-vm** (`10.0.50.45:5100`), behind
Traefik as `eshhome` / `eshhome.esteban.net`. Config is plain YAML — no
database, no UI-written state — which is why it belongs in this repo like any
other stack.

**Brought under version control 2026-08-17.** Before that it was edited in
place on the host, and had accumulated six hand-rolled `services.yaml.bak-*`
files as its only version history. Those were removed; git is the history now.
Edit here, then `scripts/deploy-stack.sh esh-docker-vm homepage`.

## How services get on the dashboard

Two paths, and mixing them is the classic failure:

1. **Docker label auto-discovery** — the default. A stack carries
   `homepage.group=` / `homepage.name=` / `homepage.icon=` / `homepage.description=`
   / `homepage.href=` labels and appears automatically. `conf/docker.yaml`
   wires **five** engines over plaintext `:2375`: esh-docker-vm, ana-docker,
   nh3-docker, ana-ml2, irv-ml1.
2. **Manual entries in `conf/services.yaml`** — for anything that is not a
   labelled container on one of those five hosts: hardware, BMCs, hypervisors,
   printers, and user-level systemd services (The Booth, Voice Design Studio).

⚠ **Never list a labelled container manually — it renders twice.** The
comments in `services.yaml` mark which groups are auto-populated (AI ×7,
Service Networking, Monitoring). Respect them.

⚠ **Labels only apply on container recreate.** Changing `homepage.group=` on a
compose file and running `restart` does nothing; the container must be
recreated.

## Layout

`conf/settings.yaml` owns tabs, group order, and column counts — `services.yaml`
owns *what exists*, `settings.yaml` owns *where it sits*. Four tabs: Main, AI,
Infrastructure, Toolchain. A group listed in `layout:` with no members simply
renders empty, so a group can look "dead" when its provider host is unreachable
rather than when the group is wrong.

## Foot-guns found in the 2026-08-17 audit

- **`HOMEPAGE_ALLOWED_HOSTS` matches host *and port*.** The entry `10.0.50.45`
  does **not** cover `http://10.0.50.45:5100/` — that combination was being
  rejected with `Host validation failed` in the container log while the Traefik
  hostnames worked fine. **Fixed 2026-08-17**; every `host:port` the dashboard
  is reached by needs its own entry. See `.env.example`.
- **The `.env` was mode 644** with the Plex and Jellyfin API keys in it. Now
  `600`. It is root-owned, so it needs the `infra-ops` identity to edit —
  `lkraven` on this host has password-sudo only.
- **Homepage renders client-side.** Grepping the served HTML to verify a config
  change gives false readings — first a stale prerender, then an empty page.
  `GET /api/services` is the honest check; it returns the resolved group/service
  tree. Config edits also need a **container recreate**, not a restart: a
  restart keeps the writable layer and its cached render.
- **`:2375` is plaintext and unauthenticated** on all five engines. Fine on a
  trusted LAN, and unchanged by this commit, but it is real exposure: anything
  that can reach those ports has full Docker control of that host. `docker.yaml`
  carries a commented TLS example for when that stops being acceptable.
- **`ping:` cards can only be judged from esh-docker-vm.** Probing them from
  another box gives false FAILs — ICMP is filtered across some site links. All
  34 entries were verified reachable *from the dashboard host* on 2026-08-17.

## 2026-08-18 cleanup

Three fixes, all in this stack's config except where noted:

- **UltraSeedbox appeared on all four tabs.** The bookmark group had no entry
  in `settings.yaml`'s `layout:` block at all, and Homepage's documented
  behaviour is that *"if a group has no tab specified (and tabs are set on
  other groups), services and bookmarks will be shown on all tabs."* It now
  carries `tab: Main` plus `style: row` / `columns: 4`, which also turns eight
  full-width bars into a compact grid. **Any group added without a `tab:` will
  do this again** — the rule is now written at the top of the layout block.
- **Uptime Kuma rendered twice.** It was listed manually under Monitoring in
  `services.yaml` *and* labelled `homepage.group=Apps` on its container. The
  manual block is gone; the container's label now says `Monitoring` and
  carries `homepage.siteMonitor`. The container was adopted into this repo at
  `stacks/uptimekuma/` in the same commit — it had been running unmanaged.
- **Column counts were fiction.** Several groups declared more columns than
  they had members, so the last row of each was mostly dead space (Notes: 1
  card in a 4-wide row). Columns now track member counts; see the rule in
  `settings.yaml`. Check with `GET /api/services`, which prints live per-group
  counts.

## The tab bar goes missing after a recreate, then comes back on its own

**Status: self-healing, cause not pinned. Do not chase it.** After a recreate
the client render comes up with no tab bar, no wallpaper and no i18n (the
search box shows the raw key `search.search`), and groups fall back to
side-by-side columns. It restores itself with no intervention.

**Timing, measured rather than assumed:** five minutes is NOT enough — a fresh
container was still tab-less at 4m30s, twice. It was observed healthy again
after a gap of roughly an hour. So the window is somewhere between those, and
`docker ps` reporting `healthy` says nothing about it: the container is
serving, the page is just wrong.

**Practical rule: recreate, then walk away and re-check later.** Do not start
editing config to fix it — that is how an hour got spent ruling out four
causes that were never the cause:

- **Not the config.** Restoring `settings.yaml` *and* `services.yaml` to their
  committed versions reproduces it, as does the pre-adoption backup config in
  `/opt/docker-bu/conf/homepage/`.
- **Not the v2.0.0 release.** A throwaway container on `v1.13.2` (the last v1)
  shows identical symptoms, and the image never changed anyway — working and
  broken containers both report `v2.0.0` / rev `17456f2`.
- **Not `PUID`/`PGID`**, and not Docker discovery — tested with both, and with
  the socket unmounted entirely.
- **Not a server-side failure.** The server-rendered HTML still contains the
  tab markup, the background URL and `useEqualHeights` throughout, and
  `GET /api/validate` returns `[]`. Whatever is lost, is lost client-side, and
  with no page error, no failed chunk and no non-200.

If someone does want to pin it: the shape (recovers on a long timescale, no
client exception) points at a cache or revalidation window rather than a
config fault. Start there, not in this directory.

**What it is not** — both obvious suspects were tested and cleared:

- **Not the config changes above.** Restoring `settings.yaml` *and*
  `services.yaml` to their committed pre-cleanup versions reproduces the
  breakage exactly. So does the pre-adoption backup config in
  `/opt/docker-bu/conf/homepage/`.
- **Not the v2.0.0 release.** A throwaway container on `v1.13.2` (the last
  v1, 2026-06-09) against the same config shows identical symptoms. The image
  never changed anyway: the working container and the broken one both report
  `v2.0.0` / rev `17456f2`, and only one homepage image exists on the host.

**What is known.** The *server-rendered* HTML still contains the tab markup,
the wallhaven background URL and `useEqualHeights` — so `settings.yaml` is
being read and delivered correctly. The loss happens client-side, with **no
page error, no failed chunk and no non-200** beyond two unrelated Uptime Kuma
widget 403s. `GET /api/validate` returns `[]`. A fresh container never
renders tabs here regardless of image version, config version, `PUID`/`PGID`,
or whether Docker discovery is mounted at all.

Every throwaway container in that list was judged within ~30s of starting, so
they were all inside the same window — and that consistency read as a
reproduction when it was really the same measurement mistake five times over.
This sits alongside the existing warning that `docker ps` health and a correct
render are different questions.

Before/after evidence: `~/booth-data/homepage-cleanup/` on nh3-dev →
`http://10.100.10.50:8090/b/homepage-cleanup/` (24h TTL).

## Theme — Australis Skyfall

The dashboard is themed with **Australis Skyfall**, the operator-supplied
design system (handoff bundle, 2026-08-18). Skyfall is a dual-theme web system
built on the OKLCH rebuild of the Australis terminal palette: one lightness law
across every chromatic family, all hues cooler than neutral, a Sea neutral ramp
that drifts from ice-blue to ocean green as it brightens, and a "calm depth"
language of hairline + two-layer shadow on every elevated surface.

### `theme/` is the source; `conf/custom.css` is generated

```
theme/colors.css      vendored VERBATIM from the bundle
theme/layout.css      ditto — radii, the shadow/glow/rail system, motion
theme/typography.css  ditto — families, scale, weights, tracking
theme/fonts/          Supreme 400/500/700 woff2 (Fontshare ITF license)
theme/skyfall.css.in  the Homepage bindings — the only part that is ours
theme/build.py        inlines the fonts + tokens -> conf/custom.css
```

**Do not hand-edit `conf/custom.css`.** Change `skyfall.css.in`, run
`python3 stacks/homepage/theme/build.py`, then deploy.

### Iterating on the theme — do NOT recreate the container

`custom.css` is fetched per request from `/api/config/custom.css`, so a CSS
change needs a **browser reload**, nothing more. It does **not** need a
container recreate, and it therefore does not owe the multi-minute layout
warm-up described below — that penalty only applies to the full tab/layout
render. Conflating the two costs an operator-visible 10 minutes per attempt,
which is exactly how the first version of this theme shipped overcooked
(operator, 2026-08-19).

Faster still, and the right way to judge a colour change before it touches the
live dashboard: **inject the candidate CSS into the running page and
screenshot it.** No deploy, no restart, seconds per iteration.

```js
// playwright, against the live dashboard
await p.goto('http://10.0.50.45:5100/', { waitUntil: 'networkidle' });
await p.addStyleTag({ content: candidateCss });
await p.screenshot({ path: 'preview.png', fullPage: true });
```

Only build + deploy once the render looks right.

The build exists for one reason: Homepage serves exactly two files out of its
config directory, `custom.css` and `custom.js`. There is no static route for
anything beside them, so a `@font-face` pointing at a vendored `.woff2` would
404 — the face has to arrive inside the stylesheet as a data: URI. That is also
why the background image is mounted at `/app/public/images` instead (see
`compose.yaml`), which *is* a real static route.

Only Supreme (body/UI) is embedded. A link dashboard has no display type, so
Bespoke Sans buys nothing; Victor Mono ships as 2.4 MB TTF statics per cut,
which is 30x the whole stylesheet for the handful of latency figures on the
cards. Both are named in the font stacks so they resolve if vendored later.

### The background is generated, not stock

`images/skyfall-aurora.webp` came out of **Arbo** (irv-ml1:8201) via its
`t2i-ui-background` workflow — "abstract full-bleed backgrounds, no subject".
Job `13f0891f4e42`, seed 26, flux2-klein-9b, 2048x1152, 1.6 MB PNG re-encoded
to a 22 KB WebP (smooth gradients compress absurdly well).

Regenerate with a different seed, or reword, via:

```bash
curl -s -X POST http://10.100.79.3:8201/workflows/t2i-ui-background/run \
  -H 'content-type: application/json' \
  -d '{"prompt":{"kind":"raw","text":"...","negative":"..."},
       "width":2048,"height":1152,"seed":26}'
# poll GET /jobs/<id>, then GET /artifacts/<id>/0
```

Note the prompt field is a discriminated union, not a string — a bare string
gets a 422. `{"kind":"raw","text":...}` is the shape.

### Two deviations from the system, on the record

1. **Skyfall forbids this background.** Its rule is "flat semantic surfaces; no
   photography, no textures", with exactly one permitted decorative motif — a
   subtle aurora gradient, on hero and empty-state areas only, *"never behind
   body text blocks"*. A dashboard is a body-text block. The background is here
   on the operator's explicit instruction to generate one with Arbo, and it is
   mitigated rather than excused: abstract with no subject, strictly cool
   temperature per the imagery rule, and held at `opacity: 14` so card contrast
   never pays for it. **That opacity is load-bearing, not decoration** — the
   render is brighter than the "mostly empty negative space" brief, and turning
   it up is what would make the cards fight the ribbon. If it ever reads busy,
   the system-faithful answer is to drop the image and let `--surface-sunken`
   be the canvas.
2. **Service icons stay full-colour vendor logos.** Skyfall wants one cool,
   technical icon family. Desaturating every logo from CSS would just make them
   illegible, so this one is simply not fixed.

## Open question

`ESH-FileBot` (`10.0.50.70`) is still described as "role TBC" — it responds to
ping, but nobody has written down what it does. Worth resolving or removing.