vh
f692b7ec7a
Adds a small × on each item that hides it from the page. State is
server-side at /output/hidden.json so the same hidden set follows
the user across devices (home, ipad, laptop, work). A "Hidden (N)"
tray at the bottom shows what's hidden on the current page with a
restore button per row; older hidden ids that aren't on this page
sit silently and continue to filter future editions that include
the same article.
Architecture change: news-digest-web swaps from nginx:alpine to a
FastAPI app on uvicorn, built from the same Dockerfile as the
worker. Same image, different command (`uvicorn web:app` overrides
the worker's cron entrypoint via compose). Drops one image dependency,
adds /api/{hidden,hide,restore}.
Item ids are stable 12-char sha1 prefixes (`reddit:<post_id>` /
`miniflux:<entry_id>`) computed in digest.py at render time and
emitted as `data-id` on each .item. The frontend reads /api/hidden
once on load, applies `is-hidden` to matching items, and POSTs
hide/restore on user interaction (optimistic, with rollback on
network error).
Storage: single JSON array at /output/hidden.json, atomic writes
via tempfile + rename, threading.Lock around the read-modify-write
inside the single uvicorn worker. No auth — the digest itself is
unauthenticated on LAN; same trust boundary applies.
Playbook also drops the DOCKER_BUILDKIT=0 fallback now that
ana-docker is on docker-ce 29, and adds three verify steps
(/api/hidden returns a JSON array, app.js is reachable, full
hide/restore round-trip with a synthetic id).
152 lines
5.6 KiB
Markdown
152 lines
5.6 KiB
Markdown
# news-digest
|
||
|
||
Twice-daily LLM-curated briefing across Reddit (via JSON API) and
|
||
Miniflux's Tech aggregators category. Output is a single static HTML
|
||
page styled in the Australis design system with editorial-serif
|
||
headlines (Fraunces) — operations-center chrome wrapping news content.
|
||
|
||
## Why this stack exists
|
||
|
||
After a few subreddits + HN + Lobste.rs, the Miniflux inbox gets
|
||
noisy. This stack:
|
||
|
||
1. Pulls **top-of-day** posts per subreddit from Reddit's public JSON
|
||
API (gives us scores + upvote ratios — RSS doesn't).
|
||
2. Filters by `score >= 50` and `upvote_ratio >= 0.85` (configurable)
|
||
to drop flame-bait and low-effort posts.
|
||
3. Pulls non-Reddit recent items from Miniflux's Tech aggregators
|
||
category (HN, Lobste.rs).
|
||
4. Sends each source through `qwen3.5-35-a3b` on llama-swap (one
|
||
batched call per source — efficient) for a one-sentence tl;dr +
|
||
single-word tag (news / tutorial / release / discussion / question
|
||
/ showcase / drama / meme / other).
|
||
5. Renders an HTML page styled with Australis tokens + Fraunces
|
||
serif headlines.
|
||
6. The page is served by a tiny FastAPI app on uvicorn that also
|
||
exposes `/api/{hidden,hide,restore}` for the per-item × button
|
||
(state in `/output/hidden.json`, shared across every device the
|
||
user opens the digest from).
|
||
|
||
Two editions per day: 8am and 8pm local. Plus per-edition archives
|
||
at `/edition-YYYY-MM-DD-{am,pm}.html`.
|
||
|
||
## Architecture
|
||
|
||
Two containers built from the same Dockerfile, both on `traefik-net`,
|
||
sharing a bind-mounted output dir:
|
||
|
||
```
|
||
news-digest-worker (default ENTRYPOINT — busybox crond)
|
||
├── busybox crond fires at 0 8,20 * * *
|
||
├── digest.py:
|
||
│ ├── miniflux /v1/feeds → discover subreddits
|
||
│ ├── reddit JSON top/.json?t=day per sub (gentle 1.5s sleep)
|
||
│ ├── miniflux /v1/entries → tech aggregators
|
||
│ ├── llama-swap /v1/chat/completions → batched per source
|
||
│ └── jinja2 render → /output/index.html (atomic .tmp + rename)
|
||
│ → /output/edition-2026-04-26-pm.html
|
||
└── style.css / favicon.svg / app.js staged in /output at deploy
|
||
|
||
news-digest-web (entrypoint overridden → uvicorn web:app)
|
||
├── / → serve /output as static (index.html as default)
|
||
├── /api/hidden GET → JSON array of hidden item ids
|
||
├── /api/hide POST → {id} → adds id to hidden.json
|
||
├── /api/restore POST → {id} → removes id from hidden.json
|
||
├── /output/hidden.json — durable state (atomic writes + threading lock)
|
||
└── homepage card via container labels (group=News)
|
||
```
|
||
|
||
Hidden state is server-side and global per-user (single-user setup):
|
||
hide an article once and it stays hidden in any future edition that
|
||
includes the same article. The "Hidden (N)" tray at the bottom of
|
||
each page shows items hidden FROM THE CURRENT PAGE; older hidden ids
|
||
that aren't present on this page just sit silently in `hidden.json`
|
||
and continue to filter future editions.
|
||
|
||
## Deploy
|
||
|
||
```bash
|
||
scripts/elway ana-docker --playbook playbooks/deploy-news-digest.yaml
|
||
```
|
||
|
||
After first deploy, **fill in MINIFLUX_PASSWORD on the host**:
|
||
|
||
```bash
|
||
ssh ana-docker '
|
||
cd /opt/docker/compose/news-digest
|
||
sed -i "s|^MINIFLUX_PASSWORD=.*|MINIFLUX_PASSWORD=<your-miniflux-admin-password>|" .env
|
||
docker compose up -d
|
||
'
|
||
```
|
||
|
||
The container runs the first digest immediately if `/output/index.html`
|
||
doesn't exist, so the page populates within a minute or two of bringing
|
||
the stack up with real credentials.
|
||
|
||
Visit <http://10.250.50.70:8181> to read.
|
||
|
||
## Tuning the noise floor
|
||
|
||
Defaults in `.env.example`:
|
||
|
||
| Knob | Default | Effect |
|
||
|---|---|---|
|
||
| `DIGEST_REDDIT_HOURS` | 12 | Look-back window (matches twice-daily cadence) |
|
||
| `DIGEST_MIN_SCORE` | 50 | Reddit minimum upvotes to consider |
|
||
| `DIGEST_MIN_RATIO` | 0.85 | Reddit minimum upvote ratio (skips flamebait) |
|
||
| `DIGEST_MAX_PER_SUB` | 8 | Cap per subreddit, post-filter |
|
||
| `DIGEST_MINIFLUX_HOURS` | 12 | Look-back window for HN/Lobste.rs |
|
||
| `DIGEST_MINIFLUX_MAX` | 8 | Cap per non-Reddit feed |
|
||
|
||
For a busier day, lower `DIGEST_MIN_SCORE`. For a quieter morning
|
||
edition, raise it. Edit `.env`, no rebuild needed — the worker reads
|
||
env on each cron fire.
|
||
|
||
## Adding more subreddits
|
||
|
||
The digest picks up subreddit feeds from Miniflux automatically — any
|
||
feed whose URL starts with `https://www.reddit.com/r/<sub>/` gets
|
||
queried. To add a sub, just subscribe in Miniflux (UI or API). The
|
||
next digest run includes it.
|
||
|
||
## Updating the LLM model
|
||
|
||
```bash
|
||
ssh ana-docker '
|
||
cd /opt/docker/compose/news-digest
|
||
sed -i "s|^LLAMA_SWAP_MODEL=.*|LLAMA_SWAP_MODEL=<new-model>|" .env
|
||
docker compose up -d
|
||
'
|
||
```
|
||
|
||
The model must be loaded in llama-swap's `config.yaml`. Check
|
||
`http://10.250.50.54:9292/v1/models` for what's available. Models
|
||
with tool/JSON-mode support give better summarization quality;
|
||
`qwen3.5-35-a3b` is the current default.
|
||
|
||
## Forcing a fresh digest now
|
||
|
||
```bash
|
||
ssh ana-docker 'docker exec news-digest-worker python3 /app/digest.py'
|
||
```
|
||
|
||
Runs the full pipeline once, ignoring cron. Useful after changing
|
||
filtering knobs or adding feeds.
|
||
|
||
## Logs
|
||
|
||
```bash
|
||
ssh ana-docker 'docker logs --tail 100 news-digest-worker'
|
||
```
|
||
|
||
Worker logs each phase (subreddit discovery / fetching / summarizing /
|
||
rendering) with timestamps. Per-source LLM filter results show how
|
||
many items were kept vs skipped.
|
||
|
||
## License + attribution
|
||
|
||
Reddit content surfaced here is owned by its authors and Reddit. The
|
||
digest is a derived index pointing at original sources — every item
|
||
links back to the Reddit thread (and to the external link if the
|
||
post linked out). Same for HN / Lobste.rs.
|