stacks/news-digest: twice-daily LLM-curated briefing on ana-docker
The Miniflux inbox got noisy after a few subreddits + HN + Lobste.rs.
This stack distills a single static page twice a day — at 0800 and
2000 local — that surfaces only what cleared score + ratio filters,
each item tldr'd by qwen3.5-35-a3b on llama-swap.
Pipeline (digest.py, ~330 lines):
1. Discover subreddits from Miniflux feeds (any reddit.com/r/<sub>/
URL — single source of truth, no duplicated config).
2. Reddit JSON top-of-day per sub. Filter: score >= 50,
upvote_ratio >= 0.85. Cap 8 items per sub.
3. Miniflux /v1/entries for the 'Tech aggregators' category
(HN, Lobste.rs) — last 12 hours.
4. Batched per-source summarization via llama-swap
/v1/chat/completions. Each post gets a one-sentence tldr +
one-word tag (news / tutorial / release / discussion /
question / showcase / drama / meme).
5. Render Jinja2 template. Atomic write to /output/index.html
(.tmp + rename) so partial pages never get served. Per-edition
archive at /output/edition-YYYY-MM-DD-{am,pm}.html.
Two containers:
news-digest-worker python:3.12-alpine + busybox crond
news-digest-web nginx:alpine, port 8181, homepage card via
docker labels (group=News, fits next to Miniflux)
Both bind-mount /opt/docker/data/news-digest as /output and
/usr/share/nginx/html respectively.
Aesthetic — operations-center chrome (Australis cool-mono palette,
JetBrains Mono UPPERCASE eyebrows, mdi-glyph anchor) wrapping
editorial-serif news content (Fraunces variable serif w/ optical
sizes). Two type families that wouldn't normally meet, intentionally
combined: chrome says 'filed at 0800 from the bridge'; headlines say
'this is news, read it like news.' Sticky aurora-glow rule under the
masthead is the only sanctioned Australis gradient.
Edition stamp (AM/PM in big mono Australis-yellow) is the signature
piece — establishes the twice-daily rhythm at a glance.
All filtering + LLM + scheduling knobs in .env. Subreddit list is
implicit (read from Miniflux), so adding a sub = subscribing in
Miniflux, no config edit on this stack.
This commit is contained in:
@@ -0,0 +1,139 @@
|
||||
# news-digest
|
||||
|
||||
Twice-daily LLM-curated briefing across Reddit (via JSON API) and
|
||||
Miniflux's Tech aggregators category. Output is a single static HTML
|
||||
page styled in the Australis design system with editorial-serif
|
||||
headlines (Fraunces) — operations-center chrome wrapping news content.
|
||||
|
||||
## Why this stack exists
|
||||
|
||||
After a few subreddits + HN + Lobste.rs, the Miniflux inbox gets
|
||||
noisy. This stack:
|
||||
|
||||
1. Pulls **top-of-day** posts per subreddit from Reddit's public JSON
|
||||
API (gives us scores + upvote ratios — RSS doesn't).
|
||||
2. Filters by `score >= 50` and `upvote_ratio >= 0.85` (configurable)
|
||||
to drop flame-bait and low-effort posts.
|
||||
3. Pulls non-Reddit recent items from Miniflux's Tech aggregators
|
||||
category (HN, Lobste.rs).
|
||||
4. Sends each source through `qwen3.5-35-a3b` on llama-swap (one
|
||||
batched call per source — efficient) for a one-sentence tl;dr +
|
||||
single-word tag (news / tutorial / release / discussion / question
|
||||
/ showcase / drama / meme / other).
|
||||
5. Renders an HTML page styled with Australis tokens + Fraunces
|
||||
serif headlines.
|
||||
6. Static page is served by a tiny `nginx:alpine`. Cron writes
|
||||
`/output/index.html` atomically; nginx serves whichever copy is
|
||||
there.
|
||||
|
||||
Two editions per day: 8am and 8pm local. Plus per-edition archives
|
||||
at `/edition-YYYY-MM-DD-{am,pm}.html`.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two containers, both on `traefik-net`, sharing a bind-mounted
|
||||
output dir:
|
||||
|
||||
```
|
||||
news-digest-worker (python:3.12-alpine + cron)
|
||||
├── busybox crond fires at 0 8,20 * * *
|
||||
├── digest.py:
|
||||
│ ├── miniflux /v1/feeds → discover subreddits
|
||||
│ ├── reddit JSON top/.json?t=day per sub (gentle 1.5s sleep)
|
||||
│ ├── miniflux /v1/entries → tech aggregators
|
||||
│ ├── llama-swap /v1/chat/completions → batched per source
|
||||
│ └── jinja2 render → /output/index.html (atomic .tmp + rename)
|
||||
│ → /output/edition-2026-04-26-pm.html
|
||||
└── style.css served from /output (copied at deploy)
|
||||
|
||||
news-digest-web (nginx:alpine)
|
||||
└── serves /output as / on host port 8181
|
||||
└── homepage card via container labels (group=News)
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
scripts/elway ana-docker --playbook playbooks/deploy-news-digest.yaml
|
||||
```
|
||||
|
||||
After first deploy, **fill in MINIFLUX_PASSWORD on the host**:
|
||||
|
||||
```bash
|
||||
ssh ana-docker '
|
||||
cd /opt/docker/compose/news-digest
|
||||
sed -i "s|^MINIFLUX_PASSWORD=.*|MINIFLUX_PASSWORD=<your-miniflux-admin-password>|" .env
|
||||
docker compose up -d
|
||||
'
|
||||
```
|
||||
|
||||
The container runs the first digest immediately if `/output/index.html`
|
||||
doesn't exist, so the page populates within a minute or two of bringing
|
||||
the stack up with real credentials.
|
||||
|
||||
Visit <http://10.250.50.70:8181> to read.
|
||||
|
||||
## Tuning the noise floor
|
||||
|
||||
Defaults in `.env.example`:
|
||||
|
||||
| Knob | Default | Effect |
|
||||
|---|---|---|
|
||||
| `DIGEST_REDDIT_HOURS` | 12 | Look-back window (matches twice-daily cadence) |
|
||||
| `DIGEST_MIN_SCORE` | 50 | Reddit minimum upvotes to consider |
|
||||
| `DIGEST_MIN_RATIO` | 0.85 | Reddit minimum upvote ratio (skips flamebait) |
|
||||
| `DIGEST_MAX_PER_SUB` | 8 | Cap per subreddit, post-filter |
|
||||
| `DIGEST_MINIFLUX_HOURS` | 12 | Look-back window for HN/Lobste.rs |
|
||||
| `DIGEST_MINIFLUX_MAX` | 8 | Cap per non-Reddit feed |
|
||||
|
||||
For a busier day, lower `DIGEST_MIN_SCORE`. For a quieter morning
|
||||
edition, raise it. Edit `.env`, no rebuild needed — the worker reads
|
||||
env on each cron fire.
|
||||
|
||||
## Adding more subreddits
|
||||
|
||||
The digest picks up subreddit feeds from Miniflux automatically — any
|
||||
feed whose URL starts with `https://www.reddit.com/r/<sub>/` gets
|
||||
queried. To add a sub, just subscribe in Miniflux (UI or API). The
|
||||
next digest run includes it.
|
||||
|
||||
## Updating the LLM model
|
||||
|
||||
```bash
|
||||
ssh ana-docker '
|
||||
cd /opt/docker/compose/news-digest
|
||||
sed -i "s|^LLAMA_SWAP_MODEL=.*|LLAMA_SWAP_MODEL=<new-model>|" .env
|
||||
docker compose up -d
|
||||
'
|
||||
```
|
||||
|
||||
The model must be loaded in llama-swap's `config.yaml`. Check
|
||||
`http://10.250.50.54:9292/v1/models` for what's available. Models
|
||||
with tool/JSON-mode support give better summarization quality;
|
||||
`qwen3.5-35-a3b` is the current default.
|
||||
|
||||
## Forcing a fresh digest now
|
||||
|
||||
```bash
|
||||
ssh ana-docker 'docker exec news-digest-worker python3 /app/digest.py'
|
||||
```
|
||||
|
||||
Runs the full pipeline once, ignoring cron. Useful after changing
|
||||
filtering knobs or adding feeds.
|
||||
|
||||
## Logs
|
||||
|
||||
```bash
|
||||
ssh ana-docker 'docker logs --tail 100 news-digest-worker'
|
||||
```
|
||||
|
||||
Worker logs each phase (subreddit discovery / fetching / summarizing /
|
||||
rendering) with timestamps. Per-source LLM filter results show how
|
||||
many items were kept vs skipped.
|
||||
|
||||
## License + attribution
|
||||
|
||||
Reddit content surfaced here is owned by its authors and Reddit. The
|
||||
digest is a derived index pointing at original sources — every item
|
||||
links back to the Reddit thread (and to the external link if the
|
||||
post linked out). Same for HN / Lobste.rs.
|
||||
Reference in New Issue
Block a user