esh-pfi-infrastructure/stacks/news-digest/README.md

news-digest

Twice-daily LLM-curated briefing across Reddit (via JSON API) and Miniflux's Tech aggregators category. Output is a single static HTML page styled in the Australis design system with editorial-serif headlines (Fraunces) — operations-center chrome wrapping news content.

Why this stack exists

After a few subreddits + HN + Lobste.rs, the Miniflux inbox gets noisy. This stack:

1. Pulls top-of-day posts per subreddit from Reddit's public JSON API (gives us scores + upvote ratios — RSS doesn't).
2. Filters by score >= 50 and upvote_ratio >= 0.85 (configurable) to drop flame-bait and low-effort posts.
3. Pulls non-Reddit recent items from Miniflux's Tech aggregators category (HN, Lobste.rs).
4. Sends each source through qwen3.5-35-a3b on llama-swap (one batched call per source — efficient) for a one-sentence tl;dr + single-word tag (news / tutorial / release / discussion / question / showcase / drama / meme / other).
5. Renders an HTML page styled with Australis tokens + Fraunces serif headlines.
6. The page is served by a tiny FastAPI app on uvicorn that also exposes /api/{hidden,hide,restore} for the per-item × button (state in /output/hidden.json, shared across every device the user opens the digest from).

Two editions per day: 8am and 8pm local. Plus per-edition archives at /edition-YYYY-MM-DD-{am,pm}.html.

Architecture

Two containers built from the same Dockerfile, both on traefik-net, sharing a bind-mounted output dir:

news-digest-worker  (default ENTRYPOINT — busybox crond)
  ├── busybox crond fires at 0 8,20 * * *
  ├── digest.py:
  │     ├── miniflux /v1/feeds → discover subreddits
  │     ├── reddit JSON top/.json?t=day per sub (gentle 1.5s sleep)
  │     ├── miniflux /v1/entries → tech aggregators
  │     ├── llama-swap /v1/chat/completions → batched per source
  │     └── jinja2 render → /output/index.html (atomic .tmp + rename)
  │                       → /output/edition-2026-04-26-pm.html
  └── style.css / favicon.svg / app.js staged in /output at deploy

news-digest-web     (entrypoint overridden → uvicorn web:app)
  ├── /          → serve /output as static (index.html as default)
  ├── /api/hidden        GET   → JSON array of hidden item ids
  ├── /api/hide          POST  → {id} → adds id to hidden.json
  ├── /api/restore       POST  → {id} → removes id from hidden.json
  ├── /output/hidden.json — durable state (atomic writes + threading lock)
  └── homepage card via container labels (group=News)

Hidden state is server-side and global per-user (single-user setup): hide an article once and it stays hidden in any future edition that includes the same article. The "Hidden (N)" tray at the bottom of each page shows items hidden FROM THE CURRENT PAGE; older hidden ids that aren't present on this page just sit silently in hidden.json and continue to filter future editions.

Deploy

scripts/elway ana-docker --playbook playbooks/deploy-news-digest.yaml

After first deploy, fill in MINIFLUX_PASSWORD on the host:

ssh ana-docker '
  cd /opt/docker/compose/news-digest
  sed -i "s|^MINIFLUX_PASSWORD=.*|MINIFLUX_PASSWORD=<your-miniflux-admin-password>|" .env
  docker compose up -d
'

The container runs the first digest immediately if /output/index.html doesn't exist, so the page populates within a minute or two of bringing the stack up with real credentials.

Visit http://10.250.50.70:8181 to read.

Tuning the noise floor

Defaults in .env.example:

Knob	Default	Effect
DIGEST_REDDIT_HOURS	12	Look-back window (matches twice-daily cadence)
DIGEST_MIN_SCORE	50	Reddit minimum upvotes to consider
DIGEST_MIN_RATIO	0.85	Reddit minimum upvote ratio (skips flamebait)
DIGEST_MAX_PER_SUB	8	Cap per subreddit, post-filter
DIGEST_MINIFLUX_HOURS	12	Look-back window for HN/Lobste.rs
DIGEST_MINIFLUX_MAX	8	Cap per non-Reddit feed

For a busier day, lower DIGEST_MIN_SCORE. For a quieter morning edition, raise it. Edit .env, no rebuild needed — the worker reads env on each cron fire.

Adding more subreddits

The digest picks up subreddit feeds from Miniflux automatically — any feed whose URL starts with https://www.reddit.com/r/<sub>/ gets queried. To add a sub, just subscribe in Miniflux (UI or API). The next digest run includes it.

Updating the LLM model

ssh ana-docker '
  cd /opt/docker/compose/news-digest
  sed -i "s|^LLAMA_SWAP_MODEL=.*|LLAMA_SWAP_MODEL=<new-model>|" .env
  docker compose up -d
'

The model must be loaded in llama-swap's config.yaml. Check http://10.250.50.54:9292/v1/models for what's available. Models with tool/JSON-mode support give better summarization quality; qwen3.5-35-a3b is the current default.

Forcing a fresh digest now

ssh ana-docker 'docker exec news-digest-worker python3 /app/digest.py'

Runs the full pipeline once, ignoring cron. Useful after changing filtering knobs or adding feeds.

Logs

ssh ana-docker 'docker logs --tail 100 news-digest-worker'

Worker logs each phase (subreddit discovery / fetching / summarizing / rendering) with timestamps. Per-source LLM filter results show how many items were kept vs skipped.

License + attribution

Reddit content surfaced here is owned by its authors and Reddit. The digest is a derived index pointing at original sources — every item links back to the Reddit thread (and to the external link if the post linked out). Same for HN / Lobste.rs.