esh-pfi-infrastructure/stacks/news-digest/README.md

news-digest

Twice-daily LLM-curated briefing across Reddit (via JSON API) and Miniflux's Tech aggregators category. Output is a single static HTML page styled in the Australis design system with editorial-serif headlines (Fraunces) — operations-center chrome wrapping news content.

Why this stack exists

After a few subreddits + HN + Lobste.rs, the Miniflux inbox gets noisy. This stack:

1. Pulls top-of-day posts per subreddit from Reddit's public JSON API (gives us scores + upvote ratios — RSS doesn't).
2. Filters by score >= 50 and upvote_ratio >= 0.85 (configurable) to drop flame-bait and low-effort posts.
3. Pulls non-Reddit recent items from Miniflux's Tech aggregators category (HN, Lobste.rs).
4. Sends each source through granite-4-small on llama-swap (one batched call per source — efficient) for a 2-3 sentence summary + single-word tag (news / tutorial / release / discussion / question / showcase / drama / meme / other). World + Local headlines also get summarized in one batch call per category. (Original default was qwen3.5-35-a3b but its model file is broken on launch.)
5. Renders an HTML page styled with Australis tokens + Fraunces serif headlines.
6. The page is served by a tiny FastAPI app on uvicorn that also exposes /api/{hidden,hide,restore} for the per-item × button (state in /output/hidden.json, shared across every device the user opens the digest from).

Two editions per day by default (0800 / 2000 local), parametrized via DIGEST_CRON_AM / DIGEST_CRON_PM env so each per-user instance can fire on its own schedule. Plus per-edition archives at /edition-YYYY-MM-DD-{am,pm}.html.

Architecture

Two containers built from the same Dockerfile, both on traefik-net, sharing a bind-mounted output dir:

news-digest-worker  (default ENTRYPOINT — busybox crond)
  ├── busybox crond fires at 0 8,20 * * *
  ├── digest.py:
  │     ├── miniflux /v1/feeds → discover subreddits
  │     ├── reddit JSON top/.json?t=day per sub (gentle 1.5s sleep)
  │     ├── miniflux /v1/entries → tech aggregators
  │     ├── llama-swap /v1/chat/completions → batched per source
  │     └── jinja2 render → /output/index.html (atomic .tmp + rename)
  │                       → /output/edition-2026-04-26-pm.html
  └── style.css / favicon.svg / app.js staged in /output at deploy

news-digest-web     (entrypoint overridden → uvicorn web:app)
  ├── /          → serve /output as static (index.html as default)
  ├── /api/hidden        GET   → JSON array of hidden item ids
  ├── /api/hide          POST  → {id} → adds id to hidden.json
  ├── /api/restore       POST  → {id} → removes id from hidden.json
  ├── /output/hidden.json — durable state (atomic writes + threading lock)
  └── homepage card via container labels (group=News)

Hidden state is server-side and global per-user (single-user setup): hide an article once and it stays hidden in any future edition that includes the same article. The "Hidden (N)" tray at the bottom of each page shows items hidden FROM THE CURRENT PAGE; older hidden ids that aren't present on this page just sit silently in hidden.json and continue to filter future editions.

Deploy

scripts/elway ana-docker --playbook playbooks/deploy-news-digest.yaml

After first deploy, fill in MINIFLUX_PASSWORD on the host:

ssh ana-docker '
  cd /opt/docker/compose/news-digest
  sed -i "s|^MINIFLUX_PASSWORD=.*|MINIFLUX_PASSWORD=<your-miniflux-admin-password>|" .env
  docker compose up -d
'

The container runs the first digest immediately if /output/index.html doesn't exist, so the page populates within a minute or two of bringing the stack up with real credentials.

Visit http://10.250.50.70:8181 to read.

Tuning the noise floor

Defaults in .env.example:

Knob	Default	Effect
DIGEST_REDDIT_HOURS	12	Look-back window (matches twice-daily cadence)
DIGEST_MIN_SCORE	50	Reddit minimum upvotes to consider
DIGEST_MIN_RATIO	0.85	Reddit minimum upvote ratio (skips flamebait)
DIGEST_MAX_PER_SUB	8	Cap per subreddit, post-filter
DIGEST_MINIFLUX_HOURS	12	Look-back window for HN/Lobste.rs
DIGEST_MINIFLUX_MAX	8	Cap per non-Reddit feed

For a busier day, lower DIGEST_MIN_SCORE. For a quieter morning edition, raise it. Edit .env, no rebuild needed — the worker reads env on each cron fire.

Adding more subreddits

The digest picks up subreddit feeds from Miniflux automatically — any feed whose URL starts with https://www.reddit.com/r/<sub>/ gets queried. To add a sub, just subscribe in Miniflux (UI or API). The next digest run includes it.

Updating the LLM model

ssh ana-docker '
  cd /opt/docker/compose/news-digest
  sed -i "s|^LLAMA_SWAP_MODEL=.*|LLAMA_SWAP_MODEL=<new-model>|" .env
  docker compose up -d
'

The model must be loaded in llama-swap's config.yaml. Check http://10.250.50.54:9292/v1/models for what's available. Models with tool/JSON-mode support give better summarization quality; granite-4-small is the current default (small ~4B, fast ~1s/call, no extended-thinking phase eating the token budget). Avoid qwen3.5-35-a3b (model file broken — process exits on launch) and qwen3.6-35-a3b (defaults to thinking mode, eats budget without output).

Forcing a fresh digest now

ssh ana-docker 'docker exec news-digest-worker /usr/local/bin/run-digest.sh'

Runs the full pipeline once, ignoring cron. Useful after changing filtering knobs or adding feeds.

Customizing the run schedule

Times come from two env vars on the worker, written into the busybox crontab at container start. Standard 5-field cron syntax.

Var	Default	Effect
DIGEST_CRON_AM	0 8 * * *	morning fire
DIGEST_CRON_PM	0 20 * * *	evening fire

busybox crond honors $NEWS_DIGEST_TZ (defaults to America/Los_Angeles), so values are interpreted in the configured TZ.

# Shift the canonical instance to 7am / 6pm
ssh ana-docker '
  cd /opt/docker/compose/news-digest
  sed -i "s|^DIGEST_CRON_AM=.*|DIGEST_CRON_AM=0 7 * * *|" .env
  sed -i "s|^DIGEST_CRON_PM=.*|DIGEST_CRON_PM=0 18 * * *|" .env
  docker compose up -d --force-recreate news-digest-worker
'

Verify the rendered crontab:

ssh ana-docker 'docker exec news-digest-worker cat /etc/crontabs/root'

Multi-tenant: one instance per teammate

Architecture: shared miniflux + per-user digest stack. Miniflux already supports multi-user natively (each user has their own feeds, categories, hide-state); we layer a separate news-digest stack per user on its own port + output dir, scoped to that miniflux user's credentials.

Onboarding a new user

# Defaults — 8am / 8pm local, random password
scripts/add-digest-user.sh alice

# Custom hours
scripts/add-digest-user.sh bob --am "0 6 * * *" --pm "0 17 * * *"

# Weekday-only PM run
scripts/add-digest-user.sh carol --pm "30 18 * * 1-5"

# Pin a known password (still creates the miniflux user if missing)
scripts/add-digest-user.sh dan 'pickyourpassword' --am "0 9 * * *"

What the script does:

1. Reads miniflux admin creds from ana-docker:/opt/docker/compose/miniflux/.env.
2. Allocates the next free NEWS_DIGEST_PORT (scans existing news-digest + digest-* .env files).
3. Creates the miniflux user via the admin API. Already-exists is non-fatal (kept; password not reset).
4. Provisions per-user dirs at /opt/docker/compose/digest-<user>/ and /opt/docker/data/digest-<user>/ (one-time sudo prompt — the script falls back to printing the manual command if there's no TTY).
5. Materializes a per-user .env (inherits NEWS_DIGEST_TAG from the canonical stack so all tenants run the same image).
6. Brings the stack up via docker compose -p digest-<user> up -d.
7. Runs seed-headlines.py against miniflux as the new user (creates the World + Local categories with default feeds).
8. Triggers a first digest run so the page isn't blank.

Outputs the digest URL, miniflux login, and rendered cron schedule.

Per-user file layout

ana-docker:
  /opt/docker/compose/digest-<user>/   # compose + .env + build context
    .env                               # auto-generated, contains MINIFLUX_PASSWORD
    compose.yaml
    Dockerfile + digest.py + ...       # build context (image is shared/cached)
  /opt/docker/data/digest-<user>/      # rendered HTML + per-user hidden.json
    index.html
    edition-YYYY-MM-DD-{am,pm}.html
    hidden.json
    .article-cache.json                # extracted article text, 7-day TTL

Container names: digest-<user>-worker and digest-<user>-web.

Updating an existing user's schedule

Re-running the script with the same username is idempotent:

# Change alice's evening run to 5:30pm
scripts/add-digest-user.sh alice --pm "30 17 * * *"

The miniflux user is kept (password unchanged), the .env is re-materialized with the new schedule, and the worker container is recreated. Use the same flow to bump cron times, rotate passwords (by passing a new one explicitly), or rerun feed seeding.

Removing a user

ssh ana-docker '
  cd /opt/docker/compose/digest-alice
  docker compose -p digest-alice down -v
'
# Optional: nuke compose dir + rendered output
ssh -t ana-docker 'sudo rm -rf /opt/docker/compose/digest-alice /opt/docker/data/digest-alice'
# Optional: delete the miniflux user via the admin UI at http://10.250.50.70:8080/

The digest is gone immediately; the miniflux account stays around unless you delete it explicitly (cheap to leave; ~zero resource cost when no stack is querying its feeds).

Logs

ssh ana-docker 'docker logs --tail 100 news-digest-worker'

Worker logs each phase (subreddit discovery / fetching / summarizing / rendering) with timestamps. Per-source LLM filter results show how many items were kept vs skipped.

License + attribution

Reddit content surfaced here is owned by its authors and Reddit. The digest is a derived index pointing at original sources — every item links back to the Reddit thread (and to the external link if the post linked out). Same for HN / Lobste.rs.