feat(hrafn): register the browser-fetch service and tighten its env perms
nevermore-claude handed hrafn to infra-ops for uptime ownership. Intake: verified health independently, brought it into the inventory, and fixed one security defect found during the check. - canonicalize stacks/hrafn/ (compose.yaml + .env.example + README) - list hrafn in the ana-docker running-stacks table - README records the load-bearing bits: shm_size 1gb is required for Chromium, playwright and the base image are version-coupled, SSRF denies private targets by default, and restart:unless-stopped does NOT act on healthcheck failure Security fix applied on the host, not in this tree: /opt/docker/compose/ hrafn/.env was 0644 with a live 57-char bearer token in it, readable by every local account (verified by reading it as `nobody`). Tightened to 0600; owner unchanged so the deploy path still works, confirmed via `docker compose config` as lkraven. Container untouched and still healthy. Known gaps recorded rather than silently accepted: no CI deploy, and the image builds from an rsync'd working tree living in the compose directory, so the running image has no commit provenance.
This commit is contained in:
@@ -50,6 +50,7 @@ General-purpose Docker host for the Anaheim colo. Runs everything at `10.250.0.0
|
||||
| restic rest-server | 8000 | Anaheim-side restic endpoint (writes to the NFS mount at `/mnt/backup/restic/repo/ana/`, backed by the Debian file server at `10.250.50.50`); paired with `rest-server-nh3` on the Synology for the NH3 side |
|
||||
| backrest | 9898 | Fleet-wide restic snapshot viewer / restore UI — points at both rest-servers |
|
||||
| it-tools | 8780 | Dev utilities |
|
||||
| hrafn | internal only | Browser-fetch service (real Chromium behind a REST API) for bot-gated sites; consumers reach `http://hrafn:8080` on `traefik-net`. No host port by design. See `stacks/hrafn/` |
|
||||
| mattermost | — | Stopped; kept around for reference |
|
||||
|
||||
Portainer was retired from this host; stack management is now handled via Dockge + Beszel.
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# hrafn — fleet browser-fetch service. Copy to `.env` and set HRAFN_TOKEN.
|
||||
# This file is the canonical config surface (contract § Configuration).
|
||||
|
||||
# ── auth (required) ──────────────────────────────────────────────────
|
||||
# Shared bearer token required on POST /v1/fetch. Provision via the
|
||||
# vault (secrets-broker); rotation = rotate the vault entry + recycle.
|
||||
HRAFN_TOKEN=CHANGE_ME
|
||||
|
||||
# ── network ──────────────────────────────────────────────────────────
|
||||
HRAFN_BIND=0.0.0.0
|
||||
HRAFN_PORT=8080
|
||||
|
||||
# ── fetch limits ─────────────────────────────────────────────────────
|
||||
# Max post-content-decoding body bytes of the main navigation response;
|
||||
# oversize is rejected (413), never truncated. Both raw and dom modes.
|
||||
HRAFN_MAX_BODY_BYTES=10485760
|
||||
# Whole-request budget default + hard cap (seconds). timeout_s INCLUDES
|
||||
# politeness-queue wait. A request may not exceed the cap.
|
||||
HRAFN_DEFAULT_TIMEOUT_S=30
|
||||
HRAFN_MAX_TIMEOUT_S=120
|
||||
# Default dom-mode wait state: load | domcontentloaded | networkidle.
|
||||
HRAFN_DEFAULT_WAIT_UNTIL=domcontentloaded
|
||||
|
||||
# ── concurrency + browser lifecycle ──────────────────────────────────
|
||||
# Global in-flight cap (acquired on leaving the politeness queue).
|
||||
HRAFN_CONCURRENCY=4
|
||||
# Recycle the browser after N requests or M seconds of age.
|
||||
HRAFN_BROWSER_MAX_REQUESTS=200
|
||||
HRAFN_BROWSER_MAX_AGE_S=1800
|
||||
|
||||
# ── politeness gate (per registrable domain) ─────────────────────────
|
||||
# Minimum seconds between fetch STARTS per domain (0 = no throttle).
|
||||
HRAFN_DOMAIN_MIN_INTERVAL_S=5
|
||||
# Per-domain interval overrides: comma-separated domain=seconds pairs.
|
||||
# HRAFN_DOMAIN_OVERRIDES=reddit.com=10, example.com=2
|
||||
HRAFN_DOMAIN_OVERRIDES=
|
||||
# Waiting-request queue depth per domain (excludes the in-flight fetch;
|
||||
# 0 = no queue). Overflow → 429 queue_full.
|
||||
HRAFN_QUEUE_DEPTH=4
|
||||
# Upstream Retry-After (on 429/503) embargoes the domain; clamped to
|
||||
# this many seconds so a hostile Retry-After can't lock a domain for a day.
|
||||
HRAFN_MAX_EMBARGO_S=300
|
||||
|
||||
# ── SSRF ─────────────────────────────────────────────────────────────
|
||||
# Private/reserved targets are DENIED by default. To permit specific
|
||||
# internal targets, list IPs/CIDRs (comma-separated). Empty = deny all
|
||||
# private ranges (the safe default). Write entries in IPv4 form for IPv4
|
||||
# targets: resolved IPv4-mapped IPv6 addresses are canonicalised to IPv4
|
||||
# before matching, so an IPv6-mapped CIDR (::ffff:10.0.0.0/120) will not
|
||||
# match them.
|
||||
# HRAFN_PRIVATE_ALLOWLIST=10.0.0.0/8, 192.168.1.5
|
||||
HRAFN_PRIVATE_ALLOWLIST=
|
||||
|
||||
# ── live smoke (dev only) ────────────────────────────────────────────
|
||||
# Set to 1 to enable the opt-in Reddit RSS smoke test. Off in CI.
|
||||
# HRAFN_LIVE_SMOKE=1
|
||||
@@ -0,0 +1,81 @@
|
||||
# hrafn
|
||||
|
||||
Browser-fetch service — a genuine Chromium kept behind a REST API so consumers
|
||||
never have to embed a browser themselves. It exists because bot-gated sites
|
||||
(Reddit first) 403 every non-browser client; hrafn is the one place in the fleet
|
||||
that keeps a real browser warm.
|
||||
|
||||
**Deploys to:**
|
||||
- **ana-docker** (10.250.50.70) — internal-only, no published host port.
|
||||
Consumers reach it at `http://hrafn:8080` on `traefik-net`.
|
||||
|
||||
Built and handed to infra-ops for uptime ownership by nevermore-claude,
|
||||
2026-08-22. Upstream source: `vh/hrafn` (private gitea repo).
|
||||
|
||||
## Reaching it
|
||||
|
||||
There is **no host port** — this is deliberate (contract § Deployment). Only
|
||||
containers on `traefik-net` can talk to it:
|
||||
|
||||
```bash
|
||||
# from another container on traefik-net
|
||||
curl -H "Authorization: Bearer $HRAFN_TOKEN" \
|
||||
-X POST http://hrafn:8080/v1/fetch -d '{"url":"https://example.com"}'
|
||||
|
||||
# from the host, for diagnostics only
|
||||
ssh infra-ops@10.250.50.70 \
|
||||
'sudo docker run --rm --network traefik-net curlimages/curl:latest \
|
||||
-s -o /dev/null -w "%{http_code}\n" http://hrafn:8080/readyz'
|
||||
```
|
||||
|
||||
`/readyz` and the other health endpoints are unauthenticated; `POST /v1/fetch`
|
||||
requires the bearer token.
|
||||
|
||||
## Credentials
|
||||
|
||||
`HRAFN_TOKEN` lives in `/opt/docker/compose/hrafn/.env` on the host and is
|
||||
vaulted at `ana-docker/hrafn/bearer-token` (secrets-broker). The `.env` is
|
||||
**0600** — it was 0644 on handoff, which exposed a live bearer token to every
|
||||
local account on the box; tightened 2026-08-22. Keep it 0600.
|
||||
|
||||
## Operational notes
|
||||
|
||||
- **`shm_size: 1gb` is load-bearing.** Chromium crashes on some pages with
|
||||
Docker's 64 MB default. Don't trim it to save memory.
|
||||
- **Playwright and the base image are version-coupled.** Base is
|
||||
`mcr.microsoft.com/playwright/python:v1.62.0-noble` with playwright pinned to
|
||||
`1.62.0`. Bump both together or the browser binary and the client library
|
||||
disagree.
|
||||
- **SSRF is denied by default** — private and reserved targets are refused
|
||||
unless listed in `HRAFN_PRIVATE_ALLOWLIST`, which is empty. Leave it empty
|
||||
unless there is a specific internal target and a reason.
|
||||
- **`restart: unless-stopped` does not act on the healthcheck.** Docker restarts
|
||||
on *exit*, not on *unhealthy*. A wedged-but-running Chromium will sit there
|
||||
marked unhealthy indefinitely. Beszel on this host reports container status,
|
||||
so that is the alerting path.
|
||||
|
||||
## Deploy
|
||||
|
||||
**No CI deploy is wired.** As of the handoff the image builds from a working
|
||||
tree rsync'd into the compose directory, so `/opt/docker/compose/hrafn/`
|
||||
contains the full source (`src/`, `tests/`, `pyproject.toml`, …) rather than
|
||||
just a compose file. That is a deviation from the house convention and means
|
||||
the running image has no commit provenance.
|
||||
|
||||
Current update path:
|
||||
|
||||
```bash
|
||||
ssh infra-ops@10.250.50.70
|
||||
cd /opt/docker/compose/hrafn
|
||||
# rsync the updated working tree in, then:
|
||||
sudo docker compose build && sudo docker compose up -d
|
||||
sudo docker compose ps # expect (healthy)
|
||||
```
|
||||
|
||||
## Known gaps
|
||||
|
||||
| gap | impact | status |
|
||||
|---|---|---|
|
||||
| No CI deploy | manual rsync + build; no provenance for what is running | offered by nevermore-claude, not yet built |
|
||||
| Source tree in the compose dir | deviates from `/opt/docker/compose/<stack>/compose.yaml` convention | needs a host-clone or registry-image deploy shape |
|
||||
| No homepage entry | internal-only service, so arguably correct | intentional for now |
|
||||
@@ -0,0 +1,25 @@
|
||||
# hrafn — internal fleet service. No published host port; consumers reach
|
||||
# it at http://hrafn:8080 on the shared network (contract § Deployment).
|
||||
services:
|
||||
hrafn:
|
||||
build: .
|
||||
image: local/hrafn:${HRAFN_TAG:-v1}
|
||||
container_name: hrafn
|
||||
restart: unless-stopped
|
||||
env_file: .env
|
||||
networks:
|
||||
- traefik-net
|
||||
# Chromium wants a larger shared-memory segment than Docker's 64MB
|
||||
# default or it crashes on some pages.
|
||||
shm_size: "1gb"
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c",
|
||||
"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/readyz').status==200 else 1)"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
|
||||
networks:
|
||||
traefik-net:
|
||||
external: true
|
||||
Reference in New Issue
Block a user