feat(searxng): move to nh3-docker, update, and expose as an MCP tool
The ana-docker instance was returning zero results for every query while reporting healthy — 4.5 months stale (2026.4.17 against a current 2026.9.3), its engine scrapers rotted against sites that had changed. /healthz proves the web app answers and says nothing about whether search works, so seven days of green sat on top of a search box that found nothing. Moved to nh3-docker rather than updated in place, because the colo egress is the other half of the problem: 38.120.12.42 is a datacenter address that DuckDuckGo and Startpage CAPTCHA, while nh3-docker egresses residentially at 70.230.226.88. Same reasoning as the fleet's residential proxy for yt-dlp, applied at the source instead of around it. Config corrected along the way: base_url said searxng.pfi.local, a name retired on 2026-08-19, while the environment said something else — the env won so nothing broke and the file quietly lied. The karmasearch.videos removal key never matched, because the engine's real name has a space. scripts/searxng-health.sh asserts results > 0 across three unrelated queries. That is the check that would have caught this, and the only kind that can: the mechanism was healthy throughout. services/searxng-mcp exposes it as `web_search` at user scope, so every Claude Code session has it. Zero results raise rather than returning an empty list — an empty list is indistinguishable from a broken aggregator, which is precisely how this hid. Old instance stopped and removed; DNS alias repointed to searxng.nh3.internal.
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Required. Cryptographic signing key (cookies, etc). Generate with:
|
||||
# openssl rand -hex 32
|
||||
# The real value is vaulted at nh3-docker/searxng-secret — fetch it with
|
||||
# secret get nh3-docker/searxng-secret
|
||||
# and never commit it here.
|
||||
SEARXNG_SECRET=changeme_openssl_rand_hex_32
|
||||
@@ -0,0 +1,76 @@
|
||||
# searxng
|
||||
|
||||
Privacy-respecting metasearch. **Runs on nh3-docker** (`10.100.50.40:9996`,
|
||||
`searxng.nh3.internal`), and is exposed to every Claude Code session on nh3-dev
|
||||
as the `web_search` MCP tool.
|
||||
|
||||
## ⚠ Why NH3 and not the colo
|
||||
|
||||
Measured 2026-09-03:
|
||||
|
||||
ana-docker egress 38.120.12.42 datacenter -> DuckDuckGo/Startpage CAPTCHA
|
||||
nh3-docker egress 70.230.226.88 residential -> no gate on that account
|
||||
|
||||
Search engines gate datacenter ranges. Running the aggregator from the
|
||||
residential-egress site removes the problem at the source rather than proxying
|
||||
around it — the same reason the fleet keeps a residential SOCKS5 proxy on
|
||||
nh3-dev for yt-dlp. If NH3's egress ever changes, `outgoing.proxies` in
|
||||
`conf/searxng-settings.yml` has the fallback commented in place.
|
||||
|
||||
⚠ It is **not** a complete fix: `brave`, `duckduckgo` and `startpage` still
|
||||
CAPTCHA from here. `google cse` carries general search at ~20 results/query, and
|
||||
`yandex`, `wiby`, `github`, `stackoverflow` and `marginalia` all work. General
|
||||
search is therefore effectively single-engine — if `google cse` breaks, the
|
||||
instance goes quiet, which is exactly the failure below.
|
||||
|
||||
## ⚠ /healthz cannot tell you whether search works
|
||||
|
||||
On 2026-09-03 the old ana-docker instance was found returning **zero results for
|
||||
every query**, for an unknown number of weeks, while:
|
||||
|
||||
- the container reported `healthy` for 7 straight days, 0 restarts;
|
||||
- the Homepage card was green;
|
||||
- `/healthz` returned 200 every 30 seconds.
|
||||
|
||||
It was running **2026.4.17 while current was 2026.9.3** — 4.5 months of engine
|
||||
scrapers rotting against sites that had changed their markup. `:latest` means
|
||||
"latest at pull time", and nothing re-pulls on its own.
|
||||
|
||||
Proven by experiment before touching anything: the new image, same settings
|
||||
file, same host, same query returned **20 results where the running one returned
|
||||
0**.
|
||||
|
||||
**`scripts/searxng-health.sh` asserts the property, not the mechanism** — three
|
||||
unrelated queries must each return results > 0. Run it after any change, and
|
||||
periodically; it is the only thing that catches rot.
|
||||
|
||||
## Update
|
||||
|
||||
ssh infra-ops@nh3-docker 'cd /opt/docker/compose/searxng && \
|
||||
sudo docker compose pull && sudo docker compose up -d'
|
||||
scripts/searxng-health.sh
|
||||
|
||||
## The MCP tool
|
||||
|
||||
`services/searxng-mcp/` — installed with `uv tool install` and registered at
|
||||
**user scope** (`claude mcp add --scope user searxng searxng-mcp`), so every
|
||||
Claude Code session gets `web_search` with no per-project setup.
|
||||
|
||||
⚠ **Zero results raise an error rather than returning an empty list.** An empty
|
||||
list is indistinguishable from a broken aggregator, and that ambiguity is what
|
||||
let the old instance fail silently for weeks. Same reasoning as althing's
|
||||
"an unreachable post office is an OUTAGE, never an empty inbox".
|
||||
|
||||
⚠ Written against **mcp 2.x**, where `FastMCP` became `MCPServer` and the v1
|
||||
`@app.list_tools()` decorator API is gone. v1 examples fail at import with
|
||||
`'Server' object has no attribute 'list_tools'`.
|
||||
|
||||
⚠ `uv tool install --force` alone served a **cached build** and silently
|
||||
reinstalled the old code — the installed file still had the v1 API after the
|
||||
source no longer did. `--reinstall --no-cache` was required, and the check that
|
||||
caught it was `md5sum` of source vs installed.
|
||||
|
||||
## Secret
|
||||
|
||||
`SEARXNG_SECRET` lives in `/opt/docker/compose/searxng/.env` (0600, root) on the
|
||||
host and is vaulted at `nh3-docker/searxng-secret`. Never in git.
|
||||
+21
-55
@@ -1,38 +1,23 @@
|
||||
services:
|
||||
searxng:
|
||||
# ⚠ `:latest` means "latest AT PULL TIME", and nothing re-pulls on its own.
|
||||
# On 2026-09-03 this instance was found running 2026.4.17 — 4.5 months old —
|
||||
# while reporting `healthy` and returning ZERO results for every query,
|
||||
# because SearXNG engine scrapers rot as upstream sites change their markup
|
||||
# and the project ships near-daily releases to keep up. `docker compose pull
|
||||
# && up -d` is the update; scripts/searxng-health.sh is what tells you it is
|
||||
# needed, because /healthz cannot.
|
||||
image: searxng/searxng:latest
|
||||
container_name: searxng
|
||||
restart: unless-stopped
|
||||
# ------------------------------------------------------------------
|
||||
# Port binding — 9996 on all interfaces.
|
||||
# Change to "127.0.0.1:9996:8080" to restrict to localhost only.
|
||||
# Traefik handles public routing and TLS via the labels below.
|
||||
# ------------------------------------------------------------------
|
||||
ports:
|
||||
- 9996:8080
|
||||
# ------------------------------------------------------------------
|
||||
# Volumes
|
||||
# Config: settings.yml bind-mounted read-only into the container.
|
||||
volumes:
|
||||
- /opt/docker/conf/searxng/searxng-settings.yml:/etc/searxng/settings.yml:ro
|
||||
# ------------------------------------------------------------------
|
||||
# Environment — see https://docs.searxng.org/admin/settings/index.html
|
||||
# SEARXNG_SECRET — required for cryptographic signing (cookies, etc.)
|
||||
# BASE_URL — public URL SearXNG reports in pages/RSS/OPDS
|
||||
# INSTANCE_NAME — shown in the page title / footer
|
||||
# ------------------------------------------------------------------
|
||||
environment:
|
||||
- SEARXNG_SECRET=${SEARXNG_SECRET}
|
||||
# searxng.ana.internal, not the old searxng.pfi.local (migrated
|
||||
# 2026-08-19). `.local` is reserved for mDNS, so the old name was a
|
||||
# standards collision that happened to work; `.internal` is ICANN-
|
||||
# reserved for exactly this. The name is served by the fleet's AdGuard
|
||||
# resolvers from dns/internal.yaml — see scripts/dns-sync.py.
|
||||
- BASE_URL=https://searxng.ana.internal/
|
||||
- BASE_URL=http://10.100.50.40:9996/
|
||||
- INSTANCE_NAME=SearXNG
|
||||
# ------------------------------------------------------------------
|
||||
# Resource limits — tune for VM 102's available RAM/CPU
|
||||
# ------------------------------------------------------------------
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
@@ -40,26 +25,18 @@ services:
|
||||
cpus: "1.0"
|
||||
reservations:
|
||||
memory: 128M
|
||||
# ------------------------------------------------------------------
|
||||
# Health check — SearXNG /healthz is the canonical liveness probe.
|
||||
# ⚠ `--tries=1` MUST keep its `=1`. Written as two argv entries
|
||||
# (`- --tries` / `- --spider`) wget consumes `--spider` as the VALUE of
|
||||
# `--tries`, spider mode never engages, and every probe DOWNLOADS the
|
||||
# response to a file. On the ana-docker instance that left 295,287
|
||||
# `healthz.N` files in the container's writable layer, one per probe since
|
||||
# April; wget scanning them to pick the next free name is what blew the 10s
|
||||
# timeout and made the dashboard card flap. Self-worsening — each probe made
|
||||
# the next slower. Watch for `docker exec searxng ls | wc -l` climbing.
|
||||
#
|
||||
# ⚠️ `--tries=1` MUST keep its `=1`. This read `- --tries` / `- --spider`
|
||||
# as two separate argv entries until 2026-08-18, and in that form wget
|
||||
# consumed `--spider` as the VALUE of `--tries` — so spider mode never
|
||||
# engaged and every probe DOWNLOADED the response to a file instead of
|
||||
# just checking it. By the time it was caught the container's working
|
||||
# directory held 295,287 `healthz.N` files, one per probe since April,
|
||||
# and wget had to scan all of them to pick the next free filename. That
|
||||
# scan is what intermittently blew the 10s timeout and made the card on
|
||||
# the dashboard flap UNHEALTHY while the service itself was fine. It was
|
||||
# self-worsening: every probe made the next one slower.
|
||||
#
|
||||
# The junk lived in the container's writable layer (the only volume here
|
||||
# is the read-only settings mount), so recreating the container cleared
|
||||
# it. Symptom to watch for if this regresses: `docker exec searxng ls |
|
||||
# wc -l` climbing, and health log entries reading
|
||||
# "Health check exceeded timeout (10s)".
|
||||
# ------------------------------------------------------------------
|
||||
# ⚠ AND KNOW WHAT THIS PROBE DOES NOT TELL YOU: /healthz proves the web app
|
||||
# answers. It says nothing about whether any engine returns a result. Seven
|
||||
# days of `healthy` sat on top of a search box that found nothing.
|
||||
healthcheck:
|
||||
test:
|
||||
- CMD
|
||||
@@ -75,23 +52,12 @@ services:
|
||||
networks:
|
||||
- tnet
|
||||
labels:
|
||||
# Traefik configuration — auto-discovery via Docker provider
|
||||
- traefik.enable=true
|
||||
# Both names during the migration: `.internal` is the real one now, and
|
||||
# the old `.pfi.local` is kept as a fallback so anything still pointing
|
||||
# at it (a bookmark, a hardcoded config elsewhere) does not break the
|
||||
# day the name changes. Drop the second Host() once nothing uses it —
|
||||
# the Traefik access log will tell you when that is.
|
||||
- traefik.http.routers.searxng.rule=Host(`searxng.ana.internal`) || Host(`searxng.pfi.local`)
|
||||
- traefik.http.routers.searxng.entrypoints=websecure
|
||||
- traefik.http.routers.searxng.tls=true
|
||||
- traefik.http.routers.searxng.service=searxng
|
||||
- traefik.http.services.searxng.loadbalancer.server.port=8080
|
||||
- homepage.group=Daily
|
||||
- homepage.name=SearXNG
|
||||
- homepage.icon=si-searxng
|
||||
- homepage.description=Privacy-respecting meta-search
|
||||
- homepage.href=http://10.250.50.70:9996
|
||||
- homepage.href=http://10.100.50.40:9996
|
||||
|
||||
networks:
|
||||
tnet:
|
||||
name: traefik-net
|
||||
|
||||
@@ -1,73 +1,79 @@
|
||||
# =============================================================================
|
||||
# SearXNG Custom Settings — overrides defaults from the container image
|
||||
# Full reference: https://docs.searxng.org/admin/settings/index.html
|
||||
# =============================================================================
|
||||
# SearXNG — PFI fleet meta-search. Deployed on nh3-docker (10.100.50.40:9996).
|
||||
#
|
||||
# ⚠ WHY NH3 AND NOT THE COLO. Measured 2026-09-03:
|
||||
# ana-docker egress 38.120.12.42 (datacenter) -> DuckDuckGo + Startpage CAPTCHA
|
||||
# nh3-docker egress 70.230.226.88 (residential) -> no CAPTCHA
|
||||
# Search engines gate datacenter ranges. Same reason the fleet keeps a
|
||||
# residential SOCKS5 egress proxy on nh3-dev for yt-dlp. Running the search
|
||||
# aggregator from a residential-egress site removes the problem at the source
|
||||
# rather than proxying around it.
|
||||
|
||||
use_default_settings:
|
||||
engines:
|
||||
remove:
|
||||
- wikidata
|
||||
# Onion engines: no Tor proxy is configured here, so they only ever
|
||||
# contribute timeouts.
|
||||
- ahmia
|
||||
- torch
|
||||
# ⚠ Removal keys must match the engine's REAL name, spaces and all.
|
||||
# `karmasearch.videos` (dotted) did NOT match on the old instance and the
|
||||
# engine kept appearing in unresponsive_engines despite being "removed".
|
||||
# The name is "karmasearch videos".
|
||||
- karmasearch
|
||||
- karmasearch.videos
|
||||
- brave
|
||||
- brave.images
|
||||
- brave.news
|
||||
- brave.videos
|
||||
- karmasearch videos
|
||||
|
||||
general:
|
||||
instance_name: "SearXNG"
|
||||
instance_about_url: false
|
||||
contact_url: false
|
||||
debug: false
|
||||
# Disable public metrics page (/stats/errors) to reduce attack surface
|
||||
# Public metrics page off — smaller attack surface on an unauthenticated
|
||||
# internal service.
|
||||
enable_metrics: false
|
||||
|
||||
search:
|
||||
safe_search: 0
|
||||
# "" disables; "duckduckgo" is the most private working option
|
||||
autocomplete: ""
|
||||
default_lang: "auto"
|
||||
# `json` is what makes this usable as a tool rather than only a web page.
|
||||
# Removing it breaks every non-browser consumer, including Claude sessions.
|
||||
formats:
|
||||
- html
|
||||
- json
|
||||
# 3s is too tight; 8s covers slower engines without hanging the UI
|
||||
# 3s is too tight for slower engines; 8s covers them without hanging the UI.
|
||||
request_timeout: 8.0
|
||||
# Ban time after an engine raises a suspended-time exception (default 86400)
|
||||
# Ban an engine only briefly when it raises suspended-time. The default 86400
|
||||
# means one bad afternoon silences an engine for a day.
|
||||
ban_time_on_fail: 60
|
||||
max_ban_time_on_fail: 600
|
||||
|
||||
server:
|
||||
# REQUIRED. Generate with: openssl rand -hex 32
|
||||
# Prefer setting SEARXNG_SECRET in docker-compose and letting the entrypoint
|
||||
# substitute it; hardcoding a real secret here is a leak risk.
|
||||
#secret_key: "changeme_please_generate_a_secret"
|
||||
# secret_key comes from SEARXNG_SECRET in the environment — never hardcode it
|
||||
# here. Generated + vaulted at nh3-docker/searxng-secret.
|
||||
bind_address: "0.0.0.0"
|
||||
port: 8080
|
||||
# Enable ONLY if you ship a limiter.toml AND your proxy forwards X-Real-IP.
|
||||
# Otherwise you'll get "X-Forwarded-For nor X-Real-IP header is set!" noise.
|
||||
# Enable ONLY with a limiter.toml AND a proxy that forwards X-Real-IP;
|
||||
# otherwise it logs "X-Forwarded-For nor X-Real-IP header is set!" forever.
|
||||
limiter: false
|
||||
# Mark as true if instance is internet-facing; tightens some defaults.
|
||||
public_instance: false
|
||||
base_url: "https://searxng.pfi.local/"
|
||||
# Allow only GET to the search endpoint (simpler, works with most clients)
|
||||
# ⚠ Kept in sync with BASE_URL in compose.yaml. The old instance still said
|
||||
# `https://searxng.pfi.local/` here — a name retired on 2026-08-19 — while the
|
||||
# environment said something else. The env wins, so nothing broke, and the
|
||||
# file quietly lied to everyone who read it.
|
||||
base_url: "http://10.100.50.40:9996/"
|
||||
method: "GET"
|
||||
compression: true
|
||||
# Set to true if you need image_proxy rewriting for privacy
|
||||
image_proxy: false
|
||||
|
||||
# Outgoing HTTP pool — tuned for a low-traffic private instance.
|
||||
# Defaults are fine for most, but these reduce memory use and tighten timeouts.
|
||||
# Outgoing pool — low-traffic private instance.
|
||||
outgoing:
|
||||
request_timeout: 6.0
|
||||
max_request_timeout: 12.0
|
||||
pool_connections: 100
|
||||
pool_maxsize: 20
|
||||
enable_http2: true
|
||||
# Uncomment if you want to route outbound traffic via Tor for .onion engines
|
||||
# proxies:
|
||||
# all://:
|
||||
# - socks5h://tor:9050
|
||||
|
||||
|
||||
# No proxy needed: this host already egresses residentially (see header).
|
||||
# If that ever changes, the fleet's NH3 SOCKS5 proxy is the fallback:
|
||||
# proxies:
|
||||
# all://:
|
||||
# - socks5h://10.100.10.50:1080
|
||||
|
||||
Reference in New Issue
Block a user