# searxng Privacy-respecting metasearch. **Runs on nh3-docker** (`10.100.50.40:9996`, `searxng.nh3.internal`), and is exposed to every Claude Code session on nh3-dev as the `web_search` MCP tool. ## Current egress: direct from NH3 Hosting and egress are both nh3-docker. Outbound search requests leave from **70.230.226.88**, NH3's residential address, with no proxy in `outgoing:`. This is the configuration that works; the section below records why, and the one after it records an attempt to change it that failed. ## The ESH egress experiment (2026-09-17 → reverted 2026-09-18) Search egress was moved to `socks5h://10.0.50.65:1080` on **esh-scale** (CT 108 on esh-pve) — an application-level proxy, no host route or exit-node changes. **Reverted after one day: it cost three of the four engines.** Measured 2026-09-18 across five queries (two ad-hoc plus all three in `scripts/searxng-health.sh`), identical every time: brave Suspended: too many requests duckduckgo CAPTCHA startpage Suspended: CAPTCHA google cse 20 results <- the only engine answering The cause is the one this stack already knew about. SearXNG runs at NH3 *because* its residential egress avoids the CAPTCHA gating that gets applied to datacenter ranges; routing out through ESH (`128.177.138.182` when measured, `154.50.58.126` at cutover — ESH's WAN address moves) gave that property up. The Dependency note written at cutover called the risk and it materialised. ⚠ **The failure was invisible to both health checks.** `/healthz` cannot see it, and `scripts/searxng-health.sh` — the script this README told you to run instead — prints `✓ searching` in this exact state, because it gates on "did any results come back" and treats failed engines as informational. Search ran on one engine for a day with every instrument reporting healthy. If egress is ever changed again, fix the script to fail on blocked engines *first*, or the next regression is equally silent. The proxy side was built correctly and none of this was its fault: microsocks ran as `nobody` under `searxng-egress.service`, bound `10.0.50.65:1080` only, and allowed source `10.100.50.40` alone. The canonical unit is kept at [`configs/esh-scale/searxng-egress.service`](../../configs/esh-scale/searxng-egress.service) in case a future egress experiment wants it; the service is disabled on esh-scale. ## Historical placement: NH3 rather than the colo Measured 2026-09-03: ana-docker egress 38.120.12.42 datacenter -> DuckDuckGo/Startpage CAPTCHA nh3-docker egress 70.230.226.88 residential -> no gate on that account Search engines gate datacenter ranges. Running the aggregator from the residential-egress site was the original approach. The operator subsequently requested ESH egress; `outgoing.proxies` now explicitly selects it as above. ⚠ It is **not** a complete fix: `brave`, `duckduckgo` and `startpage` still CAPTCHA from here. `google cse` carries general search at ~20 results/query, and `yandex`, `wiby`, `github`, `stackoverflow` and `marginalia` all work. General search is therefore effectively single-engine — if `google cse` breaks, the instance goes quiet, which is exactly the failure below. ## ⚠ /healthz cannot tell you whether search works On 2026-09-03 the old ana-docker instance was found returning **zero results for every query**, for an unknown number of weeks, while: - the container reported `healthy` for 7 straight days, 0 restarts; - the Homepage card was green; - `/healthz` returned 200 every 30 seconds. It was running **2026.4.17 while current was 2026.9.3** — 4.5 months of engine scrapers rotting against sites that had changed their markup. `:latest` means "latest at pull time", and nothing re-pulls on its own. Proven by experiment before touching anything: the new image, same settings file, same host, same query returned **20 results where the running one returned 0**. **`scripts/searxng-health.sh` asserts the property, not the mechanism** — three unrelated queries must each return results > 0. Run it after any change, and periodically; it is the only thing that catches rot. ## Update ssh infra-ops@nh3-docker 'cd /opt/docker/compose/searxng && \ sudo docker compose pull && sudo docker compose up -d' scripts/searxng-health.sh ## The MCP tool `services/searxng-mcp/` — installed with `uv tool install` and registered at **user scope** (`claude mcp add --scope user searxng searxng-mcp`), so every Claude Code session gets `web_search` with no per-project setup. ⚠ **Zero results raise an error rather than returning an empty list.** An empty list is indistinguishable from a broken aggregator, and that ambiguity is what let the old instance fail silently for weeks. Same reasoning as althing's "an unreachable post office is an OUTAGE, never an empty inbox". ⚠ Written against **mcp 2.x**, where `FastMCP` became `MCPServer` and the v1 `@app.list_tools()` decorator API is gone. v1 examples fail at import with `'Server' object has no attribute 'list_tools'`. ⚠ `uv tool install --force` alone served a **cached build** and silently reinstalled the old code — the installed file still had the v1 API after the source no longer did. `--reinstall --no-cache` was required, and the check that caught it was `md5sum` of source vs installed. ## Secret `SEARXNG_SECRET` lives in `/opt/docker/compose/searxng/.env` (0600, root) on the host and is vaulted at `nh3-docker/searxng-secret`. Never in git.