3.6 KiB
[2026-09-03]SearXNG was returning ZERO results for every query while reportinghealthy— moved to nh3-docker, updated, and exposed to every CC session as an MCP tool.
The failure. The ana-docker instance answered /healthz every 30s, showed
Up 7 days (healthy) with 0 restarts, and had a green Homepage card — while
returning 0 results for every query tested. It was running 2026.4.17
against a current 2026.9.3: 4.5 months of engine scrapers rotting against
sites that had changed their markup. SearXNG ships near-daily releases for
exactly this reason.
⚠ :latest means "latest AT PULL TIME". Nothing re-pulls on its own. A
container created in April on :latest is pinned to April forever.
⚠ /healthz proves the web app answers and says NOTHING about whether search
works. That is the whole lesson. Same shape as the nh3-dev "failing disk" that
was a stalled backup, the runner audit that trusted liveness for identity, and
the statusline bell that measured a mechanism.
Proven before acting: the new image, same settings file, same host, same query, in a throwaway container → 20 results where the running one returned 0. Network was ruled out first — from inside the container DNS resolved and mojeek/wikipedia were reachable, so engines were reachable and the parsers were the broken part.
Why NH3 and not an in-place update (operator's call, and the measurement backs it):
ana-docker egress 38.120.12.42 datacenter -> DuckDuckGo/Startpage CAPTCHA
nh3-docker egress 70.230.226.88 residential -> not gated the same way
Search engines gate datacenter ranges. Same reason the fleet keeps a residential
SOCKS5 proxy on nh3-dev for yt-dlp — applied at the source instead of proxied
around. outgoing.proxies has the fallback commented in place if NH3's egress
ever changes.
⚠ Not a complete fix. brave, duckduckgo, startpage still CAPTCHA from
NH3. google cse carries general search at ~20 results/query; yandex, wiby,
github, stackoverflow, marginalia work. General search is effectively
single-engine — if google cse breaks, it goes quiet again.
Two config defects, both silent: base_url still named
searxng.pfi.local, retired 2026-08-19, while the env said otherwise (env wins,
so nothing broke and the file lied to every reader); and the
karmasearch.videos removal key never matched because the engine's real name
has a space in it.
scripts/searxng-health.sh asserts results > 0 across three unrelated
queries. That is the only check that could have caught this — the mechanism was
healthy throughout.
The MCP tool — services/searxng-mcp, uv tool install, registered
claude mcp add --scope user searxng searxng-mcp, so every CC session gets
web_search. ⚠ Zero results raise rather than returning an empty list: an
empty list is indistinguishable from a broken aggregator, which is precisely how
this hid. Same principle as althing's "unreachable post office is an OUTAGE,
never an empty inbox".
⚠ Written against mcp 2.x (FastMCP → MCPServer; the v1
@app.list_tools() decorator is gone and fails at import). ⚠ uv tool install --force served a CACHED build and silently reinstalled the old code — the
installed file still had the v1 API after the source no longer did.
--reinstall --no-cache fixed it; md5sum of source vs installed is what
caught it.
Old instance stopped and removed; DNS alias repointed to
searxng.nh3.internal → 10.100.50.40. Secret vaulted at
nh3-docker/searxng-secret. Commit 0f748ea. See 2026-09-03-gx10-rack-network
for the other UniFi-side change the same day.