feat(searxng): move to nh3-docker, update, and expose as an MCP tool
The ana-docker instance was returning zero results for every query while reporting healthy — 4.5 months stale (2026.4.17 against a current 2026.9.3), its engine scrapers rotted against sites that had changed. /healthz proves the web app answers and says nothing about whether search works, so seven days of green sat on top of a search box that found nothing. Moved to nh3-docker rather than updated in place, because the colo egress is the other half of the problem: 38.120.12.42 is a datacenter address that DuckDuckGo and Startpage CAPTCHA, while nh3-docker egresses residentially at 70.230.226.88. Same reasoning as the fleet's residential proxy for yt-dlp, applied at the source instead of around it. Config corrected along the way: base_url said searxng.pfi.local, a name retired on 2026-08-19, while the environment said something else — the env won so nothing broke and the file quietly lied. The karmasearch.videos removal key never matched, because the engine's real name has a space. scripts/searxng-health.sh asserts results > 0 across three unrelated queries. That is the check that would have caught this, and the only kind that can: the mechanism was healthy throughout. services/searxng-mcp exposes it as `web_search` at user scope, so every Claude Code session has it. Zero results raise rather than returning an empty list — an empty list is indistinguishable from a broken aggregator, which is precisely how this hid. Old instance stopped and removed; DNS alias repointed to searxng.nh3.internal.
This commit is contained in:
@@ -0,0 +1,13 @@
|
||||
[project]
|
||||
name = "searxng-mcp"
|
||||
version = "0.1.0"
|
||||
description = "MCP server exposing the PFI fleet's SearXNG instance as a Claude Code tool"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = ["mcp>=1.2.0", "httpx>=0.27"]
|
||||
|
||||
[project.scripts]
|
||||
searxng-mcp = "searxng_mcp.server:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
@@ -0,0 +1,117 @@
|
||||
"""MCP server exposing the fleet's SearXNG instance as a Claude Code tool.
|
||||
|
||||
Registered at USER scope so every Claude Code session on the box has it without
|
||||
per-project setup. The instance lives on nh3-docker (10.100.50.40:9996) —
|
||||
deliberately at NH3 rather than the colo, because search engines gate
|
||||
datacenter IP ranges and the NH3 site egresses residentially.
|
||||
|
||||
⚠ WRITTEN AGAINST mcp 2.x. In 2.x `FastMCP` was renamed `MCPServer` and the
|
||||
v1 decorator API (`@app.list_tools()` on `mcp.server.Server`) no longer exists —
|
||||
it fails at import with `'Server' object has no attribute 'list_tools'`. If you
|
||||
are copying an older MCP example, that is why it will not start. Pin `mcp<2`
|
||||
only if you intend to keep v1 code.
|
||||
|
||||
⚠ WHY A ZERO-RESULT SEARCH RAISES RATHER THAN RETURNING AN EMPTY LIST.
|
||||
On 2026-09-03 the previous instance was found returning zero results for every
|
||||
query — for an unknown number of weeks — while its container reported `healthy`
|
||||
and its dashboard card was green. SearXNG's /healthz proves the web app answers
|
||||
and says nothing about whether any engine works. An empty result list is
|
||||
indistinguishable from a broken aggregator, so zero results surface as a
|
||||
failure with the engine errors attached, never as a quiet "no matches".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import httpx
|
||||
from mcp.server.mcpserver import MCPServer
|
||||
|
||||
SEARXNG_URL = os.environ.get("SEARXNG_URL", "http://10.100.50.40:9996")
|
||||
TIMEOUT = float(os.environ.get("SEARXNG_TIMEOUT", "45"))
|
||||
|
||||
mcp = MCPServer(
|
||||
name="searxng",
|
||||
instructions=(
|
||||
"Private web search via the PFI fleet's SearXNG metasearch instance. "
|
||||
"Use for current information, documentation lookups, and anything past "
|
||||
"the training cutoff."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
name="web_search",
|
||||
description=(
|
||||
"Search the web through the PFI fleet's private SearXNG metasearch "
|
||||
"instance. Returns titles, URLs and snippets, with the engine that "
|
||||
"produced each result. Supports SearXNG bang syntax to target one "
|
||||
"engine, e.g. '!github asyncio' or '!stackoverflow mmap'."
|
||||
),
|
||||
)
|
||||
async def web_search(
|
||||
query: str,
|
||||
categories: str | None = None,
|
||||
max_results: int = 10,
|
||||
) -> str:
|
||||
"""Search the web.
|
||||
|
||||
Args:
|
||||
query: The search query. Supports bang syntax such as '!github ...'.
|
||||
categories: Optional comma-separated SearXNG categories, e.g.
|
||||
'general', 'it', 'science', 'news'.
|
||||
max_results: Maximum number of results to return.
|
||||
"""
|
||||
query = (query or "").strip()
|
||||
if not query:
|
||||
raise ValueError("query must not be empty")
|
||||
|
||||
params: dict[str, str] = {"q": query, "format": "json"}
|
||||
if categories:
|
||||
params["categories"] = categories
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=TIMEOUT) as client:
|
||||
resp = await client.get(f"{SEARXNG_URL}/search", params=params)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except httpx.HTTPError as exc:
|
||||
# An unreachable instance is an OUTAGE, not an absence of results, and
|
||||
# the two must never render the same way to the caller.
|
||||
raise RuntimeError(
|
||||
f"SearXNG at {SEARXNG_URL} could not be reached: {exc}. "
|
||||
"This is an outage, not an empty result set."
|
||||
) from exc
|
||||
|
||||
results = data.get("results") or []
|
||||
errors = data.get("unresponsive_engines") or []
|
||||
|
||||
if not results:
|
||||
raise RuntimeError(
|
||||
f"SearXNG returned no results for {query!r}. Unresponsive engines: "
|
||||
f"{errors or 'none reported'}. Zero results across every engine "
|
||||
"usually means the instance is stale — SearXNG engine scrapers rot "
|
||||
"as upstream sites change their markup — rather than that the query "
|
||||
"has no matches. Check scripts/searxng-health.sh in eshpfi."
|
||||
)
|
||||
|
||||
lines = [f"{len(results)} results for {query!r} via {SEARXNG_URL}"]
|
||||
if errors:
|
||||
lines.append(f"(engines that failed: {', '.join(e[0] for e in errors)})")
|
||||
lines.append("")
|
||||
for i, r in enumerate(results[:max_results], 1):
|
||||
lines.append(f"{i}. {r.get('title', '(untitled)')}")
|
||||
lines.append(f" {r.get('url', '')}")
|
||||
if r.get("content"):
|
||||
lines.append(f" {r['content'].strip()[:300]}")
|
||||
lines.append(f" [engine: {r.get('engine', '?')}]")
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
mcp.run(transport="stdio")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user