feat(searxng): route search egress through the esh-scale SOCKS5 proxy
Committing work deployed on 2026-09-17 that had been left uncommitted, so canonical intent stops disagreeing with the running host. The deployed /opt/docker/conf/searxng/searxng-settings.yml is byte-identical to the canonical file here, verified before this commit. Search requests and their DNS now exit via socks5h://10.0.50.65:1080 on esh-scale (CT 108), an application-level proxy rather than a host-wide exit node; no route or firewall changes. microsocks runs as nobody under searxng-egress.service, binds only 10.0.50.65:1080, and bypasses SOCKS auth for source 10.100.50.40 alone — every other source must supply a password regenerated at each start and never distributed. Verified active and enabled. There is deliberately no direct-NH3 fallback: an ESH outage must fail the search rather than silently revert egress. ⚠ THE CHANGE HAS NOT ACHIEVED ITS PURPOSE AS DEPLOYED. Two independent live queries, 2026-09-18, both report brave "Suspended: too many requests", duckduckgo "CAPTCHA" and startpage "Suspended: CAPTCHA", leaving google cse as the only answering engine. Moving egress off NH3's residential address is what this change did, and CAPTCHA avoidance was the stated reason searxng sits at NH3 at all. The README anticipated the risk in its Dependency note; it has materialised. Rollback procedure is in the README and the pre-change config is kept on the host as searxng-settings.yml.pre-esh-20260917. Measured egress also drifted from the value recorded at cutover: the README notes 154.50.58.126, the proxy now exits 128.177.138.182. Expected — the README pins no public IP and calls out WAN failover — but recorded here so the number in the doc is not mistaken for current. Also retargets seat-inventory.py's default host from the mesh address 100.64.0.7 to fv-ml1's LAN address 10.251.50.54, routed by the site gateway.
This commit is contained in:
@@ -0,0 +1,22 @@
|
||||
[Unit]
|
||||
Description=SearXNG-only SOCKS5 egress via ESH
|
||||
After=network-online.target tailscaled.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=nobody
|
||||
Group=nogroup
|
||||
# -w bypasses authentication ONLY for nh3-docker. Everyone else must provide
|
||||
# an unknown, freshly randomized password (never stored or distributed).
|
||||
ExecStart=/bin/sh -ec 'exec /usr/bin/microsocks -i 10.0.50.65 -p 1080 -b 10.0.50.65 -w 10.100.50.40 -u denied -P "$$(cat /proc/sys/kernel/random/uuid)"'
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
RestrictAddressFamilies=AF_INET AF_UNIX
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -16,7 +16,7 @@ containers cannot.
|
||||
"""
|
||||
import argparse, json, re, subprocess, sys, datetime, pathlib
|
||||
|
||||
DEFAULT_HOST = "100.64.0.7" # fv-ml1 over the mesh
|
||||
DEFAULT_HOST = "10.251.50.54" # fv-ml1 (LAN addr; routed by vb-gateway over the mesh)
|
||||
DEFAULT_OUT = "docs/pfi/fv-ml1-gpu-seat-inventory.md"
|
||||
GATEWAY = "10.250.50.70" # LiteLLM, for alias resolution
|
||||
|
||||
|
||||
@@ -4,7 +4,37 @@ Privacy-respecting metasearch. **Runs on nh3-docker** (`10.100.50.40:9996`,
|
||||
`searxng.nh3.internal`), and is exposed to every Claude Code session on nh3-dev
|
||||
as the `web_search` MCP tool.
|
||||
|
||||
## ⚠ Why NH3 and not the colo
|
||||
## Current egress: ESH (2026-09-17)
|
||||
|
||||
Hosting remains on nh3-docker; outbound search requests and their DNS resolution
|
||||
use `socks5h://10.0.50.65:1080` on **esh-scale**, the ESH exit-node router
|
||||
(CT 108 on esh-pve, `root@10.0.250.35`). This is an application proxy, not a
|
||||
host-wide Tailscale exit-node selection. No host/subnet routes were changed.
|
||||
The application network layer measured ESH egress **154.50.58.126** at cutover;
|
||||
nh3-docker's direct egress remains **70.230.226.88**. ESH's public IP can change
|
||||
with WAN failover; no public IP is pinned in this configuration.
|
||||
|
||||
The Debian `microsocks` package runs as `nobody` under
|
||||
`searxng-egress.service`, enabled at boot. The canonical unit is
|
||||
[`configs/esh-scale/searxng-egress.service`](../../configs/esh-scale/searxng-egress.service).
|
||||
It binds only `10.0.50.65:1080`. Only source `10.100.50.40` bypasses SOCKS
|
||||
authentication; other sources require a random, undistributed password regenerated
|
||||
at each start. Allowed-host egress and denied-host rejection were both tested.
|
||||
No gateway port-forward or firewall/routing changes were made.
|
||||
|
||||
**Dependency:** ESH WAN/mesh or proxy outages now break outbound search. There is
|
||||
no automatic direct-NH3 fallback. `/healthz` alone cannot detect this; run the
|
||||
three-query health script below. Engine CAPTCHAs may persist despite changing IP.
|
||||
|
||||
**Rollback:** on nh3-docker restore
|
||||
`/opt/docker/conf/searxng/searxng-settings.yml.pre-esh-20260917` over
|
||||
`/opt/docker/conf/searxng/searxng-settings.yml`, then `sudo docker restart searxng`
|
||||
and run `scripts/searxng-health.sh`. Remove `outgoing.proxies` in the canonical
|
||||
settings too. Once no longer used, disable the dedicated proxy via
|
||||
`ssh root@10.0.250.35 'pct exec 108 -- systemctl disable --now searxng-egress.service'`.
|
||||
This does not stop Tailscale or alter ESH subnet routing.
|
||||
|
||||
## Historical placement: NH3 rather than the colo
|
||||
|
||||
Measured 2026-09-03:
|
||||
|
||||
@@ -12,10 +42,8 @@ Measured 2026-09-03:
|
||||
nh3-docker egress 70.230.226.88 residential -> no gate on that account
|
||||
|
||||
Search engines gate datacenter ranges. Running the aggregator from the
|
||||
residential-egress site removes the problem at the source rather than proxying
|
||||
around it — the same reason the fleet keeps a residential SOCKS5 proxy on
|
||||
nh3-dev for yt-dlp. If NH3's egress ever changes, `outgoing.proxies` in
|
||||
`conf/searxng-settings.yml` has the fallback commented in place.
|
||||
residential-egress site was the original approach. The operator subsequently
|
||||
requested ESH egress; `outgoing.proxies` now explicitly selects it as above.
|
||||
|
||||
⚠ It is **not** a complete fix: `brave`, `duckduckgo` and `startpage` still
|
||||
CAPTCHA from here. `google cse` carries general search at ~20 results/query, and
|
||||
|
||||
@@ -3,10 +3,9 @@
|
||||
# ⚠ WHY NH3 AND NOT THE COLO. Measured 2026-09-03:
|
||||
# ana-docker egress 38.120.12.42 (datacenter) -> DuckDuckGo + Startpage CAPTCHA
|
||||
# nh3-docker egress 70.230.226.88 (residential) -> no CAPTCHA
|
||||
# Search engines gate datacenter ranges. Same reason the fleet keeps a
|
||||
# residential SOCKS5 egress proxy on nh3-dev for yt-dlp. Running the search
|
||||
# aggregator from a residential-egress site removes the problem at the source
|
||||
# rather than proxying around it.
|
||||
# Since 2026-09-17, search requests exit via a restricted SOCKS5 listener on
|
||||
# esh-scale (10.0.50.65:1080), per operator request. Hosting remains at NH3.
|
||||
# No host default-route or mesh routing changes. See stacks/searxng/README.md.
|
||||
|
||||
use_default_settings:
|
||||
engines:
|
||||
@@ -72,8 +71,8 @@ outgoing:
|
||||
pool_connections: 100
|
||||
pool_maxsize: 20
|
||||
enable_http2: true
|
||||
# No proxy needed: this host already egresses residentially (see header).
|
||||
# If that ever changes, the fleet's NH3 SOCKS5 proxy is the fallback:
|
||||
# proxies:
|
||||
# all://:
|
||||
# - socks5h://10.100.10.50:1080
|
||||
# ESH-only search egress; resolve engine hostnames at the proxy.
|
||||
# No direct fallback: an ESH outage must not silently switch back to NH3.
|
||||
proxies:
|
||||
all://:
|
||||
- socks5h://10.0.50.65:1080
|
||||
|
||||
Reference in New Issue
Block a user