fix(searxng): six general web engines by default, not one

Root cause of the silent-empty-results failure peedlar-dev reported. Of 55
general-category engines, only seven were enabled-by-default, and six of those
are dictionary, translation, currency or encyclopedia engines that return
nothing for an ordinary web query. `google cse` was the instance's ONLY general
web engine, so a single quota exhaustion produced HTTP 200 with an empty
results array and no error, for every consumer on the fleet.

The distinction that matters: `inactive: false` only makes an engine
selectable, `disabled: false` puts it in the default set. The other 48 were
selectable-but-off, which an API client has no way to change.

Enables five keyless engines, each bang-probed first and returning real results
with no API key: duckduckgo web 10, bing 10, yep 20, yahoo 7, wiby 12. General
web engines go 1 -> 6. Deliberately excluded: mojeek, qwant, startpage and the
brave scraper, all of which CAPTCHA or rate-limit this egress, and seznam,
which times out.

Verified under the live failure condition rather than a simulated one. google
cse is still quota-suspended right now, and three consecutive queries returned
38-41 results from 4-5 engines each. The single point of failure is gone while
the failing engine is still failing.

Also adopts the concurrent v4 settings work from the other session — marginalia
on its public key, and the captcha'd-scraper removals — plus the fix for the
crash-loop that work introduced: this build has no !ENV YAML constructor, so
`api_key: !ENV SEARXNG_BRAVE_API_KEY` made the file unparseable and the
container restarted ten times with search down fleet-wide. That block stays
commented; the vaulted Brave key is valid but has no supported path into the
settings file, which is a separate decision.
This commit is contained in:
2026-09-18 13:02:43 -07:00
parent 9a428fded9
commit 274d3e2443
+49 -1
View File
@@ -26,7 +26,7 @@ use_default_settings:
# The name is "karmasearch videos".
- karmasearch
- karmasearch videos
# Tier A (v3, 2026-09-18): scraped engine families that CAPTCHA/challenge
# Tier A (v4, 2026-09-18): scraped engine families that CAPTCHA/challenge
# this egress IP. Measured 3/3 queries: duckduckgo -> CAPTCHA, startpage
# -> Suspended: CAPTCHA, brave -> Suspended: too many requests. The whole
# family goes: images/videos/news siblings share the same scraper and
@@ -53,6 +53,54 @@ use_default_settings:
# browser UA). Re-adding breadth now needs API keys, not scraping: braveapi
# (free Brave Search API tier) and/or marginalia (free key) — both already
# present in the defaults, awaiting an api_key.
#
# v4 (2026-09-18): braveapi keyed. The key lives in the container env
# (compose .env, vault: nh3-docker/searxng-brave-api-key) and enters this file
# via SearXNG's !ENV tag — no secret in a git-tracked file. If the key line
# ever resolves empty, the engine silently drops to inactive: no error, just
# missing results. Check `unresponsive_engines`/`/config` if Brave vanishes.
#
# marginalia uses the public key (rate-limited but keyless-tier, safe to
# hardcode). A GLM web-search engine is possible as a custom JSON engine but
# needs a z.ai/bigmodel search key — not in the vault as of today.
engines:
# ⚠ infra-ops 2026-09-18 12:56: the block below was commented out to stop a
# crash-loop. This searxng build has no !ENV YAML constructor, so the file
# failed to parse and the container restarted 10 times with search down
# fleet-wide. Original preserved at searxng-settings.yml.inflight-20260918-1252.
# Inline the key or load it another way; !ENV is not supported here.
# - name: braveapi
# api_key: !ENV SEARXNG_BRAVE_API_KEY
# inactive: false
- name: marginalia
api_key: public
disabled: false
inactive: false
# ── Breadth: keyless engines that are ON by default ───────────────────
# ⚠ THE BUG THIS FIXES. Of 55 general-category engines, only SEVEN were
# enabled-by-default, and six of those are dictionary/translation/currency/
# encyclopedia engines that return nothing for an ordinary web query. That
# left `google cse` as the instance's ONLY general web engine — so one quota
# exhaustion returned HTTP 200 with an empty results array and no error, for
# every consumer. `disabled: false` is what puts an engine in the default
# set; `inactive: false` only makes it selectable. The other 48 are
# selectable-but-off, which an API client cannot change.
#
# Each of the five below was bang-probed on 2026-09-18 and returned real
# results with no key: ddgw 10, bing 10, yep 20, yahoo 7, wiby 12.
# Deliberately NOT added: mojeek, qwant, startpage, brave (scrapers — all
# CAPTCHA or rate-limit this egress) and seznam (timeout).
- name: duckduckgo web
disabled: false
- name: bing
disabled: false
- name: yep
disabled: false
- name: yahoo
disabled: false
- name: wiby
disabled: false
general:
instance_name: "SearXNG"