This commit is contained in:
hexun 2026-01-31 00:11:19 +08:00 committed by GitHub
commit b8147285a1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 488 additions and 44 deletions

View File

@ -31,6 +31,7 @@ Status: stable.
- Matrix: switch plugin SDK to @vector-im/matrix-bot-sdk. - Matrix: switch plugin SDK to @vector-im/matrix-bot-sdk.
- Tlon: format thread reply IDs as @ud. (#1837) Thanks @wca4a. - Tlon: format thread reply IDs as @ud. (#1837) Thanks @wca4a.
- Tools: add per-sender group tool policies and fix precedence. (#1757) Thanks @adam91holt. - Tools: add per-sender group tool policies and fix precedence. (#1757) Thanks @adam91holt.
- Tools: add SearXNG as a `web_search` provider. (#2317)
- Agents: summarize dropped messages during compaction safeguard pruning. (#2509) Thanks @jogi47. - Agents: summarize dropped messages during compaction safeguard pruning. (#2509) Thanks @jogi47.
- Agents: expand cron tool description with full schema docs. (#1988) Thanks @tomascupr. - Agents: expand cron tool description with full schema docs. (#1988) Thanks @tomascupr.
- Agents: honor tools.exec.safeBins in exec allowlist checks. (#2281) - Agents: honor tools.exec.safeBins in exec allowlist checks. (#2281)

View File

@ -1942,8 +1942,16 @@ of `every`, keep `HEARTBEAT.md` tiny, and/or choose a cheaper `model`.
Note: `applyPatch` is only under `tools.exec`. Note: `applyPatch` is only under `tools.exec`.
`tools.web` configures web search + fetch tools: `tools.web` configures web search + fetch tools:
- `tools.web.search.enabled` (default: true when key is present) - `tools.web.search.enabled` (default true)
- `tools.web.search.apiKey` (recommended: set via `openclaw configure --section web`, or use `BRAVE_API_KEY` env var) - `tools.web.search.provider` (`brave` | `perplexity` | `searxng`)
- `tools.web.search.apiKey` (Brave; recommended: set via `openclaw configure --section web`, or use `BRAVE_API_KEY` env var)
- `tools.web.search.perplexity.apiKey` (Perplexity/OpenRouter; optional if `PERPLEXITY_API_KEY` or `OPENROUTER_API_KEY` is set)
- `tools.web.search.perplexity.baseUrl` (optional override)
- `tools.web.search.perplexity.model` (optional override)
- `tools.web.search.searxng.baseUrl` (SearXNG; optional if `SEARXNG_BASE_URL` is set)
- `tools.web.search.searxng.apiKey` (optional auth token; sent as Authorization header)
- `tools.web.search.searxng.headers` (optional extra headers)
- `tools.web.search.searxng.params` (optional default query params)
- `tools.web.search.maxResults` (110, default 5) - `tools.web.search.maxResults` (110, default 5)
- `tools.web.search.timeoutSeconds` (default 30) - `tools.web.search.timeoutSeconds` (default 30)
- `tools.web.search.cacheTtlMinutes` (default 15) - `tools.web.search.cacheTtlMinutes` (default 15)

View File

@ -1319,10 +1319,13 @@ The Gateway watches the config and supports hotreload:
### How do I enable web search and web fetch ### How do I enable web search and web fetch
`web_fetch` works without an API key. `web_search` requires a Brave Search API `web_fetch` works without an API key. `web_search` requires provider setup:
key. **Recommended:** run `openclaw configure --section web` to store it in
`tools.web.search.apiKey`. Environment alternative: set `BRAVE_API_KEY` for the - Brave: `tools.web.search.apiKey` or `BRAVE_API_KEY`
Gateway process. - Perplexity: `tools.web.search.perplexity.apiKey` or `PERPLEXITY_API_KEY` / `OPENROUTER_API_KEY`
- SearXNG: `tools.web.search.searxng.baseUrl` or `SEARXNG_BASE_URL`
**Recommended:** run `openclaw configure --section web` to set these in config.
```json5 ```json5
{ {
@ -1330,6 +1333,7 @@ Gateway process.
web: { web: {
search: { search: {
enabled: true, enabled: true,
provider: "brave",
apiKey: "BRAVE_API_KEY_HERE", apiKey: "BRAVE_API_KEY_HERE",
maxResults: 5 maxResults: 5
}, },

View File

@ -62,11 +62,12 @@ You can keep it local with `memorySearch.provider = "local"` (no API usage).
See [Memory](/concepts/memory). See [Memory](/concepts/memory).
### 4) Web search tool (Brave / Perplexity via OpenRouter) ### 4) Web search tool (Brave / Perplexity / SearXNG)
`web_search` uses API keys and may incur usage charges: `web_search` uses API keys and may incur usage charges:
- **Brave Search API**: `BRAVE_API_KEY` or `tools.web.search.apiKey` - **Brave Search API**: `BRAVE_API_KEY` or `tools.web.search.apiKey`
- **Perplexity** (via OpenRouter): `PERPLEXITY_API_KEY` or `OPENROUTER_API_KEY` - **Perplexity** (via OpenRouter): `PERPLEXITY_API_KEY` or `OPENROUTER_API_KEY`
- **SearXNG** (self-hosted): `tools.web.search.searxng.baseUrl` or `SEARXNG_BASE_URL` (no external API billing; depends on your instance)
**Brave free tier (generous):** **Brave free tier (generous):**
- **2,000 requests/month** - **2,000 requests/month**

View File

@ -44,9 +44,8 @@ run on host, set an explicit per-agent override:
- Node `>=22` - Node `>=22`
- `pnpm` (optional; recommended if you build from source) - `pnpm` (optional; recommended if you build from source)
- **Recommended:** Brave Search API key for web search. Easiest path: - **Recommended:** configure a `web_search` provider (Brave, Perplexity, or SearXNG) for web search.
`openclaw configure --section web` (stores `tools.web.search.apiKey`). Easiest path: `openclaw configure --section web`. See [Web tools](/tools/web).
See [Web tools](/tools/web).
macOS: if you plan to build the apps, install Xcode / CLT. For the CLI + gateway only, Node is enough. macOS: if you plan to build the apps, install Xcode / CLT. For the CLI + gateway only, Node is enough.
Windows: use **WSL2** (Ubuntu recommended). WSL2 is strongly recommended; native Windows is untested, more problematic, and has poorer tool compatibility. Install WSL2 first, then run the Linux steps inside WSL. See [Windows (WSL2)](/platforms/windows). Windows: use **WSL2** (Ubuntu recommended). WSL2 is strongly recommended; native Windows is untested, more problematic, and has poorer tool compatibility. Install WSL2 first, then run the Linux steps inside WSL. See [Windows (WSL2)](/platforms/windows).

View File

@ -27,9 +27,9 @@ Followup reconfiguration:
openclaw configure openclaw configure
``` ```
Recommended: set up a Brave Search API key so the agent can use `web_search` Recommended: set up a `web_search` provider (Brave, Perplexity, or SearXNG) so the
(`web_fetch` works without a key). Easiest path: `openclaw configure --section web` agent can search the web when needed (`web_fetch` works without a key). Easiest
which stores `tools.web.search.apiKey`. Docs: [Web tools](/tools/web). path: `openclaw configure --section web`. Docs: [Web tools](/tools/web).
## QuickStart vs Advanced ## QuickStart vs Advanced

View File

@ -205,14 +205,14 @@ Notes:
- `process` is scoped per agent; sessions from other agents are not visible. - `process` is scoped per agent; sessions from other agents are not visible.
### `web_search` ### `web_search`
Search the web using Brave Search API. Search the web using your configured provider (Brave, Perplexity, or SearXNG).
Core parameters: Core parameters:
- `query` (required) - `query` (required)
- `count` (110; default from `tools.web.search.maxResults`) - `count` (110; default from `tools.web.search.maxResults`)
Notes: Notes:
- Requires a Brave API key (recommended: `openclaw configure --section web`, or set `BRAVE_API_KEY`). - Requires provider setup (recommended: `openclaw configure --section web`).
- Enable via `tools.web.search.enabled`. - Enable via `tools.web.search.enabled`.
- Responses are cached (default 15 min). - Responses are cached (default 15 min).
- See [Web tools](/tools/web) for setup. - See [Web tools](/tools/web) for setup.

View File

@ -1,16 +1,17 @@
--- ---
summary: "Web search + fetch tools (Brave Search API, Perplexity direct/OpenRouter)" summary: "Web search + fetch tools (Brave Search API, Perplexity direct/OpenRouter, SearXNG)"
read_when: read_when:
- You want to enable web_search or web_fetch - You want to enable web_search or web_fetch
- You need Brave Search API key setup - You need Brave Search API key setup
- You want to use Perplexity Sonar for web search - You want to use Perplexity Sonar for web search
- You want to use SearXNG for web search
--- ---
# Web tools # Web tools
OpenClaw ships two lightweight web tools: OpenClaw ships two lightweight web tools:
- `web_search` — Search the web via Brave Search API (default) or Perplexity Sonar (direct or via OpenRouter). - `web_search` — Search the web via Brave Search API (default), Perplexity Sonar (direct or via OpenRouter), or a self-hosted SearXNG instance.
- `web_fetch` — HTTP fetch + readable extraction (HTML → markdown/text). - `web_fetch` — HTTP fetch + readable extraction (HTML → markdown/text).
These are **not** browser automation. For JS-heavy sites or logins, use the These are **not** browser automation. For JS-heavy sites or logins, use the
@ -21,6 +22,7 @@ These are **not** browser automation. For JS-heavy sites or logins, use the
- `web_search` calls your configured provider and returns results. - `web_search` calls your configured provider and returns results.
- **Brave** (default): returns structured results (title, URL, snippet). - **Brave** (default): returns structured results (title, URL, snippet).
- **Perplexity**: returns AI-synthesized answers with citations from real-time web search. - **Perplexity**: returns AI-synthesized answers with citations from real-time web search.
- **SearXNG**: returns structured results (title, URL, snippet) from your own meta-search instance.
- Results are cached by query for 15 minutes (configurable). - Results are cached by query for 15 minutes (configurable).
- `web_fetch` does a plain HTTP GET and extracts readable content - `web_fetch` does a plain HTTP GET and extracts readable content
(HTML → markdown/text). It does **not** execute JavaScript. (HTML → markdown/text). It does **not** execute JavaScript.
@ -32,6 +34,7 @@ These are **not** browser automation. For JS-heavy sites or logins, use the
|----------|------|------|---------| |----------|------|------|---------|
| **Brave** (default) | Fast, structured results, free tier | Traditional search results | `BRAVE_API_KEY` | | **Brave** (default) | Fast, structured results, free tier | Traditional search results | `BRAVE_API_KEY` |
| **Perplexity** | AI-synthesized answers, citations, real-time | Requires Perplexity or OpenRouter access | `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` | | **Perplexity** | AI-synthesized answers, citations, real-time | Requires Perplexity or OpenRouter access | `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` |
| **SearXNG** | Self-hosted, works in private environments, flexible | Requires running SearXNG and enabling JSON format | None (needs base URL) |
See [Brave Search setup](/brave-search) and [Perplexity Sonar](/perplexity) for provider-specific details. See [Brave Search setup](/brave-search) and [Perplexity Sonar](/perplexity) for provider-specific details.
@ -42,7 +45,7 @@ Set the provider in config:
tools: { tools: {
web: { web: {
search: { search: {
provider: "brave" // or "perplexity" provider: "brave" // or "perplexity" or "searxng"
} }
} }
} }
@ -68,6 +71,34 @@ Example: switch to Perplexity Sonar (direct API):
} }
``` ```
## Using SearXNG
SearXNG exposes a simple HTTP API at `/search`. To get JSON results you need
`format=json` enabled on your instance; some public instances disable it and will
return `403 Forbidden`.
Example config:
```json5
{
tools: {
web: {
search: {
enabled: true,
provider: "searxng",
searxng: {
baseUrl: "http://localhost:8080",
// Optional extra query params merged into each request:
// params: { categories: "general", safesearch: 1 }
// Optional auth via headers (for reverse-proxy auth):
// headers: { "X-Api-Key": "..." }
}
}
}
}
}
```
## Getting a Brave API key ## Getting a Brave API key
1) Create a Brave Search API account at https://brave.com/search/api/ 1) Create a Brave Search API account at https://brave.com/search/api/
@ -145,9 +176,10 @@ Search the web using your configured provider.
### Requirements ### Requirements
- `tools.web.search.enabled` must not be `false` (default: enabled) - `tools.web.search.enabled` must not be `false` (default: enabled)
- API key for your chosen provider: - Provider setup:
- **Brave**: `BRAVE_API_KEY` or `tools.web.search.apiKey` - **Brave**: `BRAVE_API_KEY` or `tools.web.search.apiKey`
- **Perplexity**: `OPENROUTER_API_KEY`, `PERPLEXITY_API_KEY`, or `tools.web.search.perplexity.apiKey` - **Perplexity**: `OPENROUTER_API_KEY`, `PERPLEXITY_API_KEY`, or `tools.web.search.perplexity.apiKey`
- **SearXNG**: `tools.web.search.searxng.baseUrl` or `SEARXNG_BASE_URL`
### Config ### Config
@ -175,6 +207,12 @@ Search the web using your configured provider.
- `search_lang` (optional): ISO language code for search results (e.g., "de", "en", "fr") - `search_lang` (optional): ISO language code for search results (e.g., "de", "en", "fr")
- `ui_lang` (optional): ISO language code for UI elements - `ui_lang` (optional): ISO language code for UI elements
- `freshness` (optional, Brave only): filter by discovery time (`pd`, `pw`, `pm`, `py`, or `YYYY-MM-DDtoYYYY-MM-DD`) - `freshness` (optional, Brave only): filter by discovery time (`pd`, `pw`, `pm`, `py`, or `YYYY-MM-DDtoYYYY-MM-DD`)
- `categories` (optional, SearXNG only): comma-separated categories
- `engines` (optional, SearXNG only): comma-separated engines
- `language` (optional, SearXNG only): language code (if omitted, falls back to `search_lang`)
- `time_range` (optional, SearXNG only): `day`, `month`, `year`
- `safesearch` (optional, SearXNG only): `0`, `1`, `2`
- `pageno` (optional, SearXNG only): page number (default `1`)
**Examples:** **Examples:**

View File

@ -2,8 +2,13 @@ import { describe, expect, it } from "vitest";
import { __testing } from "./web-search.js"; import { __testing } from "./web-search.js";
const { inferPerplexityBaseUrlFromApiKey, resolvePerplexityBaseUrl, normalizeFreshness } = const {
__testing; inferPerplexityBaseUrlFromApiKey,
resolvePerplexityBaseUrl,
normalizeFreshness,
resolveSearxngBaseUrl,
resolveSearxngHeaders,
} = __testing;
describe("web_search perplexity baseUrl defaults", () => { describe("web_search perplexity baseUrl defaults", () => {
it("detects a Perplexity key prefix", () => { it("detects a Perplexity key prefix", () => {
@ -69,3 +74,36 @@ describe("web_search freshness normalization", () => {
expect(normalizeFreshness("2024-03-10to2024-03-01")).toBeUndefined(); expect(normalizeFreshness("2024-03-10to2024-03-01")).toBeUndefined();
}); });
}); });
describe("web_search searxng config resolution", () => {
it("prefers config baseUrl and trims trailing slash", () => {
expect(resolveSearxngBaseUrl({ baseUrl: "http://localhost:8080/" })).toBe(
"http://localhost:8080",
);
});
it("falls back to SEARXNG_BASE_URL env var", () => {
const prev = process.env.SEARXNG_BASE_URL;
process.env.SEARXNG_BASE_URL = "http://searxng.local:8888/";
try {
expect(resolveSearxngBaseUrl(undefined)).toBe("http://searxng.local:8888");
} finally {
if (prev === undefined) {
delete process.env.SEARXNG_BASE_URL;
} else {
process.env.SEARXNG_BASE_URL = prev;
}
}
});
it("builds headers with optional auth and extras", () => {
expect(resolveSearxngHeaders(undefined)).toEqual({ Accept: "application/json" });
expect(resolveSearxngHeaders({ apiKey: "token-123" }).Authorization).toBe("Bearer token-123");
expect(resolveSearxngHeaders({ apiKey: "Basic abc" }).Authorization).toBe("Basic abc");
const headers = resolveSearxngHeaders({ headers: { "X-Test": "ok" }, apiKey: "token-123" });
expect(headers["X-Test"]).toBe("ok");
expect(headers.Accept).toBe("application/json");
});
});

View File

@ -17,7 +17,7 @@ import {
writeCache, writeCache,
} from "./web-shared.js"; } from "./web-shared.js";
const SEARCH_PROVIDERS = ["brave", "perplexity"] as const; const SEARCH_PROVIDERS = ["brave", "perplexity", "searxng"] as const;
const DEFAULT_SEARCH_COUNT = 5; const DEFAULT_SEARCH_COUNT = 5;
const MAX_SEARCH_COUNT = 10; const MAX_SEARCH_COUNT = 10;
@ -41,6 +41,39 @@ const WebSearchSchema = Type.Object({
maximum: MAX_SEARCH_COUNT, maximum: MAX_SEARCH_COUNT,
}), }),
), ),
categories: Type.Optional(
Type.String({
description: "Comma-separated categories to search (SearXNG only), e.g. 'general,news'.",
}),
),
engines: Type.Optional(
Type.String({
description: "Comma-separated engines to search (SearXNG only), e.g. 'duckduckgo,bing'.",
}),
),
language: Type.Optional(
Type.String({
description: "Language code (SearXNG), e.g. 'en', 'de'.",
}),
),
time_range: Type.Optional(
Type.String({
description: "Time range (SearXNG only). Values: day, month, year.",
}),
),
safesearch: Type.Optional(
Type.Number({
description: "Safe search level (SearXNG only). Values: 0, 1, 2.",
minimum: 0,
maximum: 2,
}),
),
pageno: Type.Optional(
Type.Number({
description: "Search page number (SearXNG only). Default: 1.",
minimum: 1,
}),
),
country: Type.Optional( country: Type.Optional(
Type.String({ Type.String({
description: description:
@ -90,6 +123,13 @@ type PerplexityConfig = {
model?: string; model?: string;
}; };
type SearxngConfig = {
baseUrl?: string;
apiKey?: string;
headers?: Record<string, string>;
params?: Record<string, string | number | boolean>;
};
type PerplexityApiKeySource = "config" | "perplexity_env" | "openrouter_env" | "none"; type PerplexityApiKeySource = "config" | "perplexity_env" | "openrouter_env" | "none";
type PerplexitySearchResponse = { type PerplexitySearchResponse = {
@ -103,6 +143,17 @@ type PerplexitySearchResponse = {
type PerplexityBaseUrlHint = "direct" | "openrouter"; type PerplexityBaseUrlHint = "direct" | "openrouter";
type SearxngSearchResult = {
title?: string;
url?: string;
content?: string;
publishedDate?: string;
};
type SearxngSearchResponse = {
results?: SearxngSearchResult[];
};
function resolveSearchConfig(cfg?: OpenClawConfig): WebSearchConfig { function resolveSearchConfig(cfg?: OpenClawConfig): WebSearchConfig {
const search = cfg?.tools?.web?.search; const search = cfg?.tools?.web?.search;
if (!search || typeof search !== "object") return undefined; if (!search || typeof search !== "object") return undefined;
@ -131,6 +182,14 @@ function missingSearchKeyPayload(provider: (typeof SEARCH_PROVIDERS)[number]) {
docs: "https://docs.openclaw.ai/tools/web", docs: "https://docs.openclaw.ai/tools/web",
}; };
} }
if (provider === "searxng") {
return {
error: "missing_searxng_base_url",
message:
"web_search (searxng) needs a base URL. Configure tools.web.search.searxng.baseUrl or set SEARXNG_BASE_URL in the Gateway environment.",
docs: "https://docs.openclaw.ai/tools/web",
};
}
return { return {
error: "missing_brave_api_key", error: "missing_brave_api_key",
message: `web_search needs a Brave Search API key. Run \`${formatCliCommand("openclaw configure --section web")}\` to store it, or set BRAVE_API_KEY in the Gateway environment.`, message: `web_search needs a Brave Search API key. Run \`${formatCliCommand("openclaw configure --section web")}\` to store it, or set BRAVE_API_KEY in the Gateway environment.`,
@ -145,6 +204,7 @@ function resolveSearchProvider(search?: WebSearchConfig): (typeof SEARCH_PROVIDE
: ""; : "";
if (raw === "perplexity") return "perplexity"; if (raw === "perplexity") return "perplexity";
if (raw === "brave") return "brave"; if (raw === "brave") return "brave";
if (raw === "searxng") return "searxng";
return "brave"; return "brave";
} }
@ -155,6 +215,13 @@ function resolvePerplexityConfig(search?: WebSearchConfig): PerplexityConfig {
return perplexity as PerplexityConfig; return perplexity as PerplexityConfig;
} }
function resolveSearxngConfig(search?: WebSearchConfig): SearxngConfig {
if (!search || typeof search !== "object") return {};
const searxng = "searxng" in search ? (search as Record<string, unknown>).searxng : undefined;
if (!searxng || typeof searxng !== "object") return {};
return searxng as SearxngConfig;
}
function resolvePerplexityApiKey(perplexity?: PerplexityConfig): { function resolvePerplexityApiKey(perplexity?: PerplexityConfig): {
apiKey?: string; apiKey?: string;
source: PerplexityApiKeySource; source: PerplexityApiKeySource;
@ -221,6 +288,38 @@ function resolvePerplexityModel(perplexity?: PerplexityConfig): string {
return fromConfig || DEFAULT_PERPLEXITY_MODEL; return fromConfig || DEFAULT_PERPLEXITY_MODEL;
} }
function resolveSearxngBaseUrl(searxng?: SearxngConfig): string | undefined {
const fromConfig =
searxng && "baseUrl" in searxng && typeof searxng.baseUrl === "string" ? searxng.baseUrl : "";
const fromEnv = (process.env.SEARXNG_BASE_URL ?? "").trim();
const baseUrl = (fromConfig ?? "").trim() || fromEnv;
if (!baseUrl) return undefined;
return baseUrl.replace(/\/$/, "");
}
function resolveSearxngHeaders(searxng?: SearxngConfig): Record<string, string> {
const headers: Record<string, string> = {
Accept: "application/json",
};
if (!searxng || typeof searxng !== "object") return headers;
if (searxng.headers && typeof searxng.headers === "object") {
for (const [key, value] of Object.entries(searxng.headers)) {
if (typeof value === "string" && value.trim()) {
headers[key] = value;
}
}
}
const apiKey = typeof searxng.apiKey === "string" ? searxng.apiKey.trim() : "";
if (apiKey) {
headers.Authorization = apiKey.includes(" ") ? apiKey : `Bearer ${apiKey}`;
}
return headers;
}
function resolveSearchCount(value: unknown, fallback: number): number { function resolveSearchCount(value: unknown, fallback: number): number {
const parsed = typeof value === "number" && Number.isFinite(value) ? value : fallback; const parsed = typeof value === "number" && Number.isFinite(value) ? value : fallback;
const clamped = Math.max(1, Math.min(MAX_SEARCH_COUNT, Math.floor(parsed))); const clamped = Math.max(1, Math.min(MAX_SEARCH_COUNT, Math.floor(parsed)));
@ -306,6 +405,72 @@ async function runPerplexitySearch(params: {
return { content, citations }; return { content, citations };
} }
function addSearxngParams(
searchParams: URLSearchParams,
params?: Record<string, string | number | boolean>,
): void {
if (!params) return;
for (const [key, value] of Object.entries(params)) {
if (!key.trim()) continue;
if (value === undefined) continue;
searchParams.set(key, String(value));
}
}
async function runSearxngSearch(params: {
query: string;
baseUrl: string;
count: number;
timeoutSeconds: number;
searchParams?: Record<string, string | number | boolean>;
headers?: Record<string, string>;
categories?: string;
engines?: string;
language?: string;
time_range?: string;
safesearch?: number;
pageno?: number;
}): Promise<{ results: Array<Record<string, unknown>> }> {
const url = new URL(`${params.baseUrl.replace(/\/$/, "")}/search`);
addSearxngParams(url.searchParams, params.searchParams);
url.searchParams.set("q", params.query);
url.searchParams.set("format", "json");
if (params.categories) url.searchParams.set("categories", params.categories);
if (params.engines) url.searchParams.set("engines", params.engines);
if (params.language) url.searchParams.set("language", params.language);
if (params.time_range) url.searchParams.set("time_range", params.time_range);
if (typeof params.safesearch === "number" && Number.isFinite(params.safesearch)) {
url.searchParams.set("safesearch", String(params.safesearch));
}
if (typeof params.pageno === "number" && Number.isFinite(params.pageno) && params.pageno > 0) {
url.searchParams.set("pageno", String(Math.floor(params.pageno)));
}
const res = await fetch(url.toString(), {
method: "GET",
headers: params.headers,
signal: withTimeout(undefined, params.timeoutSeconds * 1000),
});
if (!res.ok) {
const detail = await readResponseText(res);
throw new Error(`SearXNG API error (${res.status}): ${detail || res.statusText}`);
}
const data = (await res.json()) as SearxngSearchResponse;
const results = Array.isArray(data.results) ? data.results : [];
const mapped = results.slice(0, params.count).map((entry) => ({
title: entry.title ?? "",
url: entry.url ?? "",
description: entry.content ?? "",
published: entry.publishedDate ?? undefined,
siteName: resolveSiteName(entry.url ?? ""),
}));
return { results: mapped };
}
async function runWebSearch(params: { async function runWebSearch(params: {
query: string; query: string;
count: number; count: number;
@ -319,11 +484,22 @@ async function runWebSearch(params: {
freshness?: string; freshness?: string;
perplexityBaseUrl?: string; perplexityBaseUrl?: string;
perplexityModel?: string; perplexityModel?: string;
searxngBaseUrl?: string;
searxngHeaders?: Record<string, string>;
searxngParams?: Record<string, string | number | boolean>;
categories?: string;
engines?: string;
language?: string;
time_range?: string;
safesearch?: number;
pageno?: number;
}): Promise<Record<string, unknown>> { }): Promise<Record<string, unknown>> {
const cacheKey = normalizeCacheKey( const cacheKey = normalizeCacheKey(
params.provider === "brave" params.provider === "brave"
? `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}:${params.freshness || "default"}` ? `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}:${params.freshness || "default"}`
: `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}`, : params.provider === "searxng"
? `${params.provider}:${params.query}:${params.count}:${params.categories || "default"}:${params.engines || "default"}:${params.language || "default"}:${params.time_range || "default"}:${String(params.safesearch ?? "default")}:${String(params.pageno ?? "default")}`
: `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}`,
); );
const cached = readCache(SEARCH_CACHE, cacheKey); const cached = readCache(SEARCH_CACHE, cacheKey);
if (cached) return { ...cached.value, cached: true }; if (cached) return { ...cached.value, cached: true };
@ -351,6 +527,33 @@ async function runWebSearch(params: {
return payload; return payload;
} }
if (params.provider === "searxng") {
const { results } = await runSearxngSearch({
query: params.query,
baseUrl: params.searxngBaseUrl ?? "",
count: params.count,
timeoutSeconds: params.timeoutSeconds,
searchParams: params.searxngParams,
headers: params.searxngHeaders,
categories: params.categories,
engines: params.engines,
language: params.language,
time_range: params.time_range,
safesearch: params.safesearch,
pageno: params.pageno,
});
const payload = {
query: params.query,
provider: params.provider,
count: results.length,
tookMs: Date.now() - start,
results,
};
writeCache(SEARCH_CACHE, cacheKey, payload, params.cacheTtlMs);
return payload;
}
if (params.provider !== "brave") { if (params.provider !== "brave") {
throw new Error("Unsupported web search provider."); throw new Error("Unsupported web search provider.");
} }
@ -415,11 +618,14 @@ export function createWebSearchTool(options?: {
const provider = resolveSearchProvider(search); const provider = resolveSearchProvider(search);
const perplexityConfig = resolvePerplexityConfig(search); const perplexityConfig = resolvePerplexityConfig(search);
const searxngConfig = resolveSearxngConfig(search);
const description = const description =
provider === "perplexity" provider === "perplexity"
? "Search the web using Perplexity Sonar (direct or via OpenRouter). Returns AI-synthesized answers with citations from real-time web search." ? "Search the web using Perplexity Sonar (direct or via OpenRouter). Returns AI-synthesized answers with citations from real-time web search."
: "Search the web using Brave Search API. Supports region-specific and localized search via country and language parameters. Returns titles, URLs, and snippets for fast research."; : provider === "searxng"
? "Search the web using a self-hosted SearXNG instance. Returns titles, URLs, and snippets for fast research."
: "Search the web using Brave Search API. Supports region-specific and localized search via country and language parameters. Returns titles, URLs, and snippets for fast research.";
return { return {
label: "Web Search", label: "Web Search",
@ -431,14 +637,26 @@ export function createWebSearchTool(options?: {
provider === "perplexity" ? resolvePerplexityApiKey(perplexityConfig) : undefined; provider === "perplexity" ? resolvePerplexityApiKey(perplexityConfig) : undefined;
const apiKey = const apiKey =
provider === "perplexity" ? perplexityAuth?.apiKey : resolveSearchApiKey(search); provider === "perplexity" ? perplexityAuth?.apiKey : resolveSearchApiKey(search);
const searxngBaseUrl =
provider === "searxng" ? resolveSearxngBaseUrl(searxngConfig) : undefined;
if (!apiKey) { if (provider === "searxng") {
if (!searxngBaseUrl) return jsonResult(missingSearchKeyPayload(provider));
} else if (!apiKey) {
return jsonResult(missingSearchKeyPayload(provider)); return jsonResult(missingSearchKeyPayload(provider));
} }
const params = args as Record<string, unknown>; const params = args as Record<string, unknown>;
const query = readStringParam(params, "query", { required: true }); const query = readStringParam(params, "query", { required: true });
const count = const count =
readNumberParam(params, "count", { integer: true }) ?? search?.maxResults ?? undefined; readNumberParam(params, "count", { integer: true }) ?? search?.maxResults ?? undefined;
const categories = readStringParam(params, "categories");
const engines = readStringParam(params, "engines");
const language =
readStringParam(params, "language") ??
(provider === "searxng" ? readStringParam(params, "search_lang") : undefined);
const time_range = readStringParam(params, "time_range");
const safesearch = readNumberParam(params, "safesearch", { integer: true });
const pageno = readNumberParam(params, "pageno", { integer: true });
const country = readStringParam(params, "country"); const country = readStringParam(params, "country");
const search_lang = readStringParam(params, "search_lang"); const search_lang = readStringParam(params, "search_lang");
const ui_lang = readStringParam(params, "ui_lang"); const ui_lang = readStringParam(params, "ui_lang");
@ -462,7 +680,7 @@ export function createWebSearchTool(options?: {
const result = await runWebSearch({ const result = await runWebSearch({
query, query,
count: resolveSearchCount(count, DEFAULT_SEARCH_COUNT), count: resolveSearchCount(count, DEFAULT_SEARCH_COUNT),
apiKey, apiKey: apiKey ?? "",
timeoutSeconds: resolveTimeoutSeconds(search?.timeoutSeconds, DEFAULT_TIMEOUT_SECONDS), timeoutSeconds: resolveTimeoutSeconds(search?.timeoutSeconds, DEFAULT_TIMEOUT_SECONDS),
cacheTtlMs: resolveCacheTtlMs(search?.cacheTtlMinutes, DEFAULT_CACHE_TTL_MINUTES), cacheTtlMs: resolveCacheTtlMs(search?.cacheTtlMinutes, DEFAULT_CACHE_TTL_MINUTES),
provider, provider,
@ -476,6 +694,15 @@ export function createWebSearchTool(options?: {
perplexityAuth?.apiKey, perplexityAuth?.apiKey,
), ),
perplexityModel: resolvePerplexityModel(perplexityConfig), perplexityModel: resolvePerplexityModel(perplexityConfig),
searxngBaseUrl,
searxngHeaders: provider === "searxng" ? resolveSearxngHeaders(searxngConfig) : undefined,
searxngParams: provider === "searxng" ? searxngConfig.params : undefined,
categories,
engines,
language,
time_range,
safesearch: typeof safesearch === "number" ? safesearch : undefined,
pageno: typeof pageno === "number" ? pageno : undefined,
}); });
return jsonResult(result); return jsonResult(result);
}, },
@ -486,4 +713,6 @@ export const __testing = {
inferPerplexityBaseUrlFromApiKey, inferPerplexityBaseUrlFromApiKey,
resolvePerplexityBaseUrl, resolvePerplexityBaseUrl,
normalizeFreshness, normalizeFreshness,
resolveSearxngBaseUrl,
resolveSearxngHeaders,
} as const; } as const;

View File

@ -18,7 +18,17 @@ const { nodesAction, registerNodesCli } = vi.hoisted(() => {
return { nodesAction: action, registerNodesCli: register }; return { nodesAction: action, registerNodesCli: register };
}); });
const { gatewayRunAction, registerGatewayCli } = vi.hoisted(() => {
const action = vi.fn();
const register = vi.fn((program: Command) => {
const gateway = program.command("gateway");
gateway.command("run").option("--port <port>").action(action);
});
return { gatewayRunAction: action, registerGatewayCli: register };
});
vi.mock("../acp-cli.js", () => ({ registerAcpCli })); vi.mock("../acp-cli.js", () => ({ registerAcpCli }));
vi.mock("../gateway-cli.js", () => ({ registerGatewayCli }));
vi.mock("../nodes-cli.js", () => ({ registerNodesCli })); vi.mock("../nodes-cli.js", () => ({ registerNodesCli }));
const { registerSubCliByName, registerSubCliCommands } = await import("./register.subclis.js"); const { registerSubCliByName, registerSubCliCommands } = await import("./register.subclis.js");
@ -32,6 +42,8 @@ describe("registerSubCliCommands", () => {
delete process.env.OPENCLAW_DISABLE_LAZY_SUBCOMMANDS; delete process.env.OPENCLAW_DISABLE_LAZY_SUBCOMMANDS;
registerAcpCli.mockClear(); registerAcpCli.mockClear();
acpAction.mockClear(); acpAction.mockClear();
registerGatewayCli.mockClear();
gatewayRunAction.mockClear();
registerNodesCli.mockClear(); registerNodesCli.mockClear();
nodesAction.mockClear(); nodesAction.mockClear();
}); });
@ -79,6 +91,24 @@ describe("registerSubCliCommands", () => {
expect(nodesAction).toHaveBeenCalledTimes(1); expect(nodesAction).toHaveBeenCalledTimes(1);
}); });
it("preserves options for lazy subcommands", async () => {
process.argv = ["node", "openclaw", "gateway", "run", "--port", "18889"];
const program = new Command();
program.name("openclaw");
registerSubCliCommands(program, process.argv);
expect(program.commands.map((cmd) => cmd.name())).toEqual(["gateway"]);
await program.parseAsync(process.argv);
expect(registerGatewayCli).toHaveBeenCalledTimes(1);
expect(gatewayRunAction).toHaveBeenCalledTimes(1);
expect(gatewayRunAction).toHaveBeenCalledWith(
expect.objectContaining({ port: "18889" }),
expect.anything(),
);
});
it("replaces placeholder when registering a subcommand by name", async () => { it("replaces placeholder when registering a subcommand by name", async () => {
process.argv = ["node", "openclaw", "acp", "--help"]; process.argv = ["node", "openclaw", "acp", "--help"];
const program = new Command(); const program = new Command();

View File

@ -246,7 +246,7 @@ export async function registerSubCliByName(program: Command, name: string): Prom
return true; return true;
} }
function registerLazyCommand(program: Command, entry: SubCliEntry) { function registerLazyCommand(program: Command, entry: SubCliEntry, argvOverride?: string[]) {
const placeholder = program.command(entry.name).description(entry.description); const placeholder = program.command(entry.name).description(entry.description);
placeholder.allowUnknownOption(true); placeholder.allowUnknownOption(true);
placeholder.allowExcessArguments(true); placeholder.allowExcessArguments(true);
@ -255,7 +255,7 @@ function registerLazyCommand(program: Command, entry: SubCliEntry) {
await entry.register(program); await entry.register(program);
const actionCommand = actionArgs.at(-1) as Command | undefined; const actionCommand = actionArgs.at(-1) as Command | undefined;
const root = actionCommand?.parent ?? program; const root = actionCommand?.parent ?? program;
const rawArgs = (root as Command & { rawArgs?: string[] }).rawArgs; const rawArgs = (root as Command & { rawArgs?: string[] }).rawArgs ?? argvOverride;
const actionArgsList = resolveActionArgs(actionCommand); const actionArgsList = resolveActionArgs(actionCommand);
const fallbackArgv = actionCommand?.name() const fallbackArgv = actionCommand?.name()
? [actionCommand.name(), ...actionArgsList] ? [actionCommand.name(), ...actionArgsList]
@ -280,11 +280,11 @@ export function registerSubCliCommands(program: Command, argv: string[] = proces
if (primary && shouldRegisterPrimaryOnly(argv)) { if (primary && shouldRegisterPrimaryOnly(argv)) {
const entry = entries.find((candidate) => candidate.name === primary); const entry = entries.find((candidate) => candidate.name === primary);
if (entry) { if (entry) {
registerLazyCommand(program, entry); registerLazyCommand(program, entry, argv);
return; return;
} }
} }
for (const candidate of entries) { for (const candidate of entries) {
registerLazyCommand(program, candidate); registerLazyCommand(program, candidate, argv);
} }
} }

View File

@ -193,6 +193,10 @@ const FIELD_LABELS: Record<string, string> = {
"tools.web.search.maxResults": "Web Search Max Results", "tools.web.search.maxResults": "Web Search Max Results",
"tools.web.search.timeoutSeconds": "Web Search Timeout (sec)", "tools.web.search.timeoutSeconds": "Web Search Timeout (sec)",
"tools.web.search.cacheTtlMinutes": "Web Search Cache TTL (min)", "tools.web.search.cacheTtlMinutes": "Web Search Cache TTL (min)",
"tools.web.search.searxng.baseUrl": "SearXNG Base URL",
"tools.web.search.searxng.apiKey": "SearXNG API Key",
"tools.web.search.searxng.headers": "SearXNG Extra Headers",
"tools.web.search.searxng.params": "SearXNG Default Params",
"tools.web.fetch.enabled": "Enable Web Fetch Tool", "tools.web.fetch.enabled": "Enable Web Fetch Tool",
"tools.web.fetch.maxChars": "Web Fetch Max Chars", "tools.web.fetch.maxChars": "Web Fetch Max Chars",
"tools.web.fetch.timeoutSeconds": "Web Fetch Timeout (sec)", "tools.web.fetch.timeoutSeconds": "Web Fetch Timeout (sec)",
@ -434,8 +438,9 @@ const FIELD_HELP: Record<string, string> = {
"tools.message.crossContext.marker.suffix": "tools.message.crossContext.marker.suffix":
'Text suffix for cross-context markers (supports "{channel}").', 'Text suffix for cross-context markers (supports "{channel}").',
"tools.message.broadcast.enabled": "Enable broadcast action (default: true).", "tools.message.broadcast.enabled": "Enable broadcast action (default: true).",
"tools.web.search.enabled": "Enable the web_search tool (requires a provider API key).", "tools.web.search.enabled":
"tools.web.search.provider": 'Search provider ("brave" or "perplexity").', "Enable the web_search tool (requires provider configuration: API key or base URL).",
"tools.web.search.provider": 'Search provider ("brave", "perplexity", or "searxng").',
"tools.web.search.apiKey": "Brave Search API key (fallback: BRAVE_API_KEY env var).", "tools.web.search.apiKey": "Brave Search API key (fallback: BRAVE_API_KEY env var).",
"tools.web.search.maxResults": "Default number of results to return (1-10).", "tools.web.search.maxResults": "Default number of results to return (1-10).",
"tools.web.search.timeoutSeconds": "Timeout in seconds for web_search requests.", "tools.web.search.timeoutSeconds": "Timeout in seconds for web_search requests.",
@ -446,6 +451,14 @@ const FIELD_HELP: Record<string, string> = {
"Perplexity base URL override (default: https://openrouter.ai/api/v1 or https://api.perplexity.ai).", "Perplexity base URL override (default: https://openrouter.ai/api/v1 or https://api.perplexity.ai).",
"tools.web.search.perplexity.model": "tools.web.search.perplexity.model":
'Perplexity model override (default: "perplexity/sonar-pro").', 'Perplexity model override (default: "perplexity/sonar-pro").',
"tools.web.search.searxng.baseUrl":
"SearXNG instance base URL (e.g. http://localhost:8080). Fallback: SEARXNG_BASE_URL env var.",
"tools.web.search.searxng.apiKey":
"Optional SearXNG auth token; sent as Authorization header (Bearer by default).",
"tools.web.search.searxng.headers":
"Optional extra headers for SearXNG requests (e.g. for reverse-proxy auth).",
"tools.web.search.searxng.params":
"Optional default query params for SearXNG /search (merged into each request).",
"tools.web.fetch.enabled": "Enable the web_fetch tool (lightweight HTTP fetch).", "tools.web.fetch.enabled": "Enable the web_fetch tool (lightweight HTTP fetch).",
"tools.web.fetch.maxChars": "Max characters returned by web_fetch (truncated).", "tools.web.fetch.maxChars": "Max characters returned by web_fetch (truncated).",
"tools.web.fetch.timeoutSeconds": "Timeout in seconds for web_fetch requests.", "tools.web.fetch.timeoutSeconds": "Timeout in seconds for web_fetch requests.",

View File

@ -165,7 +165,9 @@ export const ToolPolicySchema = ToolPolicyBaseSchema.superRefine((value, ctx) =>
export const ToolsWebSearchSchema = z export const ToolsWebSearchSchema = z
.object({ .object({
enabled: z.boolean().optional(), enabled: z.boolean().optional(),
provider: z.union([z.literal("brave"), z.literal("perplexity")]).optional(), provider: z
.union([z.literal("brave"), z.literal("perplexity"), z.literal("searxng")])
.optional(),
apiKey: z.string().optional(), apiKey: z.string().optional(),
maxResults: z.number().int().positive().optional(), maxResults: z.number().int().positive().optional(),
timeoutSeconds: z.number().int().positive().optional(), timeoutSeconds: z.number().int().positive().optional(),
@ -178,6 +180,15 @@ export const ToolsWebSearchSchema = z
}) })
.strict() .strict()
.optional(), .optional(),
searxng: z
.object({
baseUrl: z.string().optional(),
apiKey: z.string().optional(),
headers: z.record(z.string(), z.string()).optional(),
params: z.record(z.string(), z.union([z.string(), z.number(), z.boolean()])).optional(),
})
.strict()
.optional(),
}) })
.strict() .strict()
.optional(); .optional();

View File

@ -433,29 +433,101 @@ export async function finalizeOnboardingWizard(options: FinalizeOnboardingOption
); );
} }
const webSearchKey = (nextConfig.tools?.web?.search?.apiKey ?? "").trim(); const webSearch = nextConfig.tools?.web?.search;
const webSearchEnv = (process.env.BRAVE_API_KEY ?? "").trim(); const webSearchProviderRaw =
const hasWebSearchKey = Boolean(webSearchKey || webSearchEnv); webSearch &&
typeof webSearch === "object" &&
"provider" in webSearch &&
typeof webSearch.provider === "string"
? webSearch.provider.trim().toLowerCase()
: "";
const webSearchProvider =
webSearchProviderRaw === "perplexity"
? "perplexity"
: webSearchProviderRaw === "searxng"
? "searxng"
: "brave";
const readTrimmedString = (value: unknown): string =>
typeof value === "string" ? value.trim() : "";
const braveKey =
webSearch && typeof webSearch === "object" && "apiKey" in webSearch
? readTrimmedString(webSearch.apiKey)
: "";
const braveEnv = (process.env.BRAVE_API_KEY ?? "").trim();
const hasBraveKey = Boolean(braveKey || braveEnv);
const perplexityKey =
webSearch &&
typeof webSearch === "object" &&
"perplexity" in webSearch &&
webSearch.perplexity &&
typeof webSearch.perplexity === "object" &&
"apiKey" in webSearch.perplexity
? readTrimmedString((webSearch.perplexity as Record<string, unknown>).apiKey)
: "";
const perplexityEnv = (process.env.PERPLEXITY_API_KEY ?? "").trim();
const openrouterEnv = (process.env.OPENROUTER_API_KEY ?? "").trim();
const hasPerplexityKey = Boolean(perplexityKey || perplexityEnv || openrouterEnv);
const searxngBaseUrl =
webSearch &&
typeof webSearch === "object" &&
"searxng" in webSearch &&
webSearch.searxng &&
typeof webSearch.searxng === "object" &&
"baseUrl" in webSearch.searxng
? readTrimmedString((webSearch.searxng as Record<string, unknown>).baseUrl)
: "";
const searxngEnv = (process.env.SEARXNG_BASE_URL ?? "").trim();
const hasSearxngBaseUrl = Boolean(searxngBaseUrl || searxngEnv);
const hasWebSearchSetup =
webSearchProvider === "perplexity"
? hasPerplexityKey
: webSearchProvider === "searxng"
? hasSearxngBaseUrl
: hasBraveKey;
await prompter.note( await prompter.note(
hasWebSearchKey hasWebSearchSetup
? [ ? [
"Web search is enabled, so your agent can look things up online when needed.", "Web search is enabled, so your agent can look things up online when needed.",
"", "",
webSearchKey `Provider: ${webSearchProvider}`,
? "API key: stored in config (tools.web.search.apiKey)." webSearchProvider === "searxng"
: "API key: provided via BRAVE_API_KEY env var (Gateway environment).", ? searxngBaseUrl
? "Base URL: stored in config (tools.web.search.searxng.baseUrl)."
: "Base URL: provided via SEARXNG_BASE_URL env var (Gateway environment)."
: webSearchProvider === "perplexity"
? perplexityKey
? "API key: stored in config (tools.web.search.perplexity.apiKey)."
: perplexityEnv
? "API key: provided via PERPLEXITY_API_KEY env var (Gateway environment)."
: "API key: provided via OPENROUTER_API_KEY env var (Gateway environment)."
: braveKey
? "API key: stored in config (tools.web.search.apiKey)."
: "API key: provided via BRAVE_API_KEY env var (Gateway environment).",
"Docs: https://docs.openclaw.ai/tools/web", "Docs: https://docs.openclaw.ai/tools/web",
].join("\n") ].join("\n")
: [ : [
"If you want your agent to be able to search the web, youll need an API key.", "If you want your agent to be able to search the web, youll need to configure a provider.",
"", "",
"OpenClaw uses Brave Search for the `web_search` tool. Without a Brave Search API key, web search wont work.", `OpenClaw uses ${webSearchProvider} for the \`web_search\` tool.`,
"", "",
"Set it up interactively:", "Set it up interactively:",
`- Run: ${formatCliCommand("openclaw configure --section web")}`, `- Run: ${formatCliCommand("openclaw configure --section web")}`,
"- Enable web_search and paste your Brave Search API key", webSearchProvider === "searxng"
? "- Enable web_search and set your SearXNG base URL"
: webSearchProvider === "perplexity"
? "- Enable web_search and set your Perplexity/OpenRouter API key"
: "- Enable web_search and paste your Brave Search API key",
"", "",
"Alternative: set BRAVE_API_KEY in the Gateway environment (no config changes).", webSearchProvider === "searxng"
? "Alternative: set SEARXNG_BASE_URL in the Gateway environment (no config changes)."
: webSearchProvider === "perplexity"
? "Alternative: set PERPLEXITY_API_KEY or OPENROUTER_API_KEY in the Gateway environment (no config changes)."
: "Alternative: set BRAVE_API_KEY in the Gateway environment (no config changes).",
"Docs: https://docs.openclaw.ai/tools/web", "Docs: https://docs.openclaw.ai/tools/web",
].join("\n"), ].join("\n"),
"Web search (optional)", "Web search (optional)",