diff --git a/docs/brave-search.md b/docs/brave-search.md index e2a8a1306..874bfacfe 100644 --- a/docs/brave-search.md +++ b/docs/brave-search.md @@ -7,7 +7,7 @@ read_when: # Brave Search API -Moltbot uses Brave Search as the default provider for `web_search`. +Moltbot supports Brave Search as a web search provider for `web_search`. ## Get an API key @@ -32,9 +32,47 @@ Moltbot uses Brave Search as the default provider for `web_search`. } ``` +## Tool parameters + +| Parameter | Description | +|-----------|-------------| +| `query` | Search query (required) | +| `count` | Number of results to return (1-10, default: 5) | +| `country` | 2-letter ISO country code (e.g., "US", "DE") | +| `language` | ISO 639-1 language code for search results (e.g., "en", "de", "fr") | +| `ui_lang` | ISO language code for UI elements | +| `freshness` | Time filter: `day` (24h), `week`, `month`, or `year` | +| `date_after` | Only results published after this date (YYYY-MM-DD) | +| `date_before` | Only results published before this date (YYYY-MM-DD) | + +**Examples:** + +```javascript +// Country and language-specific search +await web_search({ + query: "renewable energy", + country: "DE", + language: "de" +}); + +// Recent results (past week) +await web_search({ + query: "AI news", + freshness: "week" +}); + +// Date range search +await web_search({ + query: "AI developments", + date_after: "2024-01-01", + date_before: "2024-06-30" +}); +``` + ## Notes - The Data for AI plan is **not** compatible with `web_search`. - Brave provides a free tier plus paid plans; check the Brave API portal for current limits. +- Results are cached for 15 minutes by default (configurable via `cacheTtlMinutes`). See [Web tools](/tools/web) for the full web_search configuration. diff --git a/docs/perplexity.md b/docs/perplexity.md index 3faacd812..386956b08 100644 --- a/docs/perplexity.md +++ b/docs/perplexity.md @@ -1,27 +1,20 @@ --- -summary: "Perplexity Sonar setup for web_search" +summary: "Perplexity Search API setup for web_search" read_when: - - You want to use Perplexity Sonar for web search - - You need PERPLEXITY_API_KEY or OpenRouter setup + - You want to use Perplexity Search for web search + - You need PERPLEXITY_API_KEY setup --- -# Perplexity Sonar +# Perplexity Search API -Moltbot can use Perplexity Sonar for the `web_search` tool. You can connect -through Perplexity’s direct API or via OpenRouter. +Moltbot uses Perplexity Search API for the `web_search` tool when `provider: "perplexity"` is set. +Perplexity Search returns structured results (title, URL, snippet) for fast research. -## API options +## Getting a Perplexity API key -### Perplexity (direct) - -- Base URL: https://api.perplexity.ai -- Environment variable: `PERPLEXITY_API_KEY` - -### OpenRouter (alternative) - -- Base URL: https://openrouter.ai/api/v1 -- Environment variable: `OPENROUTER_API_KEY` -- Supports prepaid/crypto credits. +1) Create a Perplexity account at https://www.perplexity.ai/settings/api +2) Generate an API key in the dashboard +3) Store the key in config (recommended) or set `PERPLEXITY_API_KEY` in the Gateway environment. ## Config example @@ -32,9 +25,7 @@ through Perplexity’s direct API or via OpenRouter. search: { provider: "perplexity", perplexity: { - apiKey: "pplx-...", - baseUrl: "https://api.perplexity.ai", - model: "perplexity/sonar-pro" + apiKey: "pplx-..." } } } @@ -51,8 +42,7 @@ through Perplexity’s direct API or via OpenRouter. search: { provider: "perplexity", perplexity: { - apiKey: "pplx-...", - baseUrl: "https://api.perplexity.ai" + apiKey: "pplx-..." } } } @@ -60,20 +50,83 @@ through Perplexity’s direct API or via OpenRouter. } ``` -If both `PERPLEXITY_API_KEY` and `OPENROUTER_API_KEY` are set, set -`tools.web.search.perplexity.baseUrl` (or `tools.web.search.perplexity.apiKey`) -to disambiguate. +## Where to set the key (recommended) -If no base URL is set, Moltbot chooses a default based on the API key source: +**Recommended:** run `moltbot configure --section web`. It stores the key in +`~/.clawdbot/moltbot.json` under `tools.web.search.perplexity.apiKey`. -- `PERPLEXITY_API_KEY` or `pplx-...` → direct Perplexity (`https://api.perplexity.ai`) -- `OPENROUTER_API_KEY` or `sk-or-...` → OpenRouter (`https://openrouter.ai/api/v1`) -- Unknown key formats → OpenRouter (safe fallback) +**Environment alternative:** set `PERPLEXITY_API_KEY` in the Gateway process +environment. For a gateway install, put it in `~/.clawdbot/.env` (or your +service environment). See [Env vars](/help/faq#how-does-moltbot-load-environment-variables). -## Models +## Tool parameters -- `perplexity/sonar` — fast Q&A with web search -- `perplexity/sonar-pro` (default) — multi-step reasoning + web search -- `perplexity/sonar-reasoning-pro` — deep research +| Parameter | Description | +|-----------|-------------| +| `query` | Search query (required) | +| `count` | Number of results to return (1-10, default: 5) | +| `country` | 2-letter ISO country code (e.g., "US", "DE") | +| `language` | ISO 639-1 language code (e.g., "en", "de", "fr") | +| `freshness` | Time filter: `day` (24h), `week`, `month`, or `year` | +| `date_after` | Only results published after this date (YYYY-MM-DD) | +| `date_before` | Only results published before this date (YYYY-MM-DD) | +| `domain_filter` | Domain allowlist/denylist array (max 20) | +| `max_tokens` | Total content budget (default: 25000, max: 1000000) | +| `max_tokens_per_page` | Per-page token limit (default: 2048) | + +**Examples:** + +```javascript +// Country and language-specific search +await web_search({ + query: "renewable energy", + country: "DE", + language: "de" +}); + +// Recent results (past week) +await web_search({ + query: "AI news", + freshness: "week" +}); + +// Date range search +await web_search({ + query: "AI developments", + date_after: "2024-01-01", + date_before: "2024-06-30" +}); + +// Domain filtering (allowlist) +await web_search({ + query: "climate research", + domain_filter: ["nature.com", "science.org", ".edu"] +}); + +// Domain filtering (denylist - prefix with -) +await web_search({ + query: "product reviews", + domain_filter: ["-reddit.com", "-pinterest.com"] +}); + +// More content extraction +await web_search({ + query: "detailed AI research", + max_tokens: 50000, + max_tokens_per_page: 4096 +}); +``` + +### Domain filter rules + +- Maximum 20 domains per filter +- Cannot mix allowlist and denylist in the same request +- Use `-` prefix for denylist entries (e.g., `["-reddit.com"]`) + +## Notes + +- Perplexity Search API returns structured web search results (title, URL, snippet) +- Results are cached for 15 minutes by default (configurable via `cacheTtlMinutes`) See [Web tools](/tools/web) for the full web_search configuration. +See [Perplexity Search API docs](https://docs.perplexity.ai/guides/search-quickstart) for more details. diff --git a/docs/tools/web.md b/docs/tools/web.md index be2a57f9e..cac40e861 100644 --- a/docs/tools/web.md +++ b/docs/tools/web.md @@ -1,16 +1,15 @@ --- -summary: "Web search + fetch tools (Brave Search API, Perplexity direct/OpenRouter)" +summary: "Web search + fetch tools (Perplexity Search API, Brave Search API)" read_when: - You want to enable web_search or web_fetch - - You need Brave Search API key setup - - You want to use Perplexity Sonar for web search + - You need Perplexity or Brave Search API key setup --- # Web tools Moltbot ships two lightweight web tools: -- `web_search` — Search the web via Brave Search API (default) or Perplexity Sonar (direct or via OpenRouter). +- `web_search` — Search the web using Perplexity Search API or Brave Search API. - `web_fetch` — HTTP fetch + readable extraction (HTML → markdown/text). These are **not** browser automation. For JS-heavy sites or logins, use the @@ -19,21 +18,20 @@ These are **not** browser automation. For JS-heavy sites or logins, use the ## How it works - `web_search` calls your configured provider and returns results. - - **Brave** (default): returns structured results (title, URL, snippet). - - **Perplexity**: returns AI-synthesized answers with citations from real-time web search. - Results are cached by query for 15 minutes (configurable). - `web_fetch` does a plain HTTP GET and extracts readable content (HTML → markdown/text). It does **not** execute JavaScript. - `web_fetch` is enabled by default (unless explicitly disabled). +See [Perplexity Search setup](/perplexity) and [Brave Search setup](/brave-search) for provider-specific details. + ## Choosing a search provider -| Provider | Pros | Cons | API Key | -|----------|------|------|---------| -| **Brave** (default) | Fast, structured results, free tier | Traditional search results | `BRAVE_API_KEY` | -| **Perplexity** | AI-synthesized answers, citations, real-time | Requires Perplexity or OpenRouter access | `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` | +| Provider | Pros | Cons | API Key | +|---------------------------|----------------------------------------------------------------------------------------------|-------------------------------|------------------------------------------------| +| **Perplexity Search API** | Fast, structured results; domain, language, region, and freshness filters; content extraction options; free credits for Moltbot users | — | Requires Perplexity API key `PERPLEXITY_API_KEY` | +| **Brave Search API** | Fast, structured results; free tier available | Fewer filtering options | Requires Brave API key `BRAVE_API_KEY` | -See [Brave Search setup](/brave-search) and [Perplexity Sonar](/perplexity) for provider-specific details. Set the provider in config: @@ -42,63 +40,44 @@ Set the provider in config: tools: { web: { search: { - provider: "brave" // or "perplexity" + provider: "perplexity" // or "brave" } } } } ``` -Example: switch to Perplexity Sonar (direct API): +## Setting up web search -```json5 -{ - tools: { - web: { - search: { - provider: "perplexity", - perplexity: { - apiKey: "pplx-...", - baseUrl: "https://api.perplexity.ai", - model: "perplexity/sonar-pro" - } - } - } - } -} -``` +Use `moltbot configure --section web` to set up your API key and choose a provider. -## Getting a Brave API key +### Perplexity Search + +1) Create a Perplexity account at https://www.perplexity.ai/settings/api +2) Generate an API key in the dashboard +3) Run `moltbot configure --section web` to store the key in config, or set `PERPLEXITY_API_KEY` in your environment. + +Perplexity provides $5 in API credits on a monthly rolling basis to Perplexity Pro subscribers. Additionally, Perplexity provides complementary credits for Moltbot users. + +See [Perplexity Search API Docs](https://docs.perplexity.ai/guides/search-quickstart) for more details. + +### Brave Search 1) Create a Brave Search API account at https://brave.com/search/api/ -2) In the dashboard, choose the **Data for Search** plan (not “Data for AI”) and generate an API key. +2) In the dashboard, choose the **Data for Search** plan (not "Data for AI") and generate an API key. 3) Run `moltbot configure --section web` to store the key in config (recommended), or set `BRAVE_API_KEY` in your environment. -Brave provides a free tier plus paid plans; check the Brave API portal for the -current limits and pricing. +Brave provides a free tier plus paid plans; check the Brave API portal for the current limits and pricing. -### Where to set the key (recommended) +### Where to store the key -**Recommended:** run `moltbot configure --section web`. It stores the key in -`~/.clawdbot/moltbot.json` under `tools.web.search.apiKey`. +**Via config (recommended):** run `moltbot configure --section web`. It stores the key under `tools.web.search.perplexity.apiKey` or `tools.web.search.apiKey`. -**Environment alternative:** set `BRAVE_API_KEY` in the Gateway process -environment. For a gateway install, put it in `~/.clawdbot/.env` (or your -service environment). See [Env vars](/help/faq#how-does-moltbot-load-environment-variables). +**Via environment:** set `PERPLEXITY_API_KEY` or `BRAVE_API_KEY` in the Gateway process environment. For a gateway install, put it in `~/.clawdbot/.env` (or your service environment). See [Env vars](/help/faq#how-does-moltbot-load-environment-variables). -## Using Perplexity (direct or via OpenRouter) +### Config examples -Perplexity Sonar models have built-in web search capabilities and return AI-synthesized -answers with citations. You can use them via OpenRouter (no credit card required - supports -crypto/prepaid). - -### Getting an OpenRouter API key - -1) Create an account at https://openrouter.ai/ -2) Add credits (supports crypto, prepaid, or credit card) -3) Generate an API key in your account settings - -### Setting up Perplexity search +**Perplexity Search:** ```json5 { @@ -108,12 +87,7 @@ crypto/prepaid). enabled: true, provider: "perplexity", perplexity: { - // API key (optional if OPENROUTER_API_KEY or PERPLEXITY_API_KEY is set) - apiKey: "sk-or-v1-...", - // Base URL (key-aware default if omitted) - baseUrl: "https://openrouter.ai/api/v1", - // Model (defaults to perplexity/sonar-pro) - model: "perplexity/sonar-pro" + apiKey: "pplx-..." // optional if PERPLEXITY_API_KEY is set } } } @@ -121,22 +95,21 @@ crypto/prepaid). } ``` -**Environment alternative:** set `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` in the Gateway -environment. For a gateway install, put it in `~/.clawdbot/.env`. +**Brave Search:** -If no base URL is set, Moltbot chooses a default based on the API key source: - -- `PERPLEXITY_API_KEY` or `pplx-...` → `https://api.perplexity.ai` -- `OPENROUTER_API_KEY` or `sk-or-...` → `https://openrouter.ai/api/v1` -- Unknown key formats → OpenRouter (safe fallback) - -### Available Perplexity models - -| Model | Description | Best for | -|-------|-------------|----------| -| `perplexity/sonar` | Fast Q&A with web search | Quick lookups | -| `perplexity/sonar-pro` (default) | Multi-step reasoning with web search | Complex questions | -| `perplexity/sonar-reasoning-pro` | Chain-of-thought analysis | Deep research | +```json5 +{ + tools: { + web: { + search: { + enabled: true, + provider: "brave", + apiKey: "BSA..." // optional if BRAVE_API_KEY is set + } + } + } +} +``` ## web_search @@ -147,7 +120,7 @@ Search the web using your configured provider. - `tools.web.search.enabled` must not be `false` (default: enabled) - API key for your chosen provider: - **Brave**: `BRAVE_API_KEY` or `tools.web.search.apiKey` - - **Perplexity**: `OPENROUTER_API_KEY`, `PERPLEXITY_API_KEY`, or `tools.web.search.perplexity.apiKey` + - **Perplexity**: `PERPLEXITY_API_KEY` or `tools.web.search.perplexity.apiKey` ### Config @@ -169,12 +142,21 @@ Search the web using your configured provider. ### Tool parameters -- `query` (required) -- `count` (1–10; default from config) -- `country` (optional): 2-letter country code for region-specific results (e.g., "DE", "US", "ALL"). If omitted, Brave chooses its default region. -- `search_lang` (optional): ISO language code for search results (e.g., "de", "en", "fr") -- `ui_lang` (optional): ISO language code for UI elements -- `freshness` (optional, Brave only): filter by discovery time (`pd`, `pw`, `pm`, `py`, or `YYYY-MM-DDtoYYYY-MM-DD`) +All parameters work for both Brave and Perplexity unless noted. + +| Parameter | Description | +|-----------|-------------| +| `query` | Search query (required) | +| `count` | Results to return (1-10, default: 5) | +| `country` | 2-letter ISO country code (e.g., "US", "DE") | +| `language` | ISO 639-1 language code (e.g., "en", "de") | +| `freshness` | Time filter: `day`, `week`, `month`, or `year` | +| `date_after` | Results after this date (YYYY-MM-DD) | +| `date_before` | Results before this date (YYYY-MM-DD) | +| `ui_lang` | UI language code (Brave only) | +| `domain_filter` | Domain allowlist/denylist array (Perplexity only) | +| `max_tokens` | Total content budget, default 25000 (Perplexity only) | +| `max_tokens_per_page` | Per-page token limit, default 2048 (Perplexity only) | **Examples:** @@ -182,23 +164,40 @@ Search the web using your configured provider. // German-specific search await web_search({ query: "TV online schauen", - count: 10, country: "DE", - search_lang: "de" -}); - -// French search with French UI -await web_search({ - query: "actualités", - country: "FR", - search_lang: "fr", - ui_lang: "fr" + language: "de" }); // Recent results (past week) await web_search({ query: "TMBG interview", - freshness: "pw" + freshness: "week" +}); + +// Date range search +await web_search({ + query: "AI developments", + date_after: "2024-01-01", + date_before: "2024-06-30" +}); + +// Domain filtering (Perplexity only) +await web_search({ + query: "climate research", + domain_filter: ["nature.com", "science.org", ".edu"] +}); + +// Exclude domains (Perplexity only) +await web_search({ + query: "product reviews", + domain_filter: ["-reddit.com", "-pinterest.com"] +}); + +// More content extraction (Perplexity only) +await web_search({ + query: "detailed AI research", + max_tokens: 50000, + max_tokens_per_page: 4096 }); ``` diff --git a/src/agents/tools/web-search.test.ts b/src/agents/tools/web-search.test.ts index c6d2b6405..4aeaca6fb 100644 --- a/src/agents/tools/web-search.test.ts +++ b/src/agents/tools/web-search.test.ts @@ -2,70 +2,55 @@ import { describe, expect, it } from "vitest"; import { __testing } from "./web-search.js"; -const { inferPerplexityBaseUrlFromApiKey, resolvePerplexityBaseUrl, normalizeFreshness } = - __testing; - -describe("web_search perplexity baseUrl defaults", () => { - it("detects a Perplexity key prefix", () => { - expect(inferPerplexityBaseUrlFromApiKey("pplx-123")).toBe("direct"); - }); - - it("detects an OpenRouter key prefix", () => { - expect(inferPerplexityBaseUrlFromApiKey("sk-or-v1-123")).toBe("openrouter"); - }); - - it("returns undefined for unknown key formats", () => { - expect(inferPerplexityBaseUrlFromApiKey("unknown-key")).toBeUndefined(); - }); - - it("prefers explicit baseUrl over key-based defaults", () => { - expect(resolvePerplexityBaseUrl({ baseUrl: "https://example.com" }, "config", "pplx-123")).toBe( - "https://example.com", - ); - }); - - it("defaults to direct when using PERPLEXITY_API_KEY", () => { - expect(resolvePerplexityBaseUrl(undefined, "perplexity_env")).toBe("https://api.perplexity.ai"); - }); - - it("defaults to OpenRouter when using OPENROUTER_API_KEY", () => { - expect(resolvePerplexityBaseUrl(undefined, "openrouter_env")).toBe( - "https://openrouter.ai/api/v1", - ); - }); - - it("defaults to direct when config key looks like Perplexity", () => { - expect(resolvePerplexityBaseUrl(undefined, "config", "pplx-123")).toBe( - "https://api.perplexity.ai", - ); - }); - - it("defaults to OpenRouter when config key looks like OpenRouter", () => { - expect(resolvePerplexityBaseUrl(undefined, "config", "sk-or-v1-123")).toBe( - "https://openrouter.ai/api/v1", - ); - }); - - it("defaults to OpenRouter for unknown config key formats", () => { - expect(resolvePerplexityBaseUrl(undefined, "config", "weird-key")).toBe( - "https://openrouter.ai/api/v1", - ); - }); -}); +const { normalizeFreshness, normalizeToIsoDate, isoToPerplexityDate } = __testing; describe("web_search freshness normalization", () => { - it("accepts Brave shortcut values", () => { - expect(normalizeFreshness("pd")).toBe("pd"); - expect(normalizeFreshness("PW")).toBe("pw"); + it("accepts Brave shortcut values and maps for Perplexity", () => { + expect(normalizeFreshness("pd", "brave")).toBe("pd"); + expect(normalizeFreshness("PW", "brave")).toBe("pw"); + expect(normalizeFreshness("pd", "perplexity")).toBe("day"); + expect(normalizeFreshness("pw", "perplexity")).toBe("week"); }); - it("accepts valid date ranges", () => { - expect(normalizeFreshness("2024-01-01to2024-01-31")).toBe("2024-01-01to2024-01-31"); + it("accepts Perplexity values and maps for Brave", () => { + expect(normalizeFreshness("day", "perplexity")).toBe("day"); + expect(normalizeFreshness("week", "perplexity")).toBe("week"); + expect(normalizeFreshness("day", "brave")).toBe("pd"); + expect(normalizeFreshness("week", "brave")).toBe("pw"); }); - it("rejects invalid date ranges", () => { - expect(normalizeFreshness("2024-13-01to2024-01-31")).toBeUndefined(); - expect(normalizeFreshness("2024-02-30to2024-03-01")).toBeUndefined(); - expect(normalizeFreshness("2024-03-10to2024-03-01")).toBeUndefined(); + it("rejects invalid values", () => { + expect(normalizeFreshness("yesterday", "brave")).toBeUndefined(); + expect(normalizeFreshness("yesterday", "perplexity")).toBeUndefined(); + expect(normalizeFreshness("2024-01-01to2024-01-31", "perplexity")).toBeUndefined(); + }); +}); + +describe("web_search date normalization", () => { + it("accepts ISO format", () => { + expect(normalizeToIsoDate("2024-01-15")).toBe("2024-01-15"); + expect(normalizeToIsoDate("2025-12-31")).toBe("2025-12-31"); + }); + + it("accepts Perplexity format and converts to ISO", () => { + expect(normalizeToIsoDate("1/15/2024")).toBe("2024-01-15"); + expect(normalizeToIsoDate("12/31/2025")).toBe("2025-12-31"); + }); + + it("rejects invalid formats", () => { + expect(normalizeToIsoDate("01-15-2024")).toBeUndefined(); + expect(normalizeToIsoDate("2024/01/15")).toBeUndefined(); + expect(normalizeToIsoDate("invalid")).toBeUndefined(); + }); + + it("converts ISO to Perplexity format", () => { + expect(isoToPerplexityDate("2024-01-15")).toBe("1/15/2024"); + expect(isoToPerplexityDate("2025-12-31")).toBe("12/31/2025"); + expect(isoToPerplexityDate("2024-03-05")).toBe("3/5/2024"); + }); + + it("rejects invalid ISO dates", () => { + expect(isoToPerplexityDate("1/15/2024")).toBeUndefined(); + expect(isoToPerplexityDate("invalid")).toBeUndefined(); }); }); diff --git a/src/agents/tools/web-search.ts b/src/agents/tools/web-search.ts index 1d87676e8..40ff65da4 100644 --- a/src/agents/tools/web-search.ts +++ b/src/agents/tools/web-search.ts @@ -3,7 +3,7 @@ import { Type } from "@sinclair/typebox"; import type { MoltbotConfig } from "../../config/config.js"; import { formatCliCommand } from "../../cli/command-format.js"; import type { AnyAgentTool } from "./common.js"; -import { jsonResult, readNumberParam, readStringParam } from "./common.js"; +import { jsonResult, readNumberParam, readStringArrayParam, readStringParam } from "./common.js"; import { CacheEntry, DEFAULT_CACHE_TTL_MINUTES, @@ -22,48 +22,118 @@ const DEFAULT_SEARCH_COUNT = 5; const MAX_SEARCH_COUNT = 10; const BRAVE_SEARCH_ENDPOINT = "https://api.search.brave.com/res/v1/web/search"; -const DEFAULT_PERPLEXITY_BASE_URL = "https://openrouter.ai/api/v1"; -const PERPLEXITY_DIRECT_BASE_URL = "https://api.perplexity.ai"; -const DEFAULT_PERPLEXITY_MODEL = "perplexity/sonar-pro"; -const PERPLEXITY_KEY_PREFIXES = ["pplx-"]; -const OPENROUTER_KEY_PREFIXES = ["sk-or-"]; +const PERPLEXITY_SEARCH_ENDPOINT = "https://api.perplexity.ai/search"; const SEARCH_CACHE = new Map>>(); const BRAVE_FRESHNESS_SHORTCUTS = new Set(["pd", "pw", "pm", "py"]); -const BRAVE_FRESHNESS_RANGE = /^(\d{4}-\d{2}-\d{2})to(\d{4}-\d{2}-\d{2})$/; +const PERPLEXITY_RECENCY_VALUES = new Set(["day", "week", "month", "year"]); -const WebSearchSchema = Type.Object({ - query: Type.String({ description: "Search query string." }), - count: Type.Optional( - Type.Number({ - description: "Number of results to return (1-10).", - minimum: 1, - maximum: MAX_SEARCH_COUNT, - }), - ), - country: Type.Optional( - Type.String({ - description: - "2-letter country code for region-specific results (e.g., 'DE', 'US', 'ALL'). Default: 'US'.", - }), - ), - search_lang: Type.Optional( - Type.String({ - description: "ISO language code for search results (e.g., 'de', 'en', 'fr').", - }), - ), - ui_lang: Type.Optional( - Type.String({ - description: "ISO language code for UI elements.", - }), - ), - freshness: Type.Optional( - Type.String({ - description: - "Filter results by discovery time (Brave only). Values: 'pd' (past 24h), 'pw' (past week), 'pm' (past month), 'py' (past year), or date range 'YYYY-MM-DDtoYYYY-MM-DD'.", - }), - ), -}); +const FRESHNESS_TO_RECENCY: Record = { + pd: "day", + pw: "week", + pm: "month", + py: "year", +}; +const RECENCY_TO_FRESHNESS: Record = { + day: "pd", + week: "pw", + month: "pm", + year: "py", +}; + +const ISO_DATE_PATTERN = /^(\d{4})-(\d{2})-(\d{2})$/; +const PERPLEXITY_DATE_PATTERN = /^(\d{1,2})\/(\d{1,2})\/(\d{4})$/; + +function isoToPerplexityDate(iso: string): string | undefined { + const match = iso.match(ISO_DATE_PATTERN); + if (!match) return undefined; + const [, year, month, day] = match; + return `${parseInt(month, 10)}/${parseInt(day, 10)}/${year}`; +} + +function normalizeToIsoDate(value: string): string | undefined { + const trimmed = value.trim(); + if (ISO_DATE_PATTERN.test(trimmed)) return trimmed; + const match = trimmed.match(PERPLEXITY_DATE_PATTERN); + if (match) { + const [, month, day, year] = match; + return `${year}-${month.padStart(2, "0")}-${day.padStart(2, "0")}`; + } + return undefined; +} + +function createWebSearchSchema(provider: (typeof SEARCH_PROVIDERS)[number]) { + const baseSchema = { + query: Type.String({ description: "Search query string." }), + count: Type.Optional( + Type.Number({ + description: "Number of results to return (1-10).", + minimum: 1, + maximum: MAX_SEARCH_COUNT, + }), + ), + country: Type.Optional( + Type.String({ + description: + "2-letter country code for region-specific results (e.g., 'DE', 'US', 'ALL'). Default: 'US'.", + }), + ), + language: Type.Optional( + Type.String({ + description: "ISO 639-1 language code for results (e.g., 'en', 'de', 'fr').", + }), + ), + freshness: Type.Optional( + Type.String({ + description: "Filter by time: 'day' (24h), 'week', 'month', or 'year'.", + }), + ), + date_after: Type.Optional( + Type.String({ + description: "Only results published after this date (YYYY-MM-DD).", + }), + ), + date_before: Type.Optional( + Type.String({ + description: "Only results published before this date (YYYY-MM-DD).", + }), + ), + } as const; + + if (provider === "brave") { + return Type.Object({ + ...baseSchema, + ui_lang: Type.Optional( + Type.String({ + description: "ISO language code for UI elements.", + }), + ), + }); + } + + return Type.Object({ + ...baseSchema, + domain_filter: Type.Optional( + Type.Array(Type.String(), { + description: + "Domain filter (max 20). Allowlist: ['nature.com'] or denylist: ['-reddit.com']. Cannot mix.", + }), + ), + max_tokens: Type.Optional( + Type.Number({ + description: "Total content budget across all results (default: 25000, max: 1000000).", + minimum: 1, + maximum: 1000000, + }), + ), + max_tokens_per_page: Type.Optional( + Type.Number({ + description: "Max tokens extracted per page (default: 2048).", + minimum: 1, + }), + ), + }); +} type WebSearchConfig = NonNullable["web"] extends infer Web ? Web extends { search?: infer Search } @@ -86,22 +156,22 @@ type BraveSearchResponse = { type PerplexityConfig = { apiKey?: string; - baseUrl?: string; - model?: string; }; -type PerplexityApiKeySource = "config" | "perplexity_env" | "openrouter_env" | "none"; +type PerplexityApiKeySource = "config" | "perplexity_env" | "none"; -type PerplexitySearchResponse = { - choices?: Array<{ - message?: { - content?: string; - }; - }>; - citations?: string[]; +type PerplexitySearchApiResult = { + title?: string; + url?: string; + snippet?: string; + date?: string; + last_updated?: string; }; -type PerplexityBaseUrlHint = "direct" | "openrouter"; +type PerplexitySearchApiResponse = { + results?: PerplexitySearchApiResult[]; + id?: string; +}; function resolveSearchConfig(cfg?: MoltbotConfig): WebSearchConfig { const search = cfg?.tools?.web?.search; @@ -127,13 +197,13 @@ function missingSearchKeyPayload(provider: (typeof SEARCH_PROVIDERS)[number]) { return { error: "missing_perplexity_api_key", message: - "web_search (perplexity) needs an API key. Set PERPLEXITY_API_KEY or OPENROUTER_API_KEY in the Gateway environment, or configure tools.web.search.perplexity.apiKey.", + "web_search (perplexity) needs an API key. Set PERPLEXITY_API_KEY in the Gateway environment, or configure tools.web.search.perplexity.apiKey.", docs: "https://docs.molt.bot/tools/web", }; } return { error: "missing_brave_api_key", - message: `web_search needs a Brave Search API key. Run \`${formatCliCommand("moltbot configure --section web")}\` to store it, or set BRAVE_API_KEY in the Gateway environment.`, + message: `web_search (brave) needs an API key. Run \`${formatCliCommand("moltbot configure --section web")}\` to store it, or set BRAVE_API_KEY in the Gateway environment.`, docs: "https://docs.molt.bot/tools/web", }; } @@ -169,11 +239,6 @@ function resolvePerplexityApiKey(perplexity?: PerplexityConfig): { return { apiKey: fromEnvPerplexity, source: "perplexity_env" }; } - const fromEnvOpenRouter = normalizeApiKey(process.env.OPENROUTER_API_KEY); - if (fromEnvOpenRouter) { - return { apiKey: fromEnvOpenRouter, source: "openrouter_env" }; - } - return { apiKey: undefined, source: "none" }; } @@ -181,79 +246,36 @@ function normalizeApiKey(key: unknown): string { return typeof key === "string" ? key.trim() : ""; } -function inferPerplexityBaseUrlFromApiKey(apiKey?: string): PerplexityBaseUrlHint | undefined { - if (!apiKey) return undefined; - const normalized = apiKey.toLowerCase(); - if (PERPLEXITY_KEY_PREFIXES.some((prefix) => normalized.startsWith(prefix))) { - return "direct"; - } - if (OPENROUTER_KEY_PREFIXES.some((prefix) => normalized.startsWith(prefix))) { - return "openrouter"; - } - return undefined; -} - -function resolvePerplexityBaseUrl( - perplexity?: PerplexityConfig, - apiKeySource: PerplexityApiKeySource = "none", - apiKey?: string, -): string { - const fromConfig = - perplexity && "baseUrl" in perplexity && typeof perplexity.baseUrl === "string" - ? perplexity.baseUrl.trim() - : ""; - if (fromConfig) return fromConfig; - if (apiKeySource === "perplexity_env") return PERPLEXITY_DIRECT_BASE_URL; - if (apiKeySource === "openrouter_env") return DEFAULT_PERPLEXITY_BASE_URL; - if (apiKeySource === "config") { - const inferred = inferPerplexityBaseUrlFromApiKey(apiKey); - if (inferred === "direct") return PERPLEXITY_DIRECT_BASE_URL; - if (inferred === "openrouter") return DEFAULT_PERPLEXITY_BASE_URL; - } - return DEFAULT_PERPLEXITY_BASE_URL; -} - -function resolvePerplexityModel(perplexity?: PerplexityConfig): string { - const fromConfig = - perplexity && "model" in perplexity && typeof perplexity.model === "string" - ? perplexity.model.trim() - : ""; - return fromConfig || DEFAULT_PERPLEXITY_MODEL; -} - function resolveSearchCount(value: unknown, fallback: number): number { const parsed = typeof value === "number" && Number.isFinite(value) ? value : fallback; const clamped = Math.max(1, Math.min(MAX_SEARCH_COUNT, Math.floor(parsed))); return clamped; } -function normalizeFreshness(value: string | undefined): string | undefined { +/** + * Normalizes freshness shortcut to the provider's expected format. + * Accepts both Brave format (pd/pw/pm/py) and Perplexity format (day/week/month/year). + * Use date_after/date_before for specific date ranges. + */ +function normalizeFreshness( + value: string | undefined, + provider: (typeof SEARCH_PROVIDERS)[number], +): string | undefined { if (!value) return undefined; const trimmed = value.trim(); if (!trimmed) return undefined; const lower = trimmed.toLowerCase(); - if (BRAVE_FRESHNESS_SHORTCUTS.has(lower)) return lower; - const match = trimmed.match(BRAVE_FRESHNESS_RANGE); - if (!match) return undefined; + if (BRAVE_FRESHNESS_SHORTCUTS.has(lower)) { + return provider === "brave" ? lower : FRESHNESS_TO_RECENCY[lower]; + } - const [, start, end] = match; - if (!isValidIsoDate(start) || !isValidIsoDate(end)) return undefined; - if (start > end) return undefined; + if (PERPLEXITY_RECENCY_VALUES.has(lower)) { + return provider === "perplexity" ? lower : RECENCY_TO_FRESHNESS[lower]; + } - return `${start}to${end}`; -} - -function isValidIsoDate(value: string): boolean { - if (!/^\d{4}-\d{2}-\d{2}$/.test(value)) return false; - const [year, month, day] = value.split("-").map((part) => Number.parseInt(part, 10)); - if (!Number.isFinite(year) || !Number.isFinite(month) || !Number.isFinite(day)) return false; - - const date = new Date(Date.UTC(year, month - 1, day)); - return ( - date.getUTCFullYear() === year && date.getUTCMonth() === month - 1 && date.getUTCDate() === day - ); + return undefined; } function resolveSiteName(url: string | undefined): string | undefined { @@ -265,45 +287,81 @@ function resolveSiteName(url: string | undefined): string | undefined { } } -async function runPerplexitySearch(params: { +async function runPerplexitySearchApi(params: { query: string; apiKey: string; - baseUrl: string; - model: string; + count: number; timeoutSeconds: number; -}): Promise<{ content: string; citations: string[] }> { - const endpoint = `${params.baseUrl.replace(/\/$/, "")}/chat/completions`; + country?: string; + searchDomainFilter?: string[]; + searchRecencyFilter?: string; + searchLanguageFilter?: string[]; + searchAfterDate?: string; + searchBeforeDate?: string; + maxTokens?: number; + maxTokensPerPage?: number; +}): Promise< + Array<{ title: string; url: string; description: string; published?: string; siteName?: string }> +> { + const body: Record = { + query: params.query, + max_results: params.count, + }; - const res = await fetch(endpoint, { + if (params.country) { + body.country = params.country; + } + if (params.searchDomainFilter && params.searchDomainFilter.length > 0) { + body.search_domain_filter = params.searchDomainFilter; + } + if (params.searchRecencyFilter) { + body.search_recency_filter = params.searchRecencyFilter; + } + if (params.searchLanguageFilter && params.searchLanguageFilter.length > 0) { + body.search_language_filter = params.searchLanguageFilter; + } + if (params.searchAfterDate) { + body.search_after_date = params.searchAfterDate; + } + if (params.searchBeforeDate) { + body.search_before_date = params.searchBeforeDate; + } + if (params.maxTokens !== undefined) { + body.max_tokens = params.maxTokens; + } + if (params.maxTokensPerPage !== undefined) { + body.max_tokens_per_page = params.maxTokensPerPage; + } + + const res = await fetch(PERPLEXITY_SEARCH_ENDPOINT, { method: "POST", headers: { "Content-Type": "application/json", + Accept: "application/json", Authorization: `Bearer ${params.apiKey}`, "HTTP-Referer": "https://molt.bot", "X-Title": "Moltbot Web Search", }, - body: JSON.stringify({ - model: params.model, - messages: [ - { - role: "user", - content: params.query, - }, - ], - }), + body: JSON.stringify(body), signal: withTimeout(undefined, params.timeoutSeconds * 1000), }); if (!res.ok) { const detail = await readResponseText(res); - throw new Error(`Perplexity API error (${res.status}): ${detail || res.statusText}`); + throw new Error(`Perplexity Search API error (${res.status}): ${detail || res.statusText}`); } - const data = (await res.json()) as PerplexitySearchResponse; - const content = data.choices?.[0]?.message?.content ?? "No response"; - const citations = data.citations ?? []; + const data = (await res.json()) as PerplexitySearchApiResponse; + const results = Array.isArray(data.results) ? data.results : []; - return { content, citations }; + // Map to align formats + return results.map((entry) => ({ + title: entry.title ?? "", + url: entry.url ?? "", + description: entry.snippet ?? "", + published: entry.date ?? undefined, + siteName: resolveSiteName(entry.url ?? ""), + })); } async function runWebSearch(params: { @@ -314,16 +372,17 @@ async function runWebSearch(params: { cacheTtlMs: number; provider: (typeof SEARCH_PROVIDERS)[number]; country?: string; - search_lang?: string; + language?: string; ui_lang?: string; freshness?: string; - perplexityBaseUrl?: string; - perplexityModel?: string; + dateAfter?: string; + dateBefore?: string; + searchDomainFilter?: string[]; + maxTokens?: number; + maxTokensPerPage?: number; }): Promise> { const cacheKey = normalizeCacheKey( - params.provider === "brave" - ? `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}:${params.freshness || "default"}` - : `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}`, + `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.language || "default"}:${params.ui_lang || "default"}:${params.freshness || "default"}:${params.dateAfter || "default"}:${params.dateBefore || "default"}:${params.searchDomainFilter?.join(",") || "default"}:${params.maxTokens || "default"}:${params.maxTokensPerPage || "default"}`, ); const cached = readCache(SEARCH_CACHE, cacheKey); if (cached) return { ...cached.value, cached: true }; @@ -331,21 +390,27 @@ async function runWebSearch(params: { const start = Date.now(); if (params.provider === "perplexity") { - const { content, citations } = await runPerplexitySearch({ + const results = await runPerplexitySearchApi({ query: params.query, apiKey: params.apiKey, - baseUrl: params.perplexityBaseUrl ?? DEFAULT_PERPLEXITY_BASE_URL, - model: params.perplexityModel ?? DEFAULT_PERPLEXITY_MODEL, + count: params.count, timeoutSeconds: params.timeoutSeconds, + country: params.country, + searchDomainFilter: params.searchDomainFilter, + searchRecencyFilter: params.freshness, + searchLanguageFilter: params.language ? [params.language] : undefined, + searchAfterDate: params.dateAfter ? isoToPerplexityDate(params.dateAfter) : undefined, + searchBeforeDate: params.dateBefore ? isoToPerplexityDate(params.dateBefore) : undefined, + maxTokens: params.maxTokens, + maxTokensPerPage: params.maxTokensPerPage, }); const payload = { query: params.query, provider: params.provider, - model: params.perplexityModel ?? DEFAULT_PERPLEXITY_MODEL, + count: results.length, tookMs: Date.now() - start, - content, - citations, + results, }; writeCache(SEARCH_CACHE, cacheKey, payload, params.cacheTtlMs); return payload; @@ -361,14 +426,23 @@ async function runWebSearch(params: { if (params.country) { url.searchParams.set("country", params.country); } - if (params.search_lang) { - url.searchParams.set("search_lang", params.search_lang); + if (params.language) { + url.searchParams.set("search_lang", params.language); } if (params.ui_lang) { url.searchParams.set("ui_lang", params.ui_lang); } if (params.freshness) { url.searchParams.set("freshness", params.freshness); + } else if (params.dateAfter && params.dateBefore) { + url.searchParams.set("freshness", `${params.dateAfter}to${params.dateBefore}`); + } else if (params.dateAfter) { + url.searchParams.set( + "freshness", + `${params.dateAfter}to${new Date().toISOString().slice(0, 10)}`, + ); + } else if (params.dateBefore) { + url.searchParams.set("freshness", `1970-01-01to${params.dateBefore}`); } const res = await fetch(url.toString(), { @@ -418,14 +492,14 @@ export function createWebSearchTool(options?: { const description = provider === "perplexity" - ? "Search the web using Perplexity Sonar (direct or via OpenRouter). Returns AI-synthesized answers with citations from real-time web search." + ? "Search the web using the Perplexity Search API. Returns structured results (title, URL, snippet) for fast research. Supports a wide range of web search configurations including domain and region-specific filtering." : "Search the web using Brave Search API. Supports region-specific and localized search via country and language parameters. Returns titles, URLs, and snippets for fast research."; return { label: "Web Search", name: "web_search", description, - parameters: WebSearchSchema, + parameters: createWebSearchSchema(provider), execute: async (_toolCallId, args) => { const perplexityAuth = provider === "perplexity" ? resolvePerplexityApiKey(perplexityConfig) : undefined; @@ -440,25 +514,39 @@ export function createWebSearchTool(options?: { const count = readNumberParam(params, "count", { integer: true }) ?? search?.maxResults ?? undefined; const country = readStringParam(params, "country"); - const search_lang = readStringParam(params, "search_lang"); + const language = readStringParam(params, "language"); const ui_lang = readStringParam(params, "ui_lang"); const rawFreshness = readStringParam(params, "freshness"); - if (rawFreshness && provider !== "brave") { - return jsonResult({ - error: "unsupported_freshness", - message: "freshness is only supported by the Brave web_search provider.", - docs: "https://docs.molt.bot/tools/web", - }); - } - const freshness = rawFreshness ? normalizeFreshness(rawFreshness) : undefined; + const freshness = rawFreshness ? normalizeFreshness(rawFreshness, provider) : undefined; if (rawFreshness && !freshness) { return jsonResult({ error: "invalid_freshness", - message: - "freshness must be one of pd, pw, pm, py, or a range like YYYY-MM-DDtoYYYY-MM-DD.", + message: "freshness must be day, week, month, or year.", docs: "https://docs.molt.bot/tools/web", }); } + const rawDateAfter = readStringParam(params, "date_after"); + const dateAfter = rawDateAfter ? normalizeToIsoDate(rawDateAfter) : undefined; + if (rawDateAfter && !dateAfter) { + return jsonResult({ + error: "invalid_date", + message: "date_after must be YYYY-MM-DD format.", + docs: "https://docs.molt.bot/tools/web", + }); + } + const rawDateBefore = readStringParam(params, "date_before"); + const dateBefore = rawDateBefore ? normalizeToIsoDate(rawDateBefore) : undefined; + if (rawDateBefore && !dateBefore) { + return jsonResult({ + error: "invalid_date", + message: "date_before must be YYYY-MM-DD format.", + docs: "https://docs.molt.bot/tools/web", + }); + } + const domainFilter = readStringArrayParam(params, "domain_filter"); + const maxTokens = readNumberParam(params, "max_tokens", { integer: true }); + const maxTokensPerPage = readNumberParam(params, "max_tokens_per_page", { integer: true }); + const result = await runWebSearch({ query, count: resolveSearchCount(count, DEFAULT_SEARCH_COUNT), @@ -467,15 +555,14 @@ export function createWebSearchTool(options?: { cacheTtlMs: resolveCacheTtlMs(search?.cacheTtlMinutes, DEFAULT_CACHE_TTL_MINUTES), provider, country, - search_lang, + language, ui_lang, freshness, - perplexityBaseUrl: resolvePerplexityBaseUrl( - perplexityConfig, - perplexityAuth?.source, - perplexityAuth?.apiKey, - ), - perplexityModel: resolvePerplexityModel(perplexityConfig), + dateAfter, + dateBefore, + searchDomainFilter: domainFilter, + maxTokens: maxTokens ?? undefined, + maxTokensPerPage: maxTokensPerPage ?? undefined, }); return jsonResult(result); }, @@ -483,7 +570,10 @@ export function createWebSearchTool(options?: { } export const __testing = { - inferPerplexityBaseUrlFromApiKey, - resolvePerplexityBaseUrl, normalizeFreshness, + normalizeToIsoDate, + isoToPerplexityDate, + SEARCH_CACHE, + FRESHNESS_TO_RECENCY, + RECENCY_TO_FRESHNESS, } as const; diff --git a/src/agents/tools/web-tools.enabled-defaults.test.ts b/src/agents/tools/web-tools.enabled-defaults.test.ts index 41d44b12d..64d2b6b81 100644 --- a/src/agents/tools/web-tools.enabled-defaults.test.ts +++ b/src/agents/tools/web-tools.enabled-defaults.test.ts @@ -1,6 +1,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { createWebFetchTool, createWebSearchTool } from "./web-tools.js"; +import { __testing as webSearchTesting } from "./web-search.js"; describe("web tools defaults", () => { it("enables web_fetch by default (non-sandbox)", () => { @@ -55,7 +56,7 @@ describe("web_search country and language parameters", () => { expect(url.searchParams.get("country")).toBe("DE"); }); - it("should pass search_lang parameter to Brave API", async () => { + it("should pass language parameter to Brave API as search_lang", async () => { const mockFetch = vi.fn(() => Promise.resolve({ ok: true, @@ -66,7 +67,7 @@ describe("web_search country and language parameters", () => { global.fetch = mockFetch; const tool = createWebSearchTool({ config: undefined, sandboxed: true }); - await tool?.execute?.(1, { query: "test", search_lang: "de" }); + await tool?.execute?.(1, { query: "test", language: "de" }); const url = new URL(mockFetch.mock.calls[0][0] as string); expect(url.searchParams.get("search_lang")).toBe("de"); @@ -124,21 +125,33 @@ describe("web_search country and language parameters", () => { }); }); -describe("web_search perplexity baseUrl defaults", () => { +describe("web_search perplexity Search API", () => { const priorFetch = global.fetch; afterEach(() => { vi.unstubAllEnvs(); // @ts-expect-error global fetch cleanup global.fetch = priorFetch; + // Clear search cache to prevent test pollution + webSearchTesting.SEARCH_CACHE.clear(); }); - it("defaults to Perplexity direct when PERPLEXITY_API_KEY is set", async () => { + it("uses Perplexity Search API when PERPLEXITY_API_KEY is set", async () => { vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); const mockFetch = vi.fn(() => Promise.resolve({ ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), + json: () => + Promise.resolve({ + results: [ + { + title: "Test", + url: "https://example.com", + snippet: "Test snippet", + date: "2024-01-01", + }, + ], + }), } as Response), ); // @ts-expect-error mock fetch @@ -148,18 +161,37 @@ describe("web_search perplexity baseUrl defaults", () => { config: { tools: { web: { search: { provider: "perplexity" } } } }, sandboxed: true, }); - await tool?.execute?.(1, { query: "test-openrouter" }); + const result = await tool?.execute?.(1, { query: "test" }); expect(mockFetch).toHaveBeenCalled(); - expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/chat/completions"); + expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/search"); + expect(mockFetch.mock.calls[0]?.[1]?.method).toBe("POST"); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.query).toBe("test"); + expect(result?.details).toMatchObject({ + provider: "perplexity", + results: expect.arrayContaining([ + expect.objectContaining({ title: "Test", url: "https://example.com" }), + ]), + }); }); - it("rejects freshness for Perplexity provider", async () => { + it("does not include freshness parameter for Perplexity provider", async () => { vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); const mockFetch = vi.fn(() => Promise.resolve({ ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), + json: () => + Promise.resolve({ + results: [ + { + title: "Test", + url: "https://example.com", + snippet: "Test snippet", + date: "2024-01-01", + }, + ], + }), } as Response), ); // @ts-expect-error mock fetch @@ -169,19 +201,18 @@ describe("web_search perplexity baseUrl defaults", () => { config: { tools: { web: { search: { provider: "perplexity" } } } }, sandboxed: true, }); - const result = await tool?.execute?.(1, { query: "test", freshness: "pw" }); + const result = await tool?.execute?.(1, { query: "test" }); - expect(mockFetch).not.toHaveBeenCalled(); - expect(result?.details).toMatchObject({ error: "unsupported_freshness" }); + expect(mockFetch).toHaveBeenCalled(); + expect(result?.details).toMatchObject({ provider: "perplexity", count: 1 }); }); - it("defaults to OpenRouter when OPENROUTER_API_KEY is set", async () => { - vi.stubEnv("PERPLEXITY_API_KEY", ""); - vi.stubEnv("OPENROUTER_API_KEY", "sk-or-test"); + it("passes country parameter to Perplexity Search API", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); const mockFetch = vi.fn(() => Promise.resolve({ ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), + json: () => Promise.resolve({ results: [] }), } as Response), ); // @ts-expect-error mock fetch @@ -191,69 +222,18 @@ describe("web_search perplexity baseUrl defaults", () => { config: { tools: { web: { search: { provider: "perplexity" } } } }, sandboxed: true, }); - await tool?.execute?.(1, { query: "test-openrouter-env" }); + await tool?.execute?.(1, { query: "test", country: "DE" }); expect(mockFetch).toHaveBeenCalled(); - expect(mockFetch.mock.calls[0]?.[0]).toBe("https://openrouter.ai/api/v1/chat/completions"); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.country).toBe("DE"); }); - it("prefers PERPLEXITY_API_KEY when both env keys are set", async () => { - vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); - vi.stubEnv("OPENROUTER_API_KEY", "sk-or-test"); + it("uses config API key when provided", async () => { const mockFetch = vi.fn(() => Promise.resolve({ ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), - } as Response), - ); - // @ts-expect-error mock fetch - global.fetch = mockFetch; - - const tool = createWebSearchTool({ - config: { tools: { web: { search: { provider: "perplexity" } } } }, - sandboxed: true, - }); - await tool?.execute?.(1, { query: "test-both-env" }); - - expect(mockFetch).toHaveBeenCalled(); - expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/chat/completions"); - }); - - it("uses configured baseUrl even when PERPLEXITY_API_KEY is set", async () => { - vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); - const mockFetch = vi.fn(() => - Promise.resolve({ - ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), - } as Response), - ); - // @ts-expect-error mock fetch - global.fetch = mockFetch; - - const tool = createWebSearchTool({ - config: { - tools: { - web: { - search: { - provider: "perplexity", - perplexity: { baseUrl: "https://example.com/pplx" }, - }, - }, - }, - }, - sandboxed: true, - }); - await tool?.execute?.(1, { query: "test-config-baseurl" }); - - expect(mockFetch).toHaveBeenCalled(); - expect(mockFetch.mock.calls[0]?.[0]).toBe("https://example.com/pplx/chat/completions"); - }); - - it("defaults to Perplexity direct when apiKey looks like Perplexity", async () => { - const mockFetch = vi.fn(() => - Promise.resolve({ - ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), + json: () => Promise.resolve({ results: [] }), } as Response), ); // @ts-expect-error mock fetch @@ -272,38 +252,186 @@ describe("web_search perplexity baseUrl defaults", () => { }, sandboxed: true, }); - await tool?.execute?.(1, { query: "test-config-apikey" }); + await tool?.execute?.(1, { query: "test" }); expect(mockFetch).toHaveBeenCalled(); - expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/chat/completions"); + const headers = mockFetch.mock.calls[0]?.[1]?.headers; + const authHeader = + typeof headers?.get === "function" ? headers.get("Authorization") : headers?.Authorization; + expect(authHeader).toBe("Bearer pplx-config"); }); - it("defaults to OpenRouter when apiKey looks like OpenRouter", async () => { + it("passes freshness filter to Perplexity Search API", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); const mockFetch = vi.fn(() => Promise.resolve({ ok: true, - json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }), + json: () => Promise.resolve({ results: [] }), } as Response), ); // @ts-expect-error mock fetch global.fetch = mockFetch; const tool = createWebSearchTool({ - config: { - tools: { - web: { - search: { - provider: "perplexity", - perplexity: { apiKey: "sk-or-v1-test" }, - }, - }, - }, - }, + config: { tools: { web: { search: { provider: "perplexity" } } } }, sandboxed: true, }); - await tool?.execute?.(1, { query: "test-openrouter-config" }); + await tool?.execute?.(1, { query: "test", freshness: "week" }); expect(mockFetch).toHaveBeenCalled(); - expect(mockFetch.mock.calls[0]?.[0]).toBe("https://openrouter.ai/api/v1/chat/completions"); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.search_recency_filter).toBe("week"); + }); + + it("accepts all valid freshness values for Perplexity", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); + const mockFetch = vi.fn(() => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ results: [] }), + } as Response), + ); + // @ts-expect-error mock fetch + global.fetch = mockFetch; + + const tool = createWebSearchTool({ + config: { tools: { web: { search: { provider: "perplexity" } } } }, + sandboxed: true, + }); + + for (const freshness of ["day", "week", "month", "year"]) { + webSearchTesting.SEARCH_CACHE.clear(); + await tool?.execute?.(1, { query: `test-${freshness}`, freshness }); + const body = JSON.parse(mockFetch.mock.calls.at(-1)?.[1]?.body as string); + expect(body.search_recency_filter).toBe(freshness); + } + }); + + it("rejects invalid freshness values", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); + const mockFetch = vi.fn(() => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ results: [] }), + } as Response), + ); + // @ts-expect-error mock fetch + global.fetch = mockFetch; + + const tool = createWebSearchTool({ + config: { tools: { web: { search: { provider: "perplexity" } } } }, + sandboxed: true, + }); + const result = await tool?.execute?.(1, { query: "test", freshness: "yesterday" }); + + expect(mockFetch).not.toHaveBeenCalled(); + expect(result?.details).toMatchObject({ error: "invalid_freshness" }); + }); + + it("passes domain filter to Perplexity Search API", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); + const mockFetch = vi.fn(() => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ results: [] }), + } as Response), + ); + // @ts-expect-error mock fetch + global.fetch = mockFetch; + + const tool = createWebSearchTool({ + config: { tools: { web: { search: { provider: "perplexity" } } } }, + sandboxed: true, + }); + await tool?.execute?.(1, { + query: "test", + domain_filter: ["nature.com", "science.org"], + }); + + expect(mockFetch).toHaveBeenCalled(); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.search_domain_filter).toEqual(["nature.com", "science.org"]); + }); + + it("passes denylist domain filter to Perplexity Search API", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); + const mockFetch = vi.fn(() => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ results: [] }), + } as Response), + ); + // @ts-expect-error mock fetch + global.fetch = mockFetch; + + const tool = createWebSearchTool({ + config: { tools: { web: { search: { provider: "perplexity" } } } }, + sandboxed: true, + }); + await tool?.execute?.(1, { + query: "test", + domain_filter: ["-reddit.com", "-pinterest.com"], + }); + + expect(mockFetch).toHaveBeenCalled(); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.search_domain_filter).toEqual(["-reddit.com", "-pinterest.com"]); + }); + + it("passes language to Perplexity Search API as search_language_filter array", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); + const mockFetch = vi.fn(() => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ results: [] }), + } as Response), + ); + // @ts-expect-error mock fetch + global.fetch = mockFetch; + + const tool = createWebSearchTool({ + config: { tools: { web: { search: { provider: "perplexity" } } } }, + sandboxed: true, + }); + await tool?.execute?.(1, { + query: "test", + language: "en", + }); + + expect(mockFetch).toHaveBeenCalled(); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.search_language_filter).toEqual(["en"]); + }); + + it("passes multiple filters together to Perplexity Search API", async () => { + vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test"); + const mockFetch = vi.fn(() => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ results: [] }), + } as Response), + ); + // @ts-expect-error mock fetch + global.fetch = mockFetch; + + const tool = createWebSearchTool({ + config: { tools: { web: { search: { provider: "perplexity" } } } }, + sandboxed: true, + }); + await tool?.execute?.(1, { + query: "climate research", + country: "US", + freshness: "month", + domain_filter: ["nature.com", ".gov"], + language: "en", + }); + + expect(mockFetch).toHaveBeenCalled(); + const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string); + expect(body.query).toBe("climate research"); + expect(body.country).toBe("US"); + expect(body.search_recency_filter).toBe("month"); + expect(body.search_domain_filter).toEqual(["nature.com", ".gov"]); + expect(body.search_language_filter).toEqual(["en"]); }); }); diff --git a/src/commands/configure.shared.ts b/src/commands/configure.shared.ts index bc89529d8..5d38e7b0f 100644 --- a/src/commands/configure.shared.ts +++ b/src/commands/configure.shared.ts @@ -35,7 +35,7 @@ export const CONFIGURE_SECTION_OPTIONS: Array<{ }> = [ { value: "workspace", label: "Workspace", hint: "Set workspace + sessions" }, { value: "model", label: "Model", hint: "Pick provider + credentials" }, - { value: "web", label: "Web tools", hint: "Configure Brave search + fetch" }, + { value: "web", label: "Web tools", hint: "Configure web search (Perplexity/Brave) + fetch" }, { value: "gateway", label: "Gateway", hint: "Port, bind, auth, tailscale" }, { value: "daemon", diff --git a/src/commands/configure.wizard.ts b/src/commands/configure.wizard.ts index 8db1d8379..4075f7e35 100644 --- a/src/commands/configure.wizard.ts +++ b/src/commands/configure.wizard.ts @@ -94,12 +94,18 @@ async function promptWebToolsConfig( ): Promise { const existingSearch = nextConfig.tools?.web?.search; const existingFetch = nextConfig.tools?.web?.fetch; - const hasSearchKey = Boolean(existingSearch?.apiKey); + const existingProvider = existingSearch?.provider ?? "brave"; + const hasPerplexityKey = Boolean( + existingSearch?.perplexity?.apiKey || process.env.PERPLEXITY_API_KEY, + ); + const hasBraveKey = Boolean(existingSearch?.apiKey || process.env.BRAVE_API_KEY); + const hasSearchKey = existingProvider === "perplexity" ? hasPerplexityKey : hasBraveKey; note( [ "Web search lets your agent look things up online using the `web_search` tool.", - "It requires a Brave Search API key (you can store it in the config or set BRAVE_API_KEY in the Gateway environment).", + "Choose a provider: Perplexity Search (recommended) or Brave Search.", + "Both return structured results (title, URL, snippet) for fast research.", "Docs: https://docs.molt.bot/tools/web", ].join("\n"), "Web search", @@ -107,7 +113,7 @@ async function promptWebToolsConfig( const enableSearch = guardCancel( await confirm({ - message: "Enable web_search (Brave Search)?", + message: "Enable web_search?", initialValue: existingSearch?.enabled ?? hasSearchKey, }), runtime, @@ -119,27 +125,79 @@ async function promptWebToolsConfig( }; if (enableSearch) { - const keyInput = guardCancel( - await text({ - message: hasSearchKey - ? "Brave Search API key (leave blank to keep current or use BRAVE_API_KEY)" - : "Brave Search API key (paste it here; leave blank to use BRAVE_API_KEY)", - placeholder: hasSearchKey ? "Leave blank to keep current" : "BSA...", + const providerChoice = guardCancel( + await select({ + message: "Choose web search provider", + options: [ + { + value: "perplexity", + label: "Perplexity Search", + }, + { + value: "brave", + label: "Brave Search", + }, + ], + initialValue: existingProvider, }), runtime, ); - const key = String(keyInput ?? "").trim(); - if (key) { - nextSearch = { ...nextSearch, apiKey: key }; - } else if (!hasSearchKey) { - note( - [ - "No key stored yet, so web_search will stay unavailable.", - "Store a key here or set BRAVE_API_KEY in the Gateway environment.", - "Docs: https://docs.molt.bot/tools/web", - ].join("\n"), - "Web search", + + nextSearch = { ...nextSearch, provider: providerChoice }; + + if (providerChoice === "perplexity") { + const hasKey = Boolean(existingSearch?.perplexity?.apiKey); + const keyInput = guardCancel( + await text({ + message: hasKey + ? "Perplexity API key (leave blank to keep current or use PERPLEXITY_API_KEY)" + : "Perplexity API key (paste it here; leave blank to use PERPLEXITY_API_KEY)", + placeholder: hasKey ? "Leave blank to keep current" : "pplx-...", + }), + runtime, ); + const key = String(keyInput ?? "").trim(); + if (key) { + nextSearch = { + ...nextSearch, + perplexity: { ...existingSearch?.perplexity, apiKey: key }, + }; + } else if (!hasKey && !process.env.PERPLEXITY_API_KEY) { + note( + [ + "No key stored yet, so web_search will stay unavailable.", + "Store a key here or set PERPLEXITY_API_KEY in the Gateway environment.", + "Get your API key at: https://www.perplexity.ai/settings/api", + "Docs: https://docs.molt.bot/tools/web", + ].join("\n"), + "Web search", + ); + } + } else { + const hasKey = Boolean(existingSearch?.apiKey); + const keyInput = guardCancel( + await text({ + message: hasKey + ? "Brave Search API key (leave blank to keep current or use BRAVE_API_KEY)" + : "Brave Search API key (paste it here; leave blank to use BRAVE_API_KEY)", + placeholder: hasKey ? "Leave blank to keep current" : "BSA...", + }), + runtime, + ); + const key = String(keyInput ?? "").trim(); + if (key) { + nextSearch = { ...nextSearch, apiKey: key }; + } else if (!hasKey && !process.env.BRAVE_API_KEY) { + note( + [ + "No key stored yet, so web_search will stay unavailable.", + "Store a key here or set BRAVE_API_KEY in the Gateway environment.", + "Get your API key at: https://brave.com/search/api/", + "Docs: https://docs.molt.bot/tools/web", + ].join("\n"), + "Web search", + ); + } } } diff --git a/src/config/config.web-search-provider.test.ts b/src/config/config.web-search-provider.test.ts index a91c0f438..116dc843f 100644 --- a/src/config/config.web-search-provider.test.ts +++ b/src/config/config.web-search-provider.test.ts @@ -12,8 +12,6 @@ describe("web search provider config", () => { provider: "perplexity", perplexity: { apiKey: "test-key", - baseUrl: "https://api.perplexity.ai", - model: "perplexity/sonar-pro", }, }, }, diff --git a/src/config/schema.ts b/src/config/schema.ts index 28c994f3d..a919738cf 100644 --- a/src/config/schema.ts +++ b/src/config/schema.ts @@ -441,11 +441,7 @@ const FIELD_HELP: Record = { "tools.web.search.timeoutSeconds": "Timeout in seconds for web_search requests.", "tools.web.search.cacheTtlMinutes": "Cache TTL in minutes for web_search results.", "tools.web.search.perplexity.apiKey": - "Perplexity or OpenRouter API key (fallback: PERPLEXITY_API_KEY or OPENROUTER_API_KEY env var).", - "tools.web.search.perplexity.baseUrl": - "Perplexity base URL override (default: https://openrouter.ai/api/v1 or https://api.perplexity.ai).", - "tools.web.search.perplexity.model": - 'Perplexity model override (default: "perplexity/sonar-pro").', + "Perplexity API key (fallback: PERPLEXITY_API_KEY env var).", "tools.web.fetch.enabled": "Enable the web_fetch tool (lightweight HTTP fetch).", "tools.web.fetch.maxChars": "Max characters returned by web_fetch (truncated).", "tools.web.fetch.timeoutSeconds": "Timeout in seconds for web_fetch requests.", diff --git a/src/config/types.tools.ts b/src/config/types.tools.ts index db32cb59d..1c011b291 100644 --- a/src/config/types.tools.ts +++ b/src/config/types.tools.ts @@ -348,12 +348,8 @@ export type ToolsConfig = { cacheTtlMinutes?: number; /** Perplexity-specific configuration (used when provider="perplexity"). */ perplexity?: { - /** API key for Perplexity or OpenRouter (defaults to PERPLEXITY_API_KEY or OPENROUTER_API_KEY env var). */ + /** API key for Perplexity (defaults to PERPLEXITY_API_KEY env var). */ apiKey?: string; - /** Base URL for API requests (defaults to OpenRouter: https://openrouter.ai/api/v1). */ - baseUrl?: string; - /** Model to use (defaults to "perplexity/sonar-pro"). */ - model?: string; }; }; fetch?: { diff --git a/src/config/zod-schema.agent-runtime.ts b/src/config/zod-schema.agent-runtime.ts index 7e95c3538..9b2b903ad 100644 --- a/src/config/zod-schema.agent-runtime.ts +++ b/src/config/zod-schema.agent-runtime.ts @@ -173,8 +173,6 @@ export const ToolsWebSearchSchema = z perplexity: z .object({ apiKey: z.string().optional(), - baseUrl: z.string().optional(), - model: z.string().optional(), }) .strict() .optional(), diff --git a/src/security/audit-extra.ts b/src/security/audit-extra.ts index 3a92a30a8..9d4c94d8b 100644 --- a/src/security/audit-extra.ts +++ b/src/security/audit-extra.ts @@ -408,11 +408,7 @@ function resolveToolPolicies(params: { function hasWebSearchKey(cfg: MoltbotConfig, env: NodeJS.ProcessEnv): boolean { const search = cfg.tools?.web?.search; return Boolean( - search?.apiKey || - search?.perplexity?.apiKey || - env.BRAVE_API_KEY || - env.PERPLEXITY_API_KEY || - env.OPENROUTER_API_KEY, + search?.apiKey || search?.perplexity?.apiKey || env.BRAVE_API_KEY || env.PERPLEXITY_API_KEY, ); } diff --git a/src/wizard/onboarding.finalize.ts b/src/wizard/onboarding.finalize.ts index 96a4a4bf6..c31f7d7f8 100644 --- a/src/wizard/onboarding.finalize.ts +++ b/src/wizard/onboarding.finalize.ts @@ -432,29 +432,35 @@ export async function finalizeOnboardingWizard(options: FinalizeOnboardingOption ); } - const webSearchKey = (nextConfig.tools?.web?.search?.apiKey ?? "").trim(); - const webSearchEnv = (process.env.BRAVE_API_KEY ?? "").trim(); + const webSearchProvider = nextConfig.tools?.web?.search?.provider ?? "brave"; + const webSearchKey = + webSearchProvider === "perplexity" + ? (nextConfig.tools?.web?.search?.perplexity?.apiKey ?? "").trim() + : (nextConfig.tools?.web?.search?.apiKey ?? "").trim(); + const webSearchEnv = + webSearchProvider === "perplexity" + ? (process.env.PERPLEXITY_API_KEY ?? "").trim() + : (process.env.BRAVE_API_KEY ?? "").trim(); const hasWebSearchKey = Boolean(webSearchKey || webSearchEnv); await prompter.note( hasWebSearchKey ? [ "Web search is enabled, so your agent can look things up online when needed.", "", + `Provider: ${webSearchProvider === "perplexity" ? "Perplexity Search" : "Brave Search"}`, webSearchKey - ? "API key: stored in config (tools.web.search.apiKey)." - : "API key: provided via BRAVE_API_KEY env var (Gateway environment).", + ? `API key: stored in config (tools.web.search.${webSearchProvider === "perplexity" ? "perplexity.apiKey" : "apiKey"}).` + : `API key: provided via ${webSearchProvider === "perplexity" ? "PERPLEXITY_API_KEY" : "BRAVE_API_KEY"} env var (Gateway environment).`, "Docs: https://docs.molt.bot/tools/web", ].join("\n") : [ - "If you want your agent to be able to search the web, you’ll need an API key.", - "", - "Moltbot uses Brave Search for the `web_search` tool. Without a Brave Search API key, web search won’t work.", + "To enable web search, your agent will need an API key for either Perplexity Search or Brave Search.", "", "Set it up interactively:", `- Run: ${formatCliCommand("moltbot configure --section web")}`, - "- Enable web_search and paste your Brave Search API key", + "- Choose a provider and paste your API key", "", - "Alternative: set BRAVE_API_KEY in the Gateway environment (no config changes).", + "Alternative: set PERPLEXITY_API_KEY or BRAVE_API_KEY in the Gateway environment (no config changes).", "Docs: https://docs.molt.bot/tools/web", ].join("\n"), "Web search (optional)",