Merge 0b44fcaacb into 4583f88626
This commit is contained in:
Kesku
GitHub
commit
24a9b8002c
@ -7,7 +7,7 @@ read_when:
|
||||
|
||||
# Brave Search API
|
||||
|
||||
Moltbot uses Brave Search as the default provider for `web_search`.
|
||||
Moltbot supports Brave Search as a web search provider for `web_search`.
|
||||
|
||||
## Get an API key
|
||||
|
||||
@ -32,9 +32,47 @@ Moltbot uses Brave Search as the default provider for `web_search`.
|
||||
}
|
||||
```
|
||||
|
||||
## Tool parameters
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| `query` | Search query (required) |
|
||||
| `count` | Number of results to return (1-10, default: 5) |
|
||||
| `country` | 2-letter ISO country code (e.g., "US", "DE") |
|
||||
| `language` | ISO 639-1 language code for search results (e.g., "en", "de", "fr") |
|
||||
| `ui_lang` | ISO language code for UI elements |
|
||||
| `freshness` | Time filter: `day` (24h), `week`, `month`, or `year` |
|
||||
| `date_after` | Only results published after this date (YYYY-MM-DD) |
|
||||
| `date_before` | Only results published before this date (YYYY-MM-DD) |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```javascript
|
||||
// Country and language-specific search
|
||||
await web_search({
|
||||
query: "renewable energy",
|
||||
country: "DE",
|
||||
language: "de"
|
||||
});
|
||||
|
||||
// Recent results (past week)
|
||||
await web_search({
|
||||
query: "AI news",
|
||||
freshness: "week"
|
||||
});
|
||||
|
||||
// Date range search
|
||||
await web_search({
|
||||
query: "AI developments",
|
||||
date_after: "2024-01-01",
|
||||
date_before: "2024-06-30"
|
||||
});
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- The Data for AI plan is **not** compatible with `web_search`.
|
||||
- Brave provides a free tier plus paid plans; check the Brave API portal for current limits.
|
||||
- Results are cached for 15 minutes by default (configurable via `cacheTtlMinutes`).
|
||||
|
||||
See [Web tools](/tools/web) for the full web_search configuration.
|
||||
|
||||
@ -1,27 +1,20 @@
|
||||
---
|
||||
summary: "Perplexity Sonar setup for web_search"
|
||||
summary: "Perplexity Search API setup for web_search"
|
||||
read_when:
|
||||
- You want to use Perplexity Sonar for web search
|
||||
- You need PERPLEXITY_API_KEY or OpenRouter setup
|
||||
- You want to use Perplexity Search for web search
|
||||
- You need PERPLEXITY_API_KEY setup
|
||||
---
|
||||
|
||||
# Perplexity Sonar
|
||||
# Perplexity Search API
|
||||
|
||||
Moltbot can use Perplexity Sonar for the `web_search` tool. You can connect
|
||||
through Perplexity’s direct API or via OpenRouter.
|
||||
Moltbot uses Perplexity Search API for the `web_search` tool when `provider: "perplexity"` is set.
|
||||
Perplexity Search returns structured results (title, URL, snippet) for fast research.
|
||||
|
||||
## API options
|
||||
## Getting a Perplexity API key
|
||||
|
||||
### Perplexity (direct)
|
||||
|
||||
- Base URL: https://api.perplexity.ai
|
||||
- Environment variable: `PERPLEXITY_API_KEY`
|
||||
|
||||
### OpenRouter (alternative)
|
||||
|
||||
- Base URL: https://openrouter.ai/api/v1
|
||||
- Environment variable: `OPENROUTER_API_KEY`
|
||||
- Supports prepaid/crypto credits.
|
||||
1) Create a Perplexity account at https://www.perplexity.ai/settings/api
|
||||
2) Generate an API key in the dashboard
|
||||
3) Store the key in config (recommended) or set `PERPLEXITY_API_KEY` in the Gateway environment.
|
||||
|
||||
## Config example
|
||||
|
||||
@ -32,9 +25,7 @@ through Perplexity’s direct API or via OpenRouter.
|
||||
search: {
|
||||
provider: "perplexity",
|
||||
perplexity: {
|
||||
apiKey: "pplx-...",
|
||||
baseUrl: "https://api.perplexity.ai",
|
||||
model: "perplexity/sonar-pro"
|
||||
apiKey: "pplx-..."
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -51,8 +42,7 @@ through Perplexity’s direct API or via OpenRouter.
|
||||
search: {
|
||||
provider: "perplexity",
|
||||
perplexity: {
|
||||
apiKey: "pplx-...",
|
||||
baseUrl: "https://api.perplexity.ai"
|
||||
apiKey: "pplx-..."
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -60,20 +50,83 @@ through Perplexity’s direct API or via OpenRouter.
|
||||
}
|
||||
```
|
||||
|
||||
If both `PERPLEXITY_API_KEY` and `OPENROUTER_API_KEY` are set, set
|
||||
`tools.web.search.perplexity.baseUrl` (or `tools.web.search.perplexity.apiKey`)
|
||||
to disambiguate.
|
||||
## Where to set the key (recommended)
|
||||
|
||||
If no base URL is set, Moltbot chooses a default based on the API key source:
|
||||
**Recommended:** run `moltbot configure --section web`. It stores the key in
|
||||
`~/.clawdbot/moltbot.json` under `tools.web.search.perplexity.apiKey`.
|
||||
|
||||
- `PERPLEXITY_API_KEY` or `pplx-...` → direct Perplexity (`https://api.perplexity.ai`)
|
||||
- `OPENROUTER_API_KEY` or `sk-or-...` → OpenRouter (`https://openrouter.ai/api/v1`)
|
||||
- Unknown key formats → OpenRouter (safe fallback)
|
||||
**Environment alternative:** set `PERPLEXITY_API_KEY` in the Gateway process
|
||||
environment. For a gateway install, put it in `~/.clawdbot/.env` (or your
|
||||
service environment). See [Env vars](/help/faq#how-does-moltbot-load-environment-variables).
|
||||
|
||||
## Models
|
||||
## Tool parameters
|
||||
|
||||
- `perplexity/sonar` — fast Q&A with web search
|
||||
- `perplexity/sonar-pro` (default) — multi-step reasoning + web search
|
||||
- `perplexity/sonar-reasoning-pro` — deep research
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| `query` | Search query (required) |
|
||||
| `count` | Number of results to return (1-10, default: 5) |
|
||||
| `country` | 2-letter ISO country code (e.g., "US", "DE") |
|
||||
| `language` | ISO 639-1 language code (e.g., "en", "de", "fr") |
|
||||
| `freshness` | Time filter: `day` (24h), `week`, `month`, or `year` |
|
||||
| `date_after` | Only results published after this date (YYYY-MM-DD) |
|
||||
| `date_before` | Only results published before this date (YYYY-MM-DD) |
|
||||
| `domain_filter` | Domain allowlist/denylist array (max 20) |
|
||||
| `max_tokens` | Total content budget (default: 25000, max: 1000000) |
|
||||
| `max_tokens_per_page` | Per-page token limit (default: 2048) |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```javascript
|
||||
// Country and language-specific search
|
||||
await web_search({
|
||||
query: "renewable energy",
|
||||
country: "DE",
|
||||
language: "de"
|
||||
});
|
||||
|
||||
// Recent results (past week)
|
||||
await web_search({
|
||||
query: "AI news",
|
||||
freshness: "week"
|
||||
});
|
||||
|
||||
// Date range search
|
||||
await web_search({
|
||||
query: "AI developments",
|
||||
date_after: "2024-01-01",
|
||||
date_before: "2024-06-30"
|
||||
});
|
||||
|
||||
// Domain filtering (allowlist)
|
||||
await web_search({
|
||||
query: "climate research",
|
||||
domain_filter: ["nature.com", "science.org", ".edu"]
|
||||
});
|
||||
|
||||
// Domain filtering (denylist - prefix with -)
|
||||
await web_search({
|
||||
query: "product reviews",
|
||||
domain_filter: ["-reddit.com", "-pinterest.com"]
|
||||
});
|
||||
|
||||
// More content extraction
|
||||
await web_search({
|
||||
query: "detailed AI research",
|
||||
max_tokens: 50000,
|
||||
max_tokens_per_page: 4096
|
||||
});
|
||||
```
|
||||
|
||||
### Domain filter rules
|
||||
|
||||
- Maximum 20 domains per filter
|
||||
- Cannot mix allowlist and denylist in the same request
|
||||
- Use `-` prefix for denylist entries (e.g., `["-reddit.com"]`)
|
||||
|
||||
## Notes
|
||||
|
||||
- Perplexity Search API returns structured web search results (title, URL, snippet)
|
||||
- Results are cached for 15 minutes by default (configurable via `cacheTtlMinutes`)
|
||||
|
||||
See [Web tools](/tools/web) for the full web_search configuration.
|
||||
See [Perplexity Search API docs](https://docs.perplexity.ai/guides/search-quickstart) for more details.
|
||||
|
||||
@ -1,16 +1,15 @@
|
||||
---
|
||||
summary: "Web search + fetch tools (Brave Search API, Perplexity direct/OpenRouter)"
|
||||
summary: "Web search + fetch tools (Perplexity Search API, Brave Search API)"
|
||||
read_when:
|
||||
- You want to enable web_search or web_fetch
|
||||
- You need Brave Search API key setup
|
||||
- You want to use Perplexity Sonar for web search
|
||||
- You need Perplexity or Brave Search API key setup
|
||||
---
|
||||
|
||||
# Web tools
|
||||
|
||||
Moltbot ships two lightweight web tools:
|
||||
|
||||
- `web_search` — Search the web via Brave Search API (default) or Perplexity Sonar (direct or via OpenRouter).
|
||||
- `web_search` — Search the web using Perplexity Search API or Brave Search API.
|
||||
- `web_fetch` — HTTP fetch + readable extraction (HTML → markdown/text).
|
||||
|
||||
These are **not** browser automation. For JS-heavy sites or logins, use the
|
||||
@ -19,21 +18,20 @@ These are **not** browser automation. For JS-heavy sites or logins, use the
|
||||
## How it works
|
||||
|
||||
- `web_search` calls your configured provider and returns results.
|
||||
- **Brave** (default): returns structured results (title, URL, snippet).
|
||||
- **Perplexity**: returns AI-synthesized answers with citations from real-time web search.
|
||||
- Results are cached by query for 15 minutes (configurable).
|
||||
- `web_fetch` does a plain HTTP GET and extracts readable content
|
||||
(HTML → markdown/text). It does **not** execute JavaScript.
|
||||
- `web_fetch` is enabled by default (unless explicitly disabled).
|
||||
|
||||
See [Perplexity Search setup](/perplexity) and [Brave Search setup](/brave-search) for provider-specific details.
|
||||
|
||||
## Choosing a search provider
|
||||
|
||||
| Provider | Pros | Cons | API Key |
|
||||
|----------|------|------|---------|
|
||||
| **Brave** (default) | Fast, structured results, free tier | Traditional search results | `BRAVE_API_KEY` |
|
||||
| **Perplexity** | AI-synthesized answers, citations, real-time | Requires Perplexity or OpenRouter access | `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` |
|
||||
| Provider | Pros | Cons | API Key |
|
||||
|---------------------------|----------------------------------------------------------------------------------------------|-------------------------------|------------------------------------------------|
|
||||
| **Perplexity Search API** | Fast, structured results; domain, language, region, and freshness filters; content extraction options; free credits for Moltbot users | — | Requires Perplexity API key `PERPLEXITY_API_KEY` |
|
||||
| **Brave Search API** | Fast, structured results; free tier available | Fewer filtering options | Requires Brave API key `BRAVE_API_KEY` |
|
||||
|
||||
See [Brave Search setup](/brave-search) and [Perplexity Sonar](/perplexity) for provider-specific details.
|
||||
|
||||
Set the provider in config:
|
||||
|
||||
@ -42,63 +40,44 @@ Set the provider in config:
|
||||
tools: {
|
||||
web: {
|
||||
search: {
|
||||
provider: "brave" // or "perplexity"
|
||||
provider: "perplexity" // or "brave"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Example: switch to Perplexity Sonar (direct API):
|
||||
## Setting up web search
|
||||
|
||||
```json5
|
||||
{
|
||||
tools: {
|
||||
web: {
|
||||
search: {
|
||||
provider: "perplexity",
|
||||
perplexity: {
|
||||
apiKey: "pplx-...",
|
||||
baseUrl: "https://api.perplexity.ai",
|
||||
model: "perplexity/sonar-pro"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Use `moltbot configure --section web` to set up your API key and choose a provider.
|
||||
|
||||
## Getting a Brave API key
|
||||
### Perplexity Search
|
||||
|
||||
1) Create a Perplexity account at https://www.perplexity.ai/settings/api
|
||||
2) Generate an API key in the dashboard
|
||||
3) Run `moltbot configure --section web` to store the key in config, or set `PERPLEXITY_API_KEY` in your environment.
|
||||
|
||||
Perplexity provides $5 in API credits on a monthly rolling basis to Perplexity Pro subscribers. Additionally, Perplexity provides complementary credits for Moltbot users.
|
||||
|
||||
See [Perplexity Search API Docs](https://docs.perplexity.ai/guides/search-quickstart) for more details.
|
||||
|
||||
### Brave Search
|
||||
|
||||
1) Create a Brave Search API account at https://brave.com/search/api/
|
||||
2) In the dashboard, choose the **Data for Search** plan (not “Data for AI”) and generate an API key.
|
||||
2) In the dashboard, choose the **Data for Search** plan (not "Data for AI") and generate an API key.
|
||||
3) Run `moltbot configure --section web` to store the key in config (recommended), or set `BRAVE_API_KEY` in your environment.
|
||||
|
||||
Brave provides a free tier plus paid plans; check the Brave API portal for the
|
||||
current limits and pricing.
|
||||
Brave provides a free tier plus paid plans; check the Brave API portal for the current limits and pricing.
|
||||
|
||||
### Where to set the key (recommended)
|
||||
### Where to store the key
|
||||
|
||||
**Recommended:** run `moltbot configure --section web`. It stores the key in
|
||||
`~/.clawdbot/moltbot.json` under `tools.web.search.apiKey`.
|
||||
**Via config (recommended):** run `moltbot configure --section web`. It stores the key under `tools.web.search.perplexity.apiKey` or `tools.web.search.apiKey`.
|
||||
|
||||
**Environment alternative:** set `BRAVE_API_KEY` in the Gateway process
|
||||
environment. For a gateway install, put it in `~/.clawdbot/.env` (or your
|
||||
service environment). See [Env vars](/help/faq#how-does-moltbot-load-environment-variables).
|
||||
**Via environment:** set `PERPLEXITY_API_KEY` or `BRAVE_API_KEY` in the Gateway process environment. For a gateway install, put it in `~/.clawdbot/.env` (or your service environment). See [Env vars](/help/faq#how-does-moltbot-load-environment-variables).
|
||||
|
||||
## Using Perplexity (direct or via OpenRouter)
|
||||
### Config examples
|
||||
|
||||
Perplexity Sonar models have built-in web search capabilities and return AI-synthesized
|
||||
answers with citations. You can use them via OpenRouter (no credit card required - supports
|
||||
crypto/prepaid).
|
||||
|
||||
### Getting an OpenRouter API key
|
||||
|
||||
1) Create an account at https://openrouter.ai/
|
||||
2) Add credits (supports crypto, prepaid, or credit card)
|
||||
3) Generate an API key in your account settings
|
||||
|
||||
### Setting up Perplexity search
|
||||
**Perplexity Search:**
|
||||
|
||||
```json5
|
||||
{
|
||||
@ -108,12 +87,7 @@ crypto/prepaid).
|
||||
enabled: true,
|
||||
provider: "perplexity",
|
||||
perplexity: {
|
||||
// API key (optional if OPENROUTER_API_KEY or PERPLEXITY_API_KEY is set)
|
||||
apiKey: "sk-or-v1-...",
|
||||
// Base URL (key-aware default if omitted)
|
||||
baseUrl: "https://openrouter.ai/api/v1",
|
||||
// Model (defaults to perplexity/sonar-pro)
|
||||
model: "perplexity/sonar-pro"
|
||||
apiKey: "pplx-..." // optional if PERPLEXITY_API_KEY is set
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -121,22 +95,21 @@ crypto/prepaid).
|
||||
}
|
||||
```
|
||||
|
||||
**Environment alternative:** set `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` in the Gateway
|
||||
environment. For a gateway install, put it in `~/.clawdbot/.env`.
|
||||
**Brave Search:**
|
||||
|
||||
If no base URL is set, Moltbot chooses a default based on the API key source:
|
||||
|
||||
- `PERPLEXITY_API_KEY` or `pplx-...` → `https://api.perplexity.ai`
|
||||
- `OPENROUTER_API_KEY` or `sk-or-...` → `https://openrouter.ai/api/v1`
|
||||
- Unknown key formats → OpenRouter (safe fallback)
|
||||
|
||||
### Available Perplexity models
|
||||
|
||||
| Model | Description | Best for |
|
||||
|-------|-------------|----------|
|
||||
| `perplexity/sonar` | Fast Q&A with web search | Quick lookups |
|
||||
| `perplexity/sonar-pro` (default) | Multi-step reasoning with web search | Complex questions |
|
||||
| `perplexity/sonar-reasoning-pro` | Chain-of-thought analysis | Deep research |
|
||||
```json5
|
||||
{
|
||||
tools: {
|
||||
web: {
|
||||
search: {
|
||||
enabled: true,
|
||||
provider: "brave",
|
||||
apiKey: "BSA..." // optional if BRAVE_API_KEY is set
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## web_search
|
||||
|
||||
@ -147,7 +120,7 @@ Search the web using your configured provider.
|
||||
- `tools.web.search.enabled` must not be `false` (default: enabled)
|
||||
- API key for your chosen provider:
|
||||
- **Brave**: `BRAVE_API_KEY` or `tools.web.search.apiKey`
|
||||
- **Perplexity**: `OPENROUTER_API_KEY`, `PERPLEXITY_API_KEY`, or `tools.web.search.perplexity.apiKey`
|
||||
- **Perplexity**: `PERPLEXITY_API_KEY` or `tools.web.search.perplexity.apiKey`
|
||||
|
||||
### Config
|
||||
|
||||
@ -169,12 +142,21 @@ Search the web using your configured provider.
|
||||
|
||||
### Tool parameters
|
||||
|
||||
- `query` (required)
|
||||
- `count` (1–10; default from config)
|
||||
- `country` (optional): 2-letter country code for region-specific results (e.g., "DE", "US", "ALL"). If omitted, Brave chooses its default region.
|
||||
- `search_lang` (optional): ISO language code for search results (e.g., "de", "en", "fr")
|
||||
- `ui_lang` (optional): ISO language code for UI elements
|
||||
- `freshness` (optional, Brave only): filter by discovery time (`pd`, `pw`, `pm`, `py`, or `YYYY-MM-DDtoYYYY-MM-DD`)
|
||||
All parameters work for both Brave and Perplexity unless noted.
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| `query` | Search query (required) |
|
||||
| `count` | Results to return (1-10, default: 5) |
|
||||
| `country` | 2-letter ISO country code (e.g., "US", "DE") |
|
||||
| `language` | ISO 639-1 language code (e.g., "en", "de") |
|
||||
| `freshness` | Time filter: `day`, `week`, `month`, or `year` |
|
||||
| `date_after` | Results after this date (YYYY-MM-DD) |
|
||||
| `date_before` | Results before this date (YYYY-MM-DD) |
|
||||
| `ui_lang` | UI language code (Brave only) |
|
||||
| `domain_filter` | Domain allowlist/denylist array (Perplexity only) |
|
||||
| `max_tokens` | Total content budget, default 25000 (Perplexity only) |
|
||||
| `max_tokens_per_page` | Per-page token limit, default 2048 (Perplexity only) |
|
||||
|
||||
**Examples:**
|
||||
|
||||
@ -182,23 +164,40 @@ Search the web using your configured provider.
|
||||
// German-specific search
|
||||
await web_search({
|
||||
query: "TV online schauen",
|
||||
count: 10,
|
||||
country: "DE",
|
||||
search_lang: "de"
|
||||
});
|
||||
|
||||
// French search with French UI
|
||||
await web_search({
|
||||
query: "actualités",
|
||||
country: "FR",
|
||||
search_lang: "fr",
|
||||
ui_lang: "fr"
|
||||
language: "de"
|
||||
});
|
||||
|
||||
// Recent results (past week)
|
||||
await web_search({
|
||||
query: "TMBG interview",
|
||||
freshness: "pw"
|
||||
freshness: "week"
|
||||
});
|
||||
|
||||
// Date range search
|
||||
await web_search({
|
||||
query: "AI developments",
|
||||
date_after: "2024-01-01",
|
||||
date_before: "2024-06-30"
|
||||
});
|
||||
|
||||
// Domain filtering (Perplexity only)
|
||||
await web_search({
|
||||
query: "climate research",
|
||||
domain_filter: ["nature.com", "science.org", ".edu"]
|
||||
});
|
||||
|
||||
// Exclude domains (Perplexity only)
|
||||
await web_search({
|
||||
query: "product reviews",
|
||||
domain_filter: ["-reddit.com", "-pinterest.com"]
|
||||
});
|
||||
|
||||
// More content extraction (Perplexity only)
|
||||
await web_search({
|
||||
query: "detailed AI research",
|
||||
max_tokens: 50000,
|
||||
max_tokens_per_page: 4096
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
@ -2,70 +2,55 @@ import { describe, expect, it } from "vitest";
|
||||
|
||||
import { __testing } from "./web-search.js";
|
||||
|
||||
const { inferPerplexityBaseUrlFromApiKey, resolvePerplexityBaseUrl, normalizeFreshness } =
|
||||
__testing;
|
||||
|
||||
describe("web_search perplexity baseUrl defaults", () => {
|
||||
it("detects a Perplexity key prefix", () => {
|
||||
expect(inferPerplexityBaseUrlFromApiKey("pplx-123")).toBe("direct");
|
||||
});
|
||||
|
||||
it("detects an OpenRouter key prefix", () => {
|
||||
expect(inferPerplexityBaseUrlFromApiKey("sk-or-v1-123")).toBe("openrouter");
|
||||
});
|
||||
|
||||
it("returns undefined for unknown key formats", () => {
|
||||
expect(inferPerplexityBaseUrlFromApiKey("unknown-key")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("prefers explicit baseUrl over key-based defaults", () => {
|
||||
expect(resolvePerplexityBaseUrl({ baseUrl: "https://example.com" }, "config", "pplx-123")).toBe(
|
||||
"https://example.com",
|
||||
);
|
||||
});
|
||||
|
||||
it("defaults to direct when using PERPLEXITY_API_KEY", () => {
|
||||
expect(resolvePerplexityBaseUrl(undefined, "perplexity_env")).toBe("https://api.perplexity.ai");
|
||||
});
|
||||
|
||||
it("defaults to OpenRouter when using OPENROUTER_API_KEY", () => {
|
||||
expect(resolvePerplexityBaseUrl(undefined, "openrouter_env")).toBe(
|
||||
"https://openrouter.ai/api/v1",
|
||||
);
|
||||
});
|
||||
|
||||
it("defaults to direct when config key looks like Perplexity", () => {
|
||||
expect(resolvePerplexityBaseUrl(undefined, "config", "pplx-123")).toBe(
|
||||
"https://api.perplexity.ai",
|
||||
);
|
||||
});
|
||||
|
||||
it("defaults to OpenRouter when config key looks like OpenRouter", () => {
|
||||
expect(resolvePerplexityBaseUrl(undefined, "config", "sk-or-v1-123")).toBe(
|
||||
"https://openrouter.ai/api/v1",
|
||||
);
|
||||
});
|
||||
|
||||
it("defaults to OpenRouter for unknown config key formats", () => {
|
||||
expect(resolvePerplexityBaseUrl(undefined, "config", "weird-key")).toBe(
|
||||
"https://openrouter.ai/api/v1",
|
||||
);
|
||||
});
|
||||
});
|
||||
const { normalizeFreshness, normalizeToIsoDate, isoToPerplexityDate } = __testing;
|
||||
|
||||
describe("web_search freshness normalization", () => {
|
||||
it("accepts Brave shortcut values", () => {
|
||||
expect(normalizeFreshness("pd")).toBe("pd");
|
||||
expect(normalizeFreshness("PW")).toBe("pw");
|
||||
it("accepts Brave shortcut values and maps for Perplexity", () => {
|
||||
expect(normalizeFreshness("pd", "brave")).toBe("pd");
|
||||
expect(normalizeFreshness("PW", "brave")).toBe("pw");
|
||||
expect(normalizeFreshness("pd", "perplexity")).toBe("day");
|
||||
expect(normalizeFreshness("pw", "perplexity")).toBe("week");
|
||||
});
|
||||
|
||||
it("accepts valid date ranges", () => {
|
||||
expect(normalizeFreshness("2024-01-01to2024-01-31")).toBe("2024-01-01to2024-01-31");
|
||||
it("accepts Perplexity values and maps for Brave", () => {
|
||||
expect(normalizeFreshness("day", "perplexity")).toBe("day");
|
||||
expect(normalizeFreshness("week", "perplexity")).toBe("week");
|
||||
expect(normalizeFreshness("day", "brave")).toBe("pd");
|
||||
expect(normalizeFreshness("week", "brave")).toBe("pw");
|
||||
});
|
||||
|
||||
it("rejects invalid date ranges", () => {
|
||||
expect(normalizeFreshness("2024-13-01to2024-01-31")).toBeUndefined();
|
||||
expect(normalizeFreshness("2024-02-30to2024-03-01")).toBeUndefined();
|
||||
expect(normalizeFreshness("2024-03-10to2024-03-01")).toBeUndefined();
|
||||
it("rejects invalid values", () => {
|
||||
expect(normalizeFreshness("yesterday", "brave")).toBeUndefined();
|
||||
expect(normalizeFreshness("yesterday", "perplexity")).toBeUndefined();
|
||||
expect(normalizeFreshness("2024-01-01to2024-01-31", "perplexity")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("web_search date normalization", () => {
|
||||
it("accepts ISO format", () => {
|
||||
expect(normalizeToIsoDate("2024-01-15")).toBe("2024-01-15");
|
||||
expect(normalizeToIsoDate("2025-12-31")).toBe("2025-12-31");
|
||||
});
|
||||
|
||||
it("accepts Perplexity format and converts to ISO", () => {
|
||||
expect(normalizeToIsoDate("1/15/2024")).toBe("2024-01-15");
|
||||
expect(normalizeToIsoDate("12/31/2025")).toBe("2025-12-31");
|
||||
});
|
||||
|
||||
it("rejects invalid formats", () => {
|
||||
expect(normalizeToIsoDate("01-15-2024")).toBeUndefined();
|
||||
expect(normalizeToIsoDate("2024/01/15")).toBeUndefined();
|
||||
expect(normalizeToIsoDate("invalid")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("converts ISO to Perplexity format", () => {
|
||||
expect(isoToPerplexityDate("2024-01-15")).toBe("1/15/2024");
|
||||
expect(isoToPerplexityDate("2025-12-31")).toBe("12/31/2025");
|
||||
expect(isoToPerplexityDate("2024-03-05")).toBe("3/5/2024");
|
||||
});
|
||||
|
||||
it("rejects invalid ISO dates", () => {
|
||||
expect(isoToPerplexityDate("1/15/2024")).toBeUndefined();
|
||||
expect(isoToPerplexityDate("invalid")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
@ -3,7 +3,7 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { MoltbotConfig } from "../../config/config.js";
|
||||
import { formatCliCommand } from "../../cli/command-format.js";
|
||||
import type { AnyAgentTool } from "./common.js";
|
||||
import { jsonResult, readNumberParam, readStringParam } from "./common.js";
|
||||
import { jsonResult, readNumberParam, readStringArrayParam, readStringParam } from "./common.js";
|
||||
import {
|
||||
CacheEntry,
|
||||
DEFAULT_CACHE_TTL_MINUTES,
|
||||
@ -22,48 +22,118 @@ const DEFAULT_SEARCH_COUNT = 5;
|
||||
const MAX_SEARCH_COUNT = 10;
|
||||
|
||||
const BRAVE_SEARCH_ENDPOINT = "https://api.search.brave.com/res/v1/web/search";
|
||||
const DEFAULT_PERPLEXITY_BASE_URL = "https://openrouter.ai/api/v1";
|
||||
const PERPLEXITY_DIRECT_BASE_URL = "https://api.perplexity.ai";
|
||||
const DEFAULT_PERPLEXITY_MODEL = "perplexity/sonar-pro";
|
||||
const PERPLEXITY_KEY_PREFIXES = ["pplx-"];
|
||||
const OPENROUTER_KEY_PREFIXES = ["sk-or-"];
|
||||
const PERPLEXITY_SEARCH_ENDPOINT = "https://api.perplexity.ai/search";
|
||||
|
||||
const SEARCH_CACHE = new Map<string, CacheEntry<Record<string, unknown>>>();
|
||||
const BRAVE_FRESHNESS_SHORTCUTS = new Set(["pd", "pw", "pm", "py"]);
|
||||
const BRAVE_FRESHNESS_RANGE = /^(\d{4}-\d{2}-\d{2})to(\d{4}-\d{2}-\d{2})$/;
|
||||
const PERPLEXITY_RECENCY_VALUES = new Set(["day", "week", "month", "year"]);
|
||||
|
||||
const WebSearchSchema = Type.Object({
|
||||
query: Type.String({ description: "Search query string." }),
|
||||
count: Type.Optional(
|
||||
Type.Number({
|
||||
description: "Number of results to return (1-10).",
|
||||
minimum: 1,
|
||||
maximum: MAX_SEARCH_COUNT,
|
||||
}),
|
||||
),
|
||||
country: Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
"2-letter country code for region-specific results (e.g., 'DE', 'US', 'ALL'). Default: 'US'.",
|
||||
}),
|
||||
),
|
||||
search_lang: Type.Optional(
|
||||
Type.String({
|
||||
description: "ISO language code for search results (e.g., 'de', 'en', 'fr').",
|
||||
}),
|
||||
),
|
||||
ui_lang: Type.Optional(
|
||||
Type.String({
|
||||
description: "ISO language code for UI elements.",
|
||||
}),
|
||||
),
|
||||
freshness: Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
"Filter results by discovery time (Brave only). Values: 'pd' (past 24h), 'pw' (past week), 'pm' (past month), 'py' (past year), or date range 'YYYY-MM-DDtoYYYY-MM-DD'.",
|
||||
}),
|
||||
),
|
||||
});
|
||||
const FRESHNESS_TO_RECENCY: Record<string, string> = {
|
||||
pd: "day",
|
||||
pw: "week",
|
||||
pm: "month",
|
||||
py: "year",
|
||||
};
|
||||
const RECENCY_TO_FRESHNESS: Record<string, string> = {
|
||||
day: "pd",
|
||||
week: "pw",
|
||||
month: "pm",
|
||||
year: "py",
|
||||
};
|
||||
|
||||
const ISO_DATE_PATTERN = /^(\d{4})-(\d{2})-(\d{2})$/;
|
||||
const PERPLEXITY_DATE_PATTERN = /^(\d{1,2})\/(\d{1,2})\/(\d{4})$/;
|
||||
|
||||
function isoToPerplexityDate(iso: string): string | undefined {
|
||||
const match = iso.match(ISO_DATE_PATTERN);
|
||||
if (!match) return undefined;
|
||||
const [, year, month, day] = match;
|
||||
return `${parseInt(month, 10)}/${parseInt(day, 10)}/${year}`;
|
||||
}
|
||||
|
||||
function normalizeToIsoDate(value: string): string | undefined {
|
||||
const trimmed = value.trim();
|
||||
if (ISO_DATE_PATTERN.test(trimmed)) return trimmed;
|
||||
const match = trimmed.match(PERPLEXITY_DATE_PATTERN);
|
||||
if (match) {
|
||||
const [, month, day, year] = match;
|
||||
return `${year}-${month.padStart(2, "0")}-${day.padStart(2, "0")}`;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function createWebSearchSchema(provider: (typeof SEARCH_PROVIDERS)[number]) {
|
||||
const baseSchema = {
|
||||
query: Type.String({ description: "Search query string." }),
|
||||
count: Type.Optional(
|
||||
Type.Number({
|
||||
description: "Number of results to return (1-10).",
|
||||
minimum: 1,
|
||||
maximum: MAX_SEARCH_COUNT,
|
||||
}),
|
||||
),
|
||||
country: Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
"2-letter country code for region-specific results (e.g., 'DE', 'US', 'ALL'). Default: 'US'.",
|
||||
}),
|
||||
),
|
||||
language: Type.Optional(
|
||||
Type.String({
|
||||
description: "ISO 639-1 language code for results (e.g., 'en', 'de', 'fr').",
|
||||
}),
|
||||
),
|
||||
freshness: Type.Optional(
|
||||
Type.String({
|
||||
description: "Filter by time: 'day' (24h), 'week', 'month', or 'year'.",
|
||||
}),
|
||||
),
|
||||
date_after: Type.Optional(
|
||||
Type.String({
|
||||
description: "Only results published after this date (YYYY-MM-DD).",
|
||||
}),
|
||||
),
|
||||
date_before: Type.Optional(
|
||||
Type.String({
|
||||
description: "Only results published before this date (YYYY-MM-DD).",
|
||||
}),
|
||||
),
|
||||
} as const;
|
||||
|
||||
if (provider === "brave") {
|
||||
return Type.Object({
|
||||
...baseSchema,
|
||||
ui_lang: Type.Optional(
|
||||
Type.String({
|
||||
description: "ISO language code for UI elements.",
|
||||
}),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
return Type.Object({
|
||||
...baseSchema,
|
||||
domain_filter: Type.Optional(
|
||||
Type.Array(Type.String(), {
|
||||
description:
|
||||
"Domain filter (max 20). Allowlist: ['nature.com'] or denylist: ['-reddit.com']. Cannot mix.",
|
||||
}),
|
||||
),
|
||||
max_tokens: Type.Optional(
|
||||
Type.Number({
|
||||
description: "Total content budget across all results (default: 25000, max: 1000000).",
|
||||
minimum: 1,
|
||||
maximum: 1000000,
|
||||
}),
|
||||
),
|
||||
max_tokens_per_page: Type.Optional(
|
||||
Type.Number({
|
||||
description: "Max tokens extracted per page (default: 2048).",
|
||||
minimum: 1,
|
||||
}),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
type WebSearchConfig = NonNullable<MoltbotConfig["tools"]>["web"] extends infer Web
|
||||
? Web extends { search?: infer Search }
|
||||
@ -86,22 +156,22 @@ type BraveSearchResponse = {
|
||||
|
||||
type PerplexityConfig = {
|
||||
apiKey?: string;
|
||||
baseUrl?: string;
|
||||
model?: string;
|
||||
};
|
||||
|
||||
type PerplexityApiKeySource = "config" | "perplexity_env" | "openrouter_env" | "none";
|
||||
type PerplexityApiKeySource = "config" | "perplexity_env" | "none";
|
||||
|
||||
type PerplexitySearchResponse = {
|
||||
choices?: Array<{
|
||||
message?: {
|
||||
content?: string;
|
||||
};
|
||||
}>;
|
||||
citations?: string[];
|
||||
type PerplexitySearchApiResult = {
|
||||
title?: string;
|
||||
url?: string;
|
||||
snippet?: string;
|
||||
date?: string;
|
||||
last_updated?: string;
|
||||
};
|
||||
|
||||
type PerplexityBaseUrlHint = "direct" | "openrouter";
|
||||
type PerplexitySearchApiResponse = {
|
||||
results?: PerplexitySearchApiResult[];
|
||||
id?: string;
|
||||
};
|
||||
|
||||
function resolveSearchConfig(cfg?: MoltbotConfig): WebSearchConfig {
|
||||
const search = cfg?.tools?.web?.search;
|
||||
@ -127,13 +197,13 @@ function missingSearchKeyPayload(provider: (typeof SEARCH_PROVIDERS)[number]) {
|
||||
return {
|
||||
error: "missing_perplexity_api_key",
|
||||
message:
|
||||
"web_search (perplexity) needs an API key. Set PERPLEXITY_API_KEY or OPENROUTER_API_KEY in the Gateway environment, or configure tools.web.search.perplexity.apiKey.",
|
||||
"web_search (perplexity) needs an API key. Set PERPLEXITY_API_KEY in the Gateway environment, or configure tools.web.search.perplexity.apiKey.",
|
||||
docs: "https://docs.molt.bot/tools/web",
|
||||
};
|
||||
}
|
||||
return {
|
||||
error: "missing_brave_api_key",
|
||||
message: `web_search needs a Brave Search API key. Run \`${formatCliCommand("moltbot configure --section web")}\` to store it, or set BRAVE_API_KEY in the Gateway environment.`,
|
||||
message: `web_search (brave) needs an API key. Run \`${formatCliCommand("moltbot configure --section web")}\` to store it, or set BRAVE_API_KEY in the Gateway environment.`,
|
||||
docs: "https://docs.molt.bot/tools/web",
|
||||
};
|
||||
}
|
||||
@ -169,11 +239,6 @@ function resolvePerplexityApiKey(perplexity?: PerplexityConfig): {
|
||||
return { apiKey: fromEnvPerplexity, source: "perplexity_env" };
|
||||
}
|
||||
|
||||
const fromEnvOpenRouter = normalizeApiKey(process.env.OPENROUTER_API_KEY);
|
||||
if (fromEnvOpenRouter) {
|
||||
return { apiKey: fromEnvOpenRouter, source: "openrouter_env" };
|
||||
}
|
||||
|
||||
return { apiKey: undefined, source: "none" };
|
||||
}
|
||||
|
||||
@ -181,79 +246,36 @@ function normalizeApiKey(key: unknown): string {
|
||||
return typeof key === "string" ? key.trim() : "";
|
||||
}
|
||||
|
||||
function inferPerplexityBaseUrlFromApiKey(apiKey?: string): PerplexityBaseUrlHint | undefined {
|
||||
if (!apiKey) return undefined;
|
||||
const normalized = apiKey.toLowerCase();
|
||||
if (PERPLEXITY_KEY_PREFIXES.some((prefix) => normalized.startsWith(prefix))) {
|
||||
return "direct";
|
||||
}
|
||||
if (OPENROUTER_KEY_PREFIXES.some((prefix) => normalized.startsWith(prefix))) {
|
||||
return "openrouter";
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function resolvePerplexityBaseUrl(
|
||||
perplexity?: PerplexityConfig,
|
||||
apiKeySource: PerplexityApiKeySource = "none",
|
||||
apiKey?: string,
|
||||
): string {
|
||||
const fromConfig =
|
||||
perplexity && "baseUrl" in perplexity && typeof perplexity.baseUrl === "string"
|
||||
? perplexity.baseUrl.trim()
|
||||
: "";
|
||||
if (fromConfig) return fromConfig;
|
||||
if (apiKeySource === "perplexity_env") return PERPLEXITY_DIRECT_BASE_URL;
|
||||
if (apiKeySource === "openrouter_env") return DEFAULT_PERPLEXITY_BASE_URL;
|
||||
if (apiKeySource === "config") {
|
||||
const inferred = inferPerplexityBaseUrlFromApiKey(apiKey);
|
||||
if (inferred === "direct") return PERPLEXITY_DIRECT_BASE_URL;
|
||||
if (inferred === "openrouter") return DEFAULT_PERPLEXITY_BASE_URL;
|
||||
}
|
||||
return DEFAULT_PERPLEXITY_BASE_URL;
|
||||
}
|
||||
|
||||
function resolvePerplexityModel(perplexity?: PerplexityConfig): string {
|
||||
const fromConfig =
|
||||
perplexity && "model" in perplexity && typeof perplexity.model === "string"
|
||||
? perplexity.model.trim()
|
||||
: "";
|
||||
return fromConfig || DEFAULT_PERPLEXITY_MODEL;
|
||||
}
|
||||
|
||||
function resolveSearchCount(value: unknown, fallback: number): number {
|
||||
const parsed = typeof value === "number" && Number.isFinite(value) ? value : fallback;
|
||||
const clamped = Math.max(1, Math.min(MAX_SEARCH_COUNT, Math.floor(parsed)));
|
||||
return clamped;
|
||||
}
|
||||
|
||||
function normalizeFreshness(value: string | undefined): string | undefined {
|
||||
/**
|
||||
* Normalizes freshness shortcut to the provider's expected format.
|
||||
* Accepts both Brave format (pd/pw/pm/py) and Perplexity format (day/week/month/year).
|
||||
* Use date_after/date_before for specific date ranges.
|
||||
*/
|
||||
function normalizeFreshness(
|
||||
value: string | undefined,
|
||||
provider: (typeof SEARCH_PROVIDERS)[number],
|
||||
): string | undefined {
|
||||
if (!value) return undefined;
|
||||
const trimmed = value.trim();
|
||||
if (!trimmed) return undefined;
|
||||
|
||||
const lower = trimmed.toLowerCase();
|
||||
if (BRAVE_FRESHNESS_SHORTCUTS.has(lower)) return lower;
|
||||
|
||||
const match = trimmed.match(BRAVE_FRESHNESS_RANGE);
|
||||
if (!match) return undefined;
|
||||
if (BRAVE_FRESHNESS_SHORTCUTS.has(lower)) {
|
||||
return provider === "brave" ? lower : FRESHNESS_TO_RECENCY[lower];
|
||||
}
|
||||
|
||||
const [, start, end] = match;
|
||||
if (!isValidIsoDate(start) || !isValidIsoDate(end)) return undefined;
|
||||
if (start > end) return undefined;
|
||||
if (PERPLEXITY_RECENCY_VALUES.has(lower)) {
|
||||
return provider === "perplexity" ? lower : RECENCY_TO_FRESHNESS[lower];
|
||||
}
|
||||
|
||||
return `${start}to${end}`;
|
||||
}
|
||||
|
||||
function isValidIsoDate(value: string): boolean {
|
||||
if (!/^\d{4}-\d{2}-\d{2}$/.test(value)) return false;
|
||||
const [year, month, day] = value.split("-").map((part) => Number.parseInt(part, 10));
|
||||
if (!Number.isFinite(year) || !Number.isFinite(month) || !Number.isFinite(day)) return false;
|
||||
|
||||
const date = new Date(Date.UTC(year, month - 1, day));
|
||||
return (
|
||||
date.getUTCFullYear() === year && date.getUTCMonth() === month - 1 && date.getUTCDate() === day
|
||||
);
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function resolveSiteName(url: string | undefined): string | undefined {
|
||||
@ -265,45 +287,81 @@ function resolveSiteName(url: string | undefined): string | undefined {
|
||||
}
|
||||
}
|
||||
|
||||
async function runPerplexitySearch(params: {
|
||||
async function runPerplexitySearchApi(params: {
|
||||
query: string;
|
||||
apiKey: string;
|
||||
baseUrl: string;
|
||||
model: string;
|
||||
count: number;
|
||||
timeoutSeconds: number;
|
||||
}): Promise<{ content: string; citations: string[] }> {
|
||||
const endpoint = `${params.baseUrl.replace(/\/$/, "")}/chat/completions`;
|
||||
country?: string;
|
||||
searchDomainFilter?: string[];
|
||||
searchRecencyFilter?: string;
|
||||
searchLanguageFilter?: string[];
|
||||
searchAfterDate?: string;
|
||||
searchBeforeDate?: string;
|
||||
maxTokens?: number;
|
||||
maxTokensPerPage?: number;
|
||||
}): Promise<
|
||||
Array<{ title: string; url: string; description: string; published?: string; siteName?: string }>
|
||||
> {
|
||||
const body: Record<string, unknown> = {
|
||||
query: params.query,
|
||||
max_results: params.count,
|
||||
};
|
||||
|
||||
const res = await fetch(endpoint, {
|
||||
if (params.country) {
|
||||
body.country = params.country;
|
||||
}
|
||||
if (params.searchDomainFilter && params.searchDomainFilter.length > 0) {
|
||||
body.search_domain_filter = params.searchDomainFilter;
|
||||
}
|
||||
if (params.searchRecencyFilter) {
|
||||
body.search_recency_filter = params.searchRecencyFilter;
|
||||
}
|
||||
if (params.searchLanguageFilter && params.searchLanguageFilter.length > 0) {
|
||||
body.search_language_filter = params.searchLanguageFilter;
|
||||
}
|
||||
if (params.searchAfterDate) {
|
||||
body.search_after_date = params.searchAfterDate;
|
||||
}
|
||||
if (params.searchBeforeDate) {
|
||||
body.search_before_date = params.searchBeforeDate;
|
||||
}
|
||||
if (params.maxTokens !== undefined) {
|
||||
body.max_tokens = params.maxTokens;
|
||||
}
|
||||
if (params.maxTokensPerPage !== undefined) {
|
||||
body.max_tokens_per_page = params.maxTokensPerPage;
|
||||
}
|
||||
|
||||
const res = await fetch(PERPLEXITY_SEARCH_ENDPOINT, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json",
|
||||
Authorization: `Bearer ${params.apiKey}`,
|
||||
"HTTP-Referer": "https://molt.bot",
|
||||
"X-Title": "Moltbot Web Search",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: params.model,
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: params.query,
|
||||
},
|
||||
],
|
||||
}),
|
||||
body: JSON.stringify(body),
|
||||
signal: withTimeout(undefined, params.timeoutSeconds * 1000),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const detail = await readResponseText(res);
|
||||
throw new Error(`Perplexity API error (${res.status}): ${detail || res.statusText}`);
|
||||
throw new Error(`Perplexity Search API error (${res.status}): ${detail || res.statusText}`);
|
||||
}
|
||||
|
||||
const data = (await res.json()) as PerplexitySearchResponse;
|
||||
const content = data.choices?.[0]?.message?.content ?? "No response";
|
||||
const citations = data.citations ?? [];
|
||||
const data = (await res.json()) as PerplexitySearchApiResponse;
|
||||
const results = Array.isArray(data.results) ? data.results : [];
|
||||
|
||||
return { content, citations };
|
||||
// Map to align formats
|
||||
return results.map((entry) => ({
|
||||
title: entry.title ?? "",
|
||||
url: entry.url ?? "",
|
||||
description: entry.snippet ?? "",
|
||||
published: entry.date ?? undefined,
|
||||
siteName: resolveSiteName(entry.url ?? ""),
|
||||
}));
|
||||
}
|
||||
|
||||
async function runWebSearch(params: {
|
||||
@ -314,16 +372,17 @@ async function runWebSearch(params: {
|
||||
cacheTtlMs: number;
|
||||
provider: (typeof SEARCH_PROVIDERS)[number];
|
||||
country?: string;
|
||||
search_lang?: string;
|
||||
language?: string;
|
||||
ui_lang?: string;
|
||||
freshness?: string;
|
||||
perplexityBaseUrl?: string;
|
||||
perplexityModel?: string;
|
||||
dateAfter?: string;
|
||||
dateBefore?: string;
|
||||
searchDomainFilter?: string[];
|
||||
maxTokens?: number;
|
||||
maxTokensPerPage?: number;
|
||||
}): Promise<Record<string, unknown>> {
|
||||
const cacheKey = normalizeCacheKey(
|
||||
params.provider === "brave"
|
||||
? `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}:${params.freshness || "default"}`
|
||||
: `${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.search_lang || "default"}:${params.ui_lang || "default"}`,
|
||||
`${params.provider}:${params.query}:${params.count}:${params.country || "default"}:${params.language || "default"}:${params.ui_lang || "default"}:${params.freshness || "default"}:${params.dateAfter || "default"}:${params.dateBefore || "default"}:${params.searchDomainFilter?.join(",") || "default"}:${params.maxTokens || "default"}:${params.maxTokensPerPage || "default"}`,
|
||||
);
|
||||
const cached = readCache(SEARCH_CACHE, cacheKey);
|
||||
if (cached) return { ...cached.value, cached: true };
|
||||
@ -331,21 +390,27 @@ async function runWebSearch(params: {
|
||||
const start = Date.now();
|
||||
|
||||
if (params.provider === "perplexity") {
|
||||
const { content, citations } = await runPerplexitySearch({
|
||||
const results = await runPerplexitySearchApi({
|
||||
query: params.query,
|
||||
apiKey: params.apiKey,
|
||||
baseUrl: params.perplexityBaseUrl ?? DEFAULT_PERPLEXITY_BASE_URL,
|
||||
model: params.perplexityModel ?? DEFAULT_PERPLEXITY_MODEL,
|
||||
count: params.count,
|
||||
timeoutSeconds: params.timeoutSeconds,
|
||||
country: params.country,
|
||||
searchDomainFilter: params.searchDomainFilter,
|
||||
searchRecencyFilter: params.freshness,
|
||||
searchLanguageFilter: params.language ? [params.language] : undefined,
|
||||
searchAfterDate: params.dateAfter ? isoToPerplexityDate(params.dateAfter) : undefined,
|
||||
searchBeforeDate: params.dateBefore ? isoToPerplexityDate(params.dateBefore) : undefined,
|
||||
maxTokens: params.maxTokens,
|
||||
maxTokensPerPage: params.maxTokensPerPage,
|
||||
});
|
||||
|
||||
const payload = {
|
||||
query: params.query,
|
||||
provider: params.provider,
|
||||
model: params.perplexityModel ?? DEFAULT_PERPLEXITY_MODEL,
|
||||
count: results.length,
|
||||
tookMs: Date.now() - start,
|
||||
content,
|
||||
citations,
|
||||
results,
|
||||
};
|
||||
writeCache(SEARCH_CACHE, cacheKey, payload, params.cacheTtlMs);
|
||||
return payload;
|
||||
@ -361,14 +426,23 @@ async function runWebSearch(params: {
|
||||
if (params.country) {
|
||||
url.searchParams.set("country", params.country);
|
||||
}
|
||||
if (params.search_lang) {
|
||||
url.searchParams.set("search_lang", params.search_lang);
|
||||
if (params.language) {
|
||||
url.searchParams.set("search_lang", params.language);
|
||||
}
|
||||
if (params.ui_lang) {
|
||||
url.searchParams.set("ui_lang", params.ui_lang);
|
||||
}
|
||||
if (params.freshness) {
|
||||
url.searchParams.set("freshness", params.freshness);
|
||||
} else if (params.dateAfter && params.dateBefore) {
|
||||
url.searchParams.set("freshness", `${params.dateAfter}to${params.dateBefore}`);
|
||||
} else if (params.dateAfter) {
|
||||
url.searchParams.set(
|
||||
"freshness",
|
||||
`${params.dateAfter}to${new Date().toISOString().slice(0, 10)}`,
|
||||
);
|
||||
} else if (params.dateBefore) {
|
||||
url.searchParams.set("freshness", `1970-01-01to${params.dateBefore}`);
|
||||
}
|
||||
|
||||
const res = await fetch(url.toString(), {
|
||||
@ -418,14 +492,14 @@ export function createWebSearchTool(options?: {
|
||||
|
||||
const description =
|
||||
provider === "perplexity"
|
||||
? "Search the web using Perplexity Sonar (direct or via OpenRouter). Returns AI-synthesized answers with citations from real-time web search."
|
||||
? "Search the web using the Perplexity Search API. Returns structured results (title, URL, snippet) for fast research. Supports a wide range of web search configurations including domain and region-specific filtering."
|
||||
: "Search the web using Brave Search API. Supports region-specific and localized search via country and language parameters. Returns titles, URLs, and snippets for fast research.";
|
||||
|
||||
return {
|
||||
label: "Web Search",
|
||||
name: "web_search",
|
||||
description,
|
||||
parameters: WebSearchSchema,
|
||||
parameters: createWebSearchSchema(provider),
|
||||
execute: async (_toolCallId, args) => {
|
||||
const perplexityAuth =
|
||||
provider === "perplexity" ? resolvePerplexityApiKey(perplexityConfig) : undefined;
|
||||
@ -440,25 +514,39 @@ export function createWebSearchTool(options?: {
|
||||
const count =
|
||||
readNumberParam(params, "count", { integer: true }) ?? search?.maxResults ?? undefined;
|
||||
const country = readStringParam(params, "country");
|
||||
const search_lang = readStringParam(params, "search_lang");
|
||||
const language = readStringParam(params, "language");
|
||||
const ui_lang = readStringParam(params, "ui_lang");
|
||||
const rawFreshness = readStringParam(params, "freshness");
|
||||
if (rawFreshness && provider !== "brave") {
|
||||
return jsonResult({
|
||||
error: "unsupported_freshness",
|
||||
message: "freshness is only supported by the Brave web_search provider.",
|
||||
docs: "https://docs.molt.bot/tools/web",
|
||||
});
|
||||
}
|
||||
const freshness = rawFreshness ? normalizeFreshness(rawFreshness) : undefined;
|
||||
const freshness = rawFreshness ? normalizeFreshness(rawFreshness, provider) : undefined;
|
||||
if (rawFreshness && !freshness) {
|
||||
return jsonResult({
|
||||
error: "invalid_freshness",
|
||||
message:
|
||||
"freshness must be one of pd, pw, pm, py, or a range like YYYY-MM-DDtoYYYY-MM-DD.",
|
||||
message: "freshness must be day, week, month, or year.",
|
||||
docs: "https://docs.molt.bot/tools/web",
|
||||
});
|
||||
}
|
||||
const rawDateAfter = readStringParam(params, "date_after");
|
||||
const dateAfter = rawDateAfter ? normalizeToIsoDate(rawDateAfter) : undefined;
|
||||
if (rawDateAfter && !dateAfter) {
|
||||
return jsonResult({
|
||||
error: "invalid_date",
|
||||
message: "date_after must be YYYY-MM-DD format.",
|
||||
docs: "https://docs.molt.bot/tools/web",
|
||||
});
|
||||
}
|
||||
const rawDateBefore = readStringParam(params, "date_before");
|
||||
const dateBefore = rawDateBefore ? normalizeToIsoDate(rawDateBefore) : undefined;
|
||||
if (rawDateBefore && !dateBefore) {
|
||||
return jsonResult({
|
||||
error: "invalid_date",
|
||||
message: "date_before must be YYYY-MM-DD format.",
|
||||
docs: "https://docs.molt.bot/tools/web",
|
||||
});
|
||||
}
|
||||
const domainFilter = readStringArrayParam(params, "domain_filter");
|
||||
const maxTokens = readNumberParam(params, "max_tokens", { integer: true });
|
||||
const maxTokensPerPage = readNumberParam(params, "max_tokens_per_page", { integer: true });
|
||||
|
||||
const result = await runWebSearch({
|
||||
query,
|
||||
count: resolveSearchCount(count, DEFAULT_SEARCH_COUNT),
|
||||
@ -467,15 +555,14 @@ export function createWebSearchTool(options?: {
|
||||
cacheTtlMs: resolveCacheTtlMs(search?.cacheTtlMinutes, DEFAULT_CACHE_TTL_MINUTES),
|
||||
provider,
|
||||
country,
|
||||
search_lang,
|
||||
language,
|
||||
ui_lang,
|
||||
freshness,
|
||||
perplexityBaseUrl: resolvePerplexityBaseUrl(
|
||||
perplexityConfig,
|
||||
perplexityAuth?.source,
|
||||
perplexityAuth?.apiKey,
|
||||
),
|
||||
perplexityModel: resolvePerplexityModel(perplexityConfig),
|
||||
dateAfter,
|
||||
dateBefore,
|
||||
searchDomainFilter: domainFilter,
|
||||
maxTokens: maxTokens ?? undefined,
|
||||
maxTokensPerPage: maxTokensPerPage ?? undefined,
|
||||
});
|
||||
return jsonResult(result);
|
||||
},
|
||||
@ -483,7 +570,10 @@ export function createWebSearchTool(options?: {
|
||||
}
|
||||
|
||||
export const __testing = {
|
||||
inferPerplexityBaseUrlFromApiKey,
|
||||
resolvePerplexityBaseUrl,
|
||||
normalizeFreshness,
|
||||
normalizeToIsoDate,
|
||||
isoToPerplexityDate,
|
||||
SEARCH_CACHE,
|
||||
FRESHNESS_TO_RECENCY,
|
||||
RECENCY_TO_FRESHNESS,
|
||||
} as const;
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { createWebFetchTool, createWebSearchTool } from "./web-tools.js";
|
||||
import { __testing as webSearchTesting } from "./web-search.js";
|
||||
|
||||
describe("web tools defaults", () => {
|
||||
it("enables web_fetch by default (non-sandbox)", () => {
|
||||
@ -55,7 +56,7 @@ describe("web_search country and language parameters", () => {
|
||||
expect(url.searchParams.get("country")).toBe("DE");
|
||||
});
|
||||
|
||||
it("should pass search_lang parameter to Brave API", async () => {
|
||||
it("should pass language parameter to Brave API as search_lang", async () => {
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
@ -66,7 +67,7 @@ describe("web_search country and language parameters", () => {
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({ config: undefined, sandboxed: true });
|
||||
await tool?.execute?.(1, { query: "test", search_lang: "de" });
|
||||
await tool?.execute?.(1, { query: "test", language: "de" });
|
||||
|
||||
const url = new URL(mockFetch.mock.calls[0][0] as string);
|
||||
expect(url.searchParams.get("search_lang")).toBe("de");
|
||||
@ -124,21 +125,33 @@ describe("web_search country and language parameters", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("web_search perplexity baseUrl defaults", () => {
|
||||
describe("web_search perplexity Search API", () => {
|
||||
const priorFetch = global.fetch;
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
// @ts-expect-error global fetch cleanup
|
||||
global.fetch = priorFetch;
|
||||
// Clear search cache to prevent test pollution
|
||||
webSearchTesting.SEARCH_CACHE.clear();
|
||||
});
|
||||
|
||||
it("defaults to Perplexity direct when PERPLEXITY_API_KEY is set", async () => {
|
||||
it("uses Perplexity Search API when PERPLEXITY_API_KEY is set", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
json: () =>
|
||||
Promise.resolve({
|
||||
results: [
|
||||
{
|
||||
title: "Test",
|
||||
url: "https://example.com",
|
||||
snippet: "Test snippet",
|
||||
date: "2024-01-01",
|
||||
},
|
||||
],
|
||||
}),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
@ -148,18 +161,37 @@ describe("web_search perplexity baseUrl defaults", () => {
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, { query: "test-openrouter" });
|
||||
const result = await tool?.execute?.(1, { query: "test" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/chat/completions");
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/search");
|
||||
expect(mockFetch.mock.calls[0]?.[1]?.method).toBe("POST");
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.query).toBe("test");
|
||||
expect(result?.details).toMatchObject({
|
||||
provider: "perplexity",
|
||||
results: expect.arrayContaining([
|
||||
expect.objectContaining({ title: "Test", url: "https://example.com" }),
|
||||
]),
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects freshness for Perplexity provider", async () => {
|
||||
it("does not include freshness parameter for Perplexity provider", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
json: () =>
|
||||
Promise.resolve({
|
||||
results: [
|
||||
{
|
||||
title: "Test",
|
||||
url: "https://example.com",
|
||||
snippet: "Test snippet",
|
||||
date: "2024-01-01",
|
||||
},
|
||||
],
|
||||
}),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
@ -169,19 +201,18 @@ describe("web_search perplexity baseUrl defaults", () => {
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
const result = await tool?.execute?.(1, { query: "test", freshness: "pw" });
|
||||
const result = await tool?.execute?.(1, { query: "test" });
|
||||
|
||||
expect(mockFetch).not.toHaveBeenCalled();
|
||||
expect(result?.details).toMatchObject({ error: "unsupported_freshness" });
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(result?.details).toMatchObject({ provider: "perplexity", count: 1 });
|
||||
});
|
||||
|
||||
it("defaults to OpenRouter when OPENROUTER_API_KEY is set", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "");
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "sk-or-test");
|
||||
it("passes country parameter to Perplexity Search API", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
@ -191,69 +222,18 @@ describe("web_search perplexity baseUrl defaults", () => {
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, { query: "test-openrouter-env" });
|
||||
await tool?.execute?.(1, { query: "test", country: "DE" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://openrouter.ai/api/v1/chat/completions");
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.country).toBe("DE");
|
||||
});
|
||||
|
||||
it("prefers PERPLEXITY_API_KEY when both env keys are set", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
vi.stubEnv("OPENROUTER_API_KEY", "sk-or-test");
|
||||
it("uses config API key when provided", async () => {
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, { query: "test-both-env" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/chat/completions");
|
||||
});
|
||||
|
||||
it("uses configured baseUrl even when PERPLEXITY_API_KEY is set", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: {
|
||||
tools: {
|
||||
web: {
|
||||
search: {
|
||||
provider: "perplexity",
|
||||
perplexity: { baseUrl: "https://example.com/pplx" },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, { query: "test-config-baseurl" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://example.com/pplx/chat/completions");
|
||||
});
|
||||
|
||||
it("defaults to Perplexity direct when apiKey looks like Perplexity", async () => {
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
@ -272,38 +252,186 @@ describe("web_search perplexity baseUrl defaults", () => {
|
||||
},
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, { query: "test-config-apikey" });
|
||||
await tool?.execute?.(1, { query: "test" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://api.perplexity.ai/chat/completions");
|
||||
const headers = mockFetch.mock.calls[0]?.[1]?.headers;
|
||||
const authHeader =
|
||||
typeof headers?.get === "function" ? headers.get("Authorization") : headers?.Authorization;
|
||||
expect(authHeader).toBe("Bearer pplx-config");
|
||||
});
|
||||
|
||||
it("defaults to OpenRouter when apiKey looks like OpenRouter", async () => {
|
||||
it("passes freshness filter to Perplexity Search API", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ choices: [{ message: { content: "ok" } }], citations: [] }),
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: {
|
||||
tools: {
|
||||
web: {
|
||||
search: {
|
||||
provider: "perplexity",
|
||||
perplexity: { apiKey: "sk-or-v1-test" },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, { query: "test-openrouter-config" });
|
||||
await tool?.execute?.(1, { query: "test", freshness: "week" });
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
expect(mockFetch.mock.calls[0]?.[0]).toBe("https://openrouter.ai/api/v1/chat/completions");
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.search_recency_filter).toBe("week");
|
||||
});
|
||||
|
||||
it("accepts all valid freshness values for Perplexity", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
|
||||
for (const freshness of ["day", "week", "month", "year"]) {
|
||||
webSearchTesting.SEARCH_CACHE.clear();
|
||||
await tool?.execute?.(1, { query: `test-${freshness}`, freshness });
|
||||
const body = JSON.parse(mockFetch.mock.calls.at(-1)?.[1]?.body as string);
|
||||
expect(body.search_recency_filter).toBe(freshness);
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects invalid freshness values", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
const result = await tool?.execute?.(1, { query: "test", freshness: "yesterday" });
|
||||
|
||||
expect(mockFetch).not.toHaveBeenCalled();
|
||||
expect(result?.details).toMatchObject({ error: "invalid_freshness" });
|
||||
});
|
||||
|
||||
it("passes domain filter to Perplexity Search API", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, {
|
||||
query: "test",
|
||||
domain_filter: ["nature.com", "science.org"],
|
||||
});
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.search_domain_filter).toEqual(["nature.com", "science.org"]);
|
||||
});
|
||||
|
||||
it("passes denylist domain filter to Perplexity Search API", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, {
|
||||
query: "test",
|
||||
domain_filter: ["-reddit.com", "-pinterest.com"],
|
||||
});
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.search_domain_filter).toEqual(["-reddit.com", "-pinterest.com"]);
|
||||
});
|
||||
|
||||
it("passes language to Perplexity Search API as search_language_filter array", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, {
|
||||
query: "test",
|
||||
language: "en",
|
||||
});
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.search_language_filter).toEqual(["en"]);
|
||||
});
|
||||
|
||||
it("passes multiple filters together to Perplexity Search API", async () => {
|
||||
vi.stubEnv("PERPLEXITY_API_KEY", "pplx-test");
|
||||
const mockFetch = vi.fn(() =>
|
||||
Promise.resolve({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ results: [] }),
|
||||
} as Response),
|
||||
);
|
||||
// @ts-expect-error mock fetch
|
||||
global.fetch = mockFetch;
|
||||
|
||||
const tool = createWebSearchTool({
|
||||
config: { tools: { web: { search: { provider: "perplexity" } } } },
|
||||
sandboxed: true,
|
||||
});
|
||||
await tool?.execute?.(1, {
|
||||
query: "climate research",
|
||||
country: "US",
|
||||
freshness: "month",
|
||||
domain_filter: ["nature.com", ".gov"],
|
||||
language: "en",
|
||||
});
|
||||
|
||||
expect(mockFetch).toHaveBeenCalled();
|
||||
const body = JSON.parse(mockFetch.mock.calls[0]?.[1]?.body as string);
|
||||
expect(body.query).toBe("climate research");
|
||||
expect(body.country).toBe("US");
|
||||
expect(body.search_recency_filter).toBe("month");
|
||||
expect(body.search_domain_filter).toEqual(["nature.com", ".gov"]);
|
||||
expect(body.search_language_filter).toEqual(["en"]);
|
||||
});
|
||||
});
|
||||
|
||||
@ -35,7 +35,7 @@ export const CONFIGURE_SECTION_OPTIONS: Array<{
|
||||
}> = [
|
||||
{ value: "workspace", label: "Workspace", hint: "Set workspace + sessions" },
|
||||
{ value: "model", label: "Model", hint: "Pick provider + credentials" },
|
||||
{ value: "web", label: "Web tools", hint: "Configure Brave search + fetch" },
|
||||
{ value: "web", label: "Web tools", hint: "Configure web search (Perplexity/Brave) + fetch" },
|
||||
{ value: "gateway", label: "Gateway", hint: "Port, bind, auth, tailscale" },
|
||||
{
|
||||
value: "daemon",
|
||||
|
||||
@ -94,12 +94,18 @@ async function promptWebToolsConfig(
|
||||
): Promise<MoltbotConfig> {
|
||||
const existingSearch = nextConfig.tools?.web?.search;
|
||||
const existingFetch = nextConfig.tools?.web?.fetch;
|
||||
const hasSearchKey = Boolean(existingSearch?.apiKey);
|
||||
const existingProvider = existingSearch?.provider ?? "brave";
|
||||
const hasPerplexityKey = Boolean(
|
||||
existingSearch?.perplexity?.apiKey || process.env.PERPLEXITY_API_KEY,
|
||||
);
|
||||
const hasBraveKey = Boolean(existingSearch?.apiKey || process.env.BRAVE_API_KEY);
|
||||
const hasSearchKey = existingProvider === "perplexity" ? hasPerplexityKey : hasBraveKey;
|
||||
|
||||
note(
|
||||
[
|
||||
"Web search lets your agent look things up online using the `web_search` tool.",
|
||||
"It requires a Brave Search API key (you can store it in the config or set BRAVE_API_KEY in the Gateway environment).",
|
||||
"Choose a provider: Perplexity Search (recommended) or Brave Search.",
|
||||
"Both return structured results (title, URL, snippet) for fast research.",
|
||||
"Docs: https://docs.molt.bot/tools/web",
|
||||
].join("\n"),
|
||||
"Web search",
|
||||
@ -107,7 +113,7 @@ async function promptWebToolsConfig(
|
||||
|
||||
const enableSearch = guardCancel(
|
||||
await confirm({
|
||||
message: "Enable web_search (Brave Search)?",
|
||||
message: "Enable web_search?",
|
||||
initialValue: existingSearch?.enabled ?? hasSearchKey,
|
||||
}),
|
||||
runtime,
|
||||
@ -119,27 +125,79 @@ async function promptWebToolsConfig(
|
||||
};
|
||||
|
||||
if (enableSearch) {
|
||||
const keyInput = guardCancel(
|
||||
await text({
|
||||
message: hasSearchKey
|
||||
? "Brave Search API key (leave blank to keep current or use BRAVE_API_KEY)"
|
||||
: "Brave Search API key (paste it here; leave blank to use BRAVE_API_KEY)",
|
||||
placeholder: hasSearchKey ? "Leave blank to keep current" : "BSA...",
|
||||
const providerChoice = guardCancel(
|
||||
await select({
|
||||
message: "Choose web search provider",
|
||||
options: [
|
||||
{
|
||||
value: "perplexity",
|
||||
label: "Perplexity Search",
|
||||
},
|
||||
{
|
||||
value: "brave",
|
||||
label: "Brave Search",
|
||||
},
|
||||
],
|
||||
initialValue: existingProvider,
|
||||
}),
|
||||
runtime,
|
||||
);
|
||||
const key = String(keyInput ?? "").trim();
|
||||
if (key) {
|
||||
nextSearch = { ...nextSearch, apiKey: key };
|
||||
} else if (!hasSearchKey) {
|
||||
note(
|
||||
[
|
||||
"No key stored yet, so web_search will stay unavailable.",
|
||||
"Store a key here or set BRAVE_API_KEY in the Gateway environment.",
|
||||
"Docs: https://docs.molt.bot/tools/web",
|
||||
].join("\n"),
|
||||
"Web search",
|
||||
|
||||
nextSearch = { ...nextSearch, provider: providerChoice };
|
||||
|
||||
if (providerChoice === "perplexity") {
|
||||
const hasKey = Boolean(existingSearch?.perplexity?.apiKey);
|
||||
const keyInput = guardCancel(
|
||||
await text({
|
||||
message: hasKey
|
||||
? "Perplexity API key (leave blank to keep current or use PERPLEXITY_API_KEY)"
|
||||
: "Perplexity API key (paste it here; leave blank to use PERPLEXITY_API_KEY)",
|
||||
placeholder: hasKey ? "Leave blank to keep current" : "pplx-...",
|
||||
}),
|
||||
runtime,
|
||||
);
|
||||
const key = String(keyInput ?? "").trim();
|
||||
if (key) {
|
||||
nextSearch = {
|
||||
...nextSearch,
|
||||
perplexity: { ...existingSearch?.perplexity, apiKey: key },
|
||||
};
|
||||
} else if (!hasKey && !process.env.PERPLEXITY_API_KEY) {
|
||||
note(
|
||||
[
|
||||
"No key stored yet, so web_search will stay unavailable.",
|
||||
"Store a key here or set PERPLEXITY_API_KEY in the Gateway environment.",
|
||||
"Get your API key at: https://www.perplexity.ai/settings/api",
|
||||
"Docs: https://docs.molt.bot/tools/web",
|
||||
].join("\n"),
|
||||
"Web search",
|
||||
);
|
||||
}
|
||||
} else {
|
||||
const hasKey = Boolean(existingSearch?.apiKey);
|
||||
const keyInput = guardCancel(
|
||||
await text({
|
||||
message: hasKey
|
||||
? "Brave Search API key (leave blank to keep current or use BRAVE_API_KEY)"
|
||||
: "Brave Search API key (paste it here; leave blank to use BRAVE_API_KEY)",
|
||||
placeholder: hasKey ? "Leave blank to keep current" : "BSA...",
|
||||
}),
|
||||
runtime,
|
||||
);
|
||||
const key = String(keyInput ?? "").trim();
|
||||
if (key) {
|
||||
nextSearch = { ...nextSearch, apiKey: key };
|
||||
} else if (!hasKey && !process.env.BRAVE_API_KEY) {
|
||||
note(
|
||||
[
|
||||
"No key stored yet, so web_search will stay unavailable.",
|
||||
"Store a key here or set BRAVE_API_KEY in the Gateway environment.",
|
||||
"Get your API key at: https://brave.com/search/api/",
|
||||
"Docs: https://docs.molt.bot/tools/web",
|
||||
].join("\n"),
|
||||
"Web search",
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -12,8 +12,6 @@ describe("web search provider config", () => {
|
||||
provider: "perplexity",
|
||||
perplexity: {
|
||||
apiKey: "test-key",
|
||||
baseUrl: "https://api.perplexity.ai",
|
||||
model: "perplexity/sonar-pro",
|
||||
},
|
||||
},
|
||||
},
|
||||
|
||||
@ -441,11 +441,7 @@ const FIELD_HELP: Record<string, string> = {
|
||||
"tools.web.search.timeoutSeconds": "Timeout in seconds for web_search requests.",
|
||||
"tools.web.search.cacheTtlMinutes": "Cache TTL in minutes for web_search results.",
|
||||
"tools.web.search.perplexity.apiKey":
|
||||
"Perplexity or OpenRouter API key (fallback: PERPLEXITY_API_KEY or OPENROUTER_API_KEY env var).",
|
||||
"tools.web.search.perplexity.baseUrl":
|
||||
"Perplexity base URL override (default: https://openrouter.ai/api/v1 or https://api.perplexity.ai).",
|
||||
"tools.web.search.perplexity.model":
|
||||
'Perplexity model override (default: "perplexity/sonar-pro").',
|
||||
"Perplexity API key (fallback: PERPLEXITY_API_KEY env var).",
|
||||
"tools.web.fetch.enabled": "Enable the web_fetch tool (lightweight HTTP fetch).",
|
||||
"tools.web.fetch.maxChars": "Max characters returned by web_fetch (truncated).",
|
||||
"tools.web.fetch.timeoutSeconds": "Timeout in seconds for web_fetch requests.",
|
||||
|
||||
@ -348,12 +348,8 @@ export type ToolsConfig = {
|
||||
cacheTtlMinutes?: number;
|
||||
/** Perplexity-specific configuration (used when provider="perplexity"). */
|
||||
perplexity?: {
|
||||
/** API key for Perplexity or OpenRouter (defaults to PERPLEXITY_API_KEY or OPENROUTER_API_KEY env var). */
|
||||
/** API key for Perplexity (defaults to PERPLEXITY_API_KEY env var). */
|
||||
apiKey?: string;
|
||||
/** Base URL for API requests (defaults to OpenRouter: https://openrouter.ai/api/v1). */
|
||||
baseUrl?: string;
|
||||
/** Model to use (defaults to "perplexity/sonar-pro"). */
|
||||
model?: string;
|
||||
};
|
||||
};
|
||||
fetch?: {
|
||||
|
||||
@ -173,8 +173,6 @@ export const ToolsWebSearchSchema = z
|
||||
perplexity: z
|
||||
.object({
|
||||
apiKey: z.string().optional(),
|
||||
baseUrl: z.string().optional(),
|
||||
model: z.string().optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
|
||||
@ -408,11 +408,7 @@ function resolveToolPolicies(params: {
|
||||
function hasWebSearchKey(cfg: MoltbotConfig, env: NodeJS.ProcessEnv): boolean {
|
||||
const search = cfg.tools?.web?.search;
|
||||
return Boolean(
|
||||
search?.apiKey ||
|
||||
search?.perplexity?.apiKey ||
|
||||
env.BRAVE_API_KEY ||
|
||||
env.PERPLEXITY_API_KEY ||
|
||||
env.OPENROUTER_API_KEY,
|
||||
search?.apiKey || search?.perplexity?.apiKey || env.BRAVE_API_KEY || env.PERPLEXITY_API_KEY,
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@ -432,29 +432,35 @@ export async function finalizeOnboardingWizard(options: FinalizeOnboardingOption
|
||||
);
|
||||
}
|
||||
|
||||
const webSearchKey = (nextConfig.tools?.web?.search?.apiKey ?? "").trim();
|
||||
const webSearchEnv = (process.env.BRAVE_API_KEY ?? "").trim();
|
||||
const webSearchProvider = nextConfig.tools?.web?.search?.provider ?? "brave";
|
||||
const webSearchKey =
|
||||
webSearchProvider === "perplexity"
|
||||
? (nextConfig.tools?.web?.search?.perplexity?.apiKey ?? "").trim()
|
||||
: (nextConfig.tools?.web?.search?.apiKey ?? "").trim();
|
||||
const webSearchEnv =
|
||||
webSearchProvider === "perplexity"
|
||||
? (process.env.PERPLEXITY_API_KEY ?? "").trim()
|
||||
: (process.env.BRAVE_API_KEY ?? "").trim();
|
||||
const hasWebSearchKey = Boolean(webSearchKey || webSearchEnv);
|
||||
await prompter.note(
|
||||
hasWebSearchKey
|
||||
? [
|
||||
"Web search is enabled, so your agent can look things up online when needed.",
|
||||
"",
|
||||
`Provider: ${webSearchProvider === "perplexity" ? "Perplexity Search" : "Brave Search"}`,
|
||||
webSearchKey
|
||||
? "API key: stored in config (tools.web.search.apiKey)."
|
||||
: "API key: provided via BRAVE_API_KEY env var (Gateway environment).",
|
||||
? `API key: stored in config (tools.web.search.${webSearchProvider === "perplexity" ? "perplexity.apiKey" : "apiKey"}).`
|
||||
: `API key: provided via ${webSearchProvider === "perplexity" ? "PERPLEXITY_API_KEY" : "BRAVE_API_KEY"} env var (Gateway environment).`,
|
||||
"Docs: https://docs.molt.bot/tools/web",
|
||||
].join("\n")
|
||||
: [
|
||||
"If you want your agent to be able to search the web, you’ll need an API key.",
|
||||
"",
|
||||
"Moltbot uses Brave Search for the `web_search` tool. Without a Brave Search API key, web search won’t work.",
|
||||
"To enable web search, your agent will need an API key for either Perplexity Search or Brave Search.",
|
||||
"",
|
||||
"Set it up interactively:",
|
||||
`- Run: ${formatCliCommand("moltbot configure --section web")}`,
|
||||
"- Enable web_search and paste your Brave Search API key",
|
||||
"- Choose a provider and paste your API key",
|
||||
"",
|
||||
"Alternative: set BRAVE_API_KEY in the Gateway environment (no config changes).",
|
||||
"Alternative: set PERPLEXITY_API_KEY or BRAVE_API_KEY in the Gateway environment (no config changes).",
|
||||
"Docs: https://docs.molt.bot/tools/web",
|
||||
].join("\n"),
|
||||
"Web search (optional)",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user