Jon Shapiro 92697edb7c fix(venice): add compat settings to prevent HTTP 400 errors

Venice's API doesn't support certain OpenAI-compatible parameters that
Clawdbot sends by default:

- `store`: Venice returns HTTP 400 with no body when this is present
- `developer` role: Not supported by Venice's API

This adds VENICE_COMPAT settings (supportsStore: false,
supportsDeveloperRole: false) to all Venice model definitions, both
from the static catalog and dynamically discovered models.

Fixes issues reported in PR #1666 where users experienced silent
failures (HTTP 400, no body) when using Venice models.

Co-authored-by: jonisjongithub <jonisjongithub@users.noreply.github.com>
Co-authored-by: Clawdbot <bot@clawd.bot>

2026-01-26 17:56:01 -08:00

1.6 KiB

Raw Blame History

summary

read_when

Firecrawl fallback for web_fetch (anti-bot + cached extraction)

You want Firecrawl-backed web extraction

You need a Firecrawl API key

You want anti-bot extraction for web_fetch

Firecrawl

Clawdbot can use Firecrawl as a fallback extractor for web_fetch. It is a hosted content extraction service that supports bot circumvention and caching, which helps with JS-heavy sites or pages that block plain HTTP fetches.

Get an API key

Create a Firecrawl account and generate an API key.
Store it in config or set FIRECRAWL_API_KEY in the gateway environment.

Configure Firecrawl

{
  tools: {
    web: {
      fetch: {
        firecrawl: {
          apiKey: "FIRECRAWL_API_KEY_HERE",
          baseUrl: "https://api.firecrawl.dev",
          onlyMainContent: true,
          maxAgeMs: 172800000,
          timeoutSeconds: 60
        }
      }
    }
  }
}

Notes:

firecrawl.enabled defaults to true when an API key is present.
maxAgeMs controls how old cached results can be (ms). Default is 2 days.

Stealth / bot circumvention

Firecrawl exposes a proxy mode parameter for bot circumvention (basic, stealth, or auto). Clawdbot always uses proxy: "auto" plus storeInCache: true for Firecrawl requests. If proxy is omitted, Firecrawl defaults to auto. auto retries with stealth proxies if a basic attempt fails, which may use more credits than basic-only scraping.

How `web_fetch` uses Firecrawl

web_fetch extraction order:

Readability (local)
Firecrawl (if configured)
Basic HTML cleanup (last fallback)

See Web tools for the full web tool setup.

1.6 KiB Raw Blame History