Charles-Henri ROBICHE efd827b526

feat(providers): add LiteLLM provider support

Add LiteLLM as a new OpenAI-compatible proxy provider:

- Add onboarding flow with API key, base URL, and model selection
- Fetch available models from LiteLLM /v1/models endpoint
- Auto-detect context window from /model/info endpoint
- Set supportsStore: false to avoid "Extra inputs are not permitted" errors
  with providers that don't support the OpenAI Responses API store parameter
- Preserve compat settings through model resolution pipeline
- Add provider documentation

Closes #2639
Closes #2305

Co-Authored-By: Claude <noreply@anthropic.com>

2026-01-28 08:55:48 +01:00

2.5 KiB

Raw Blame History

summary

read_when

Use LiteLLM as an OpenAI-compatible proxy in Clawdbot

You want to use LiteLLM as a model provider

You need to connect to a self-hosted LiteLLM proxy

You want to use any model through an OpenAI-compatible API

LiteLLM

LiteLLM is an OpenAI-compatible proxy that supports 100+ LLM APIs. Clawdbot registers it as the litellm provider and uses the OpenAI Completions API.

Quick setup

Set up your LiteLLM proxy (see LiteLLM docs)
Set environment variables (optional):
- LITELLM_API_KEY - your LiteLLM API key
- LITELLM_BASE_URL - your LiteLLM endpoint (default: http://localhost:4000)
- LITELLM_MODEL - default model name (default: gpt-4)
Run onboarding:

clawdbot onboard --auth-choice litellm-api-key

The wizard will prompt for:

Base URL (your LiteLLM proxy endpoint)
API key
Model name (as configured in your LiteLLM proxy)

Config example

{
  env: { LITELLM_API_KEY: "sk-..." },
  agents: {
    defaults: {
      model: { primary: "litellm/gpt-4" },
      models: { "litellm/gpt-4": { alias: "GPT-4" } }
    }
  },
  models: {
    mode: "merge",
    providers: {
      litellm: {
        baseUrl: "http://localhost:4000",
        apiKey: "${LITELLM_API_KEY}",
        api: "openai-completions",
        models: [
          {
            id: "gpt-4",
            name: "GPT-4",
            reasoning: false,
            input: ["text"],
            contextWindow: 128000,
            maxTokens: 8192
          }
        ]
      }
    }
  }
}

Multiple models

Add additional models to your config as needed:

{
  models: {
    providers: {
      litellm: {
        baseUrl: "http://localhost:4000",
        apiKey: "${LITELLM_API_KEY}",
        api: "openai-completions",
        models: [
          { id: "gpt-4", name: "GPT-4", contextWindow: 128000, maxTokens: 8192 },
          { id: "claude-3-opus", name: "Claude Opus", contextWindow: 200000, maxTokens: 4096 },
          { id: "gemini-pro", name: "Gemini Pro", contextWindow: 32000, maxTokens: 8192 }
        ]
      }
    }
  }
}

Then switch models using:

clawdbot config set agents.defaults.model.primary litellm/claude-3-opus

Notes

Model refs use litellm/<modelId> where modelId matches your LiteLLM config.
The base URL should not include /v1 - Clawdbot's OpenAI client appends it.
Supported LiteLLM models depend on your proxy configuration.
See Model providers for provider rules.

2.5 KiB Raw Blame History