Merge ff8b6d86c8 into 4583f88626

2026-01-29 21:53:30 -05:00 · 2026-01-29 21:53:30 -05:00 · 38859e2da9
commit 38859e2da9
parent 4583f88626 ff8b6d86c8
7 changed files with 305 additions and 8 deletions
--- a/docs/concepts/model-providers.md
+++ b/docs/concepts/model-providers.md
@ -263,7 +263,26 @@ ollama pull llama3.3

 Ollama is automatically detected when running locally at `http://127.0.0.1:11434/v1`. See [/providers/ollama](/providers/ollama) for model recommendations and custom configuration.

-### Local proxies (LM Studio, vLLM, LiteLLM, etc.)
+### jan.ai
+
+jan.ai is a local LLM runtime built on llama.cpp with OpenAI-compatible API:
+
+- Provider: `jan`
+- Auth: None required (local server), but needs `JAN_API_KEY` set to any value for auto-discovery
+- Example model: `jan/<model-name>`
+- Installation: https://jan.ai
+
+```json5
+{
+  agents: {
+    defaults: { model: { primary: "jan/llama-3.3-70b" } }
+  }
+}
+```
+
+jan.ai is automatically detected when running locally at `http://127.0.0.1:1337/v1`. See [/providers/jan](/providers/jan) for detailed setup and configuration.
+
+### Local proxies (LM Studio, vLLM, jan.ai, etc.)

 Example (OpenAI‑compatible):

--- a/docs/help/faq.md
+++ b/docs/help/faq.md
@ -1878,16 +1878,16 @@ injection and unsafe behavior. See [Security](/gateway/security).

 More context: [Models](/concepts/models).

-### Can I use selfhosted models llamacpp vLLM Ollama
+### Can I use selfhosted models llamacpp vLLM Ollama jan.ai

 Yes. If your local server exposes an OpenAI-compatible API, you can point a
-custom provider at it. Ollama is supported directly and is the easiest path.
+custom provider at it. Ollama and jan.ai are supported directly with auto-discovery and are the easiest paths.

 Security note: smaller or heavily quantized models are more vulnerable to prompt
 injection. We strongly recommend **large models** for any bot that can use tools.
 If you still want small models, enable sandboxing and strict tool allowlists.

-Docs: [Ollama](/providers/ollama), [Local models](/gateway/local-models),
+Docs: [Ollama](/providers/ollama), [jan.ai](/providers/jan), [Local models](/gateway/local-models),
 [Model providers](/concepts/model-providers), [Security](/gateway/security),
 [Sandboxing](/gateway/sandboxing).

--- a/docs/providers/jan.md
+++ b/docs/providers/jan.md
@ -0,0 +1,193 @@
+---
+summary: "Run Clawdbot with jan.ai (local LLM runtime using llama.cpp)"
+read_when:
+  - You want to run Clawdbot with local models via jan.ai
+  - You need jan.ai setup and configuration guidance
+---
+# jan.ai
+
+jan.ai is a local LLM runtime built on llama.cpp with OpenAI-compatible API. Clawdbot integrates with jan.ai and can **auto-discover available models** when you opt in with `JAN_API_KEY` (or an auth profile) and do not define an explicit `models.providers.jan` entry.
+
+## Quick start
+
+1) Install jan.ai: https://jan.ai
+
+2) Download models using jan.ai's UI or CLI
+
+3) Enable jan.ai for Clawdbot (any value works; jan.ai doesn't require a real key):
+
+```bash
+# Set environment variable
+export JAN_API_KEY="jan-local"
+
+# Or configure in your config file
+clawdbot config set models.providers.jan.apiKey "jan-local"
+```
+
+4) Use jan.ai models:
+
+```json5
+{
+  agents: {
+    defaults: {
+      model: { primary: "jan/llama-3.3-70b" }
+    }
+  }
+}
+```
+
+## Model discovery (implicit provider)
+
+When you set `JAN_API_KEY` (or an auth profile) and **do not** define `models.providers.jan`, Clawdbot discovers models from the local jan.ai instance at `http://127.0.0.1:1337/v1`:
+
+- Queries `/v1/models` endpoint
+- Includes all models from jan.ai
+- Marks `reasoning` when model ID contains "r1" or "reasoning" (case-insensitive)
+- Sets `input: ["text"]` for all models (jan.ai primarily supports text models)
+- Sets `contextWindow` to 128000
+- Sets `maxTokens` to 8192
+- Sets all costs to `0` (local provider)
+
+This avoids manual model entries while keeping the catalog aligned with your jan.ai installation.
+
+To see what models are available:
+
+```bash
+clawdbot models list
+```
+
+If you set `models.providers.jan` explicitly, auto-discovery is skipped and you must define models manually (see below).
+
+## Configuration
+
+### Basic setup (implicit discovery)
+
+The simplest way to enable jan.ai is via environment variable:
+
+```bash
+export JAN_API_KEY="jan-local"
+```
+
+### Explicit setup (manual models)
+
+Use explicit config when:
+- jan.ai runs on another host/port.
+- You want to force specific context windows or model lists.
+- You want to override default model settings.
+
+```json5
+{
+  models: {
+    providers: {
+      jan: {
+        baseUrl: "http://127.0.0.1:1337/v1",
+        apiKey: "jan-local",
+        api: "openai-completions",
+        models: [
+          {
+            id: "llama-3.3-70b",
+            name: "Llama 3.3 70B",
+            reasoning: false,
+            input: ["text"],
+            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+            contextWindow: 128000,
+            maxTokens: 8192
+          }
+        ]
+      }
+    }
+  }
+}
+```
+
+If `JAN_API_KEY` is set, you can omit `apiKey` in the provider entry and Clawdbot will fill it for availability checks.
+
+### Custom base URL (explicit config)
+
+If jan.ai is running on a different host or port (explicit config disables auto-discovery, so define models manually):
+
+```json5
+{
+  models: {
+    providers: {
+      jan: {
+        apiKey: "jan-local",
+        baseUrl: "http://jan-host:1337/v1",
+        api: "openai-completions"
+      }
+    }
+  }
+}
+```
+
+### Model selection
+
+Once configured, all your jan.ai models are available:
+
+```json5
+{
+  agents: {
+    defaults: {
+      model: {
+        primary: "jan/llama-3.3-70b",
+        fallback: ["jan/qwen2.5-coder-32b"]
+      }
+    }
+  }
+}
+```
+
+## Advanced
+
+### Reasoning models
+
+Clawdbot marks models as reasoning-capable when the model ID contains "r1" or "reasoning" (case-insensitive). This includes models like DeepSeek-R1 and other reasoning models.
+
+### Model Costs
+
+jan.ai runs locally, so all model costs are set to $0.
+
+### Context windows
+
+For auto-discovered models, Clawdbot defaults to a context window of 128000 and maxTokens of 8192. You can override these values in explicit provider config.
+
+## Troubleshooting
+
+### jan.ai not detected
+
+Make sure jan.ai is running and that you set `JAN_API_KEY` (or an auth profile), and that you did **not** define an explicit `models.providers.jan` entry.
+
+And that the API is accessible:
+
+```bash
+curl http://localhost:1337/v1/models
+```
+
+### No models available
+
+Make sure jan.ai has models downloaded and available. Check the jan.ai UI to ensure models are installed, or download models through jan.ai's interface.
+
+To verify API endpoint accessibility:
+
+```bash
+curl http://localhost:1337/v1/models
+```
+
+### Connection refused
+
+Check that jan.ai is running on the correct port (default 1337):
+
+```bash
+# Check if jan.ai is running on port 1337
+netstat -an | grep 1337
+
+# Or restart jan.ai
+# Restart through the jan.ai application or service
+```
+
+## See Also
+
+- [Model Providers](/concepts/model-providers) - Overview of all providers
+- [Model Selection](/concepts/models) - How to choose models
+- [Gateway Configuration](/gateway/configuration) - Full config reference
+- [Ollama Provider](/providers/ollama) - Similar local provider for comparison
--- a/scripts/test-parallel.mjs
+++ b/scripts/test-parallel.mjs
@ -42,6 +42,7 @@ const WARNING_SUPPRESSION_FLAGS = [
  "--disable-warning=ExperimentalWarning",
  "--disable-warning=DEP0040",
  "--disable-warning=DEP0060",
+  "--max-old-space-size=4096",
 ];

 const runOnce = (entry, extraArgs = []) =>
@ -61,6 +62,11 @@ const runOnce = (entry, extraArgs = []) =>
    });
    children.add(child);
    child.on("exit", (code, signal) => {
+      if (signal === 'SIGKILL' || signal === 'SIGABRT' || signal === 'SIGSEGV') {
+        console.error(`Worker ${entry.name} crashed with signal ${signal} (possible OOM or resource exhaustion)`);
+      } else if (signal) {
+        console.warn(`Worker ${entry.name} terminated with signal ${signal}`);
+      }
      children.delete(child);
      resolve(code ?? (signal ? 1 : 0));
    });
--- a/src/agents/models-config.providers.ts
+++ b/src/agents/models-config.providers.ts
@ -86,6 +86,17 @@ const OLLAMA_DEFAULT_COST = {
  cacheWrite: 0,
 };

+const JAN_BASE_URL = "http://127.0.0.1:1337/v1";
+const JAN_API_BASE_URL = "http://127.0.0.1:1337";
+const JAN_DEFAULT_CONTEXT_WINDOW = 128000;
+const JAN_DEFAULT_MAX_TOKENS = 8192;
+const JAN_DEFAULT_COST = {
+  input: 0,
+  output: 0,
+  cacheRead: 0,
+  cacheWrite: 0,
+};
+
 interface OllamaModel {
  name: string;
  modified_at: string;
@ -101,6 +112,18 @@ interface OllamaTagsResponse {
  models: OllamaModel[];
 }

+interface JanModel {
+  id: string;
+  object: string;
+  created: number;
+  owned_by: string;
+}
+
+interface JanModelsResponse {
+  object: string;
+  data: JanModel[];
+}
+
 async function discoverOllamaModels(): Promise<ModelDefinitionConfig[]> {
  // Skip Ollama discovery in test environments
  if (process.env.VITEST || process.env.NODE_ENV === "test") {
@ -139,6 +162,44 @@ async function discoverOllamaModels(): Promise<ModelDefinitionConfig[]> {
  }
 }

+async function discoverJanModels(): Promise<ModelDefinitionConfig[]> {
+  // Skip jan.ai discovery in test environments
+  if (process.env.VITEST || process.env.NODE_ENV === "test") {
+    return [];
+  }
+  try {
+    const response = await fetch(`${JAN_API_BASE_URL}/v1/models`, {
+      signal: AbortSignal.timeout(5000),
+    });
+    if (!response.ok) {
+      console.warn(`Failed to discover jan.ai models: ${response.status}`);
+      return [];
+    }
+    const data = (await response.json()) as JanModelsResponse;
+    if (!data.data || data.data.length === 0) {
+      console.warn("No jan.ai models found on local instance");
+      return [];
+    }
+    return data.data.map((model) => {
+      const modelId = model.id;
+      const isReasoning =
+        modelId.toLowerCase().includes("r1") || modelId.toLowerCase().includes("reasoning");
+      return {
+        id: modelId,
+        name: modelId,
+        reasoning: isReasoning,
+        input: ["text"],
+        cost: JAN_DEFAULT_COST,
+        contextWindow: JAN_DEFAULT_CONTEXT_WINDOW,
+        maxTokens: JAN_DEFAULT_MAX_TOKENS,
+      };
+    });
+  } catch (error) {
+    console.warn(`Failed to discover jan.ai models: ${String(error)}`);
+    return [];
+  }
+}
+
 function normalizeApiKeyConfig(value: string): string {
  const trimmed = value.trim();
  const match = /^\$\{([A-Z0-9_]+)\}$/.exec(trimmed);
@ -388,6 +449,15 @@ async function buildOllamaProvider(): Promise<ProviderConfig> {
  };
 }

+async function buildJanProvider(): Promise<ProviderConfig> {
+  const models = await discoverJanModels();
+  return {
+    baseUrl: JAN_BASE_URL,
+    api: "openai-completions",
+    models,
+  };
+}
+
 export async function resolveImplicitProviders(params: {
  agentDir: string;
 }): Promise<ModelsConfig["providers"]> {
@ -454,6 +524,14 @@ export async function resolveImplicitProviders(params: {
    providers.ollama = { ...(await buildOllamaProvider()), apiKey: ollamaKey };
  }

+  // jan.ai provider - only add if explicitly configured
+  const janKey =
+    resolveEnvApiKeyVarName("jan") ??
+    resolveApiKeyFromProfiles({ provider: "jan", store: authStore });
+  if (janKey) {
+    providers.jan = { ...(await buildJanProvider()), apiKey: janKey };
+  }
+
  return providers;
 }

--- a/tsconfig.json
+++ b/tsconfig.json
@ -11,7 +11,8 @@
    "skipLibCheck": true,
    "resolveJsonModule": true,
    "noEmitOnError": true,
-    "allowSyntheticDefaultImports": true
+    "allowSyntheticDefaultImports": true,
+    "types": ["node"]
  },
  "include": ["src/**/*"],
  "exclude": [
--- a/vitest.config.ts
+++ b/vitest.config.ts
@ -7,7 +7,7 @@ const repoRoot = path.dirname(fileURLToPath(import.meta.url));
 const isCI = process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true";
 const isWindows = process.platform === "win32";
 const localWorkers = Math.max(4, Math.min(16, os.cpus().length));
-const ciWorkers = isWindows ? 2 : 3;
+const ciWorkers = isWindows ? 1 : 2;

 export default defineConfig({
  resolve: {
@ -16,8 +16,8 @@ export default defineConfig({
    },
  },
  test: {
-    testTimeout: 120_000,
-    hookTimeout: isWindows ? 180_000 : 120_000,
+    testTimeout: 300_000,
+    hookTimeout: isWindows ? 300_000 : 240_000,
    pool: "forks",
    maxWorkers: isCI ? ciWorkers : localWorkers,
    include: [