add fixes
This commit is contained in:
parent
140a640ebf
commit
79a157db62
@ -8,15 +8,16 @@ read_when:
|
|||||||
|
|
||||||
# Cognee Memory Provider
|
# Cognee Memory Provider
|
||||||
|
|
||||||
Clawdbot supports **Cognee** as an optional memory provider. Unlike the default SQLite-based vector memory, Cognee builds a knowledge graph with entity extraction and semantic relationships, providing richer contextual memory for your AI agent.
|
MoltBot supports [**Cognee**](https://www.cognee.ai/) as an optional memory provider. Unlike the default SQLite-based vector memory, Cognee builds a knowledge graph with entity extraction and semantic relationships, providing richer contextual memory for your AI agent.
|
||||||
|
|
||||||
## What is Cognee?
|
## What is Cognee?
|
||||||
|
|
||||||
Cognee is an AI memory framework that:
|
Cognee is an [open source AI memory](https://github.com/topoteretes/cognee) framework that:
|
||||||
|
- Ingests unstructured/structured data from 30+ sources
|
||||||
- Extracts entities (people, places, concepts) from documents
|
- Extracts entities (people, places, concepts) from documents
|
||||||
- Builds a knowledge graph of relationships
|
- Builds a knowledge graph of relationships backed by embeddings
|
||||||
- Enables semantic search with LLM-powered reasoning
|
- Enables semantic search with LLM-powered reasoning
|
||||||
- Supports multiple search modes (GRAPH_COMPLETION, chunks, summaries)
|
- Supports multiple search modes (e.g., GRAPH_COMPLETION, chunks, summaries)
|
||||||
|
|
||||||
Learn more at [docs.cognee.ai](https://docs.cognee.ai/).
|
Learn more at [docs.cognee.ai](https://docs.cognee.ai/).
|
||||||
|
|
||||||
@ -38,6 +39,8 @@ This binds to `127.0.0.1:8000`, so Cognee is only reachable from your machine.
|
|||||||
curl http://localhost:8000/health
|
curl http://localhost:8000/health
|
||||||
```
|
```
|
||||||
|
|
||||||
|
You can use `http://localhost:8000/docs` to register user and login to get your token.
|
||||||
|
|
||||||
### Option 2: Cognee Cloud
|
### Option 2: Cognee Cloud
|
||||||
|
|
||||||
Use the hosted Cognee service:
|
Use the hosted Cognee service:
|
||||||
@ -48,17 +51,16 @@ Use the hosted Cognee service:
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
Clawdbot reads `~/.clawdbot/clawdbot.json` (JSON5). For local + secure setup, use an env var for the Cognee token and reference it in config.
|
Moltbot reads `~/.clawdbot/moltbot.json` (JSON5). For local + secure setup, use an env var for the Cognee token and reference it in config.
|
||||||
|
|
||||||
### Step 1: Put tokens in `examples/.env`
|
### Step 1: Put tokens in `~/.clawdbot/.env`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
LLM_API_KEY="your-llm-api-key"
|
|
||||||
COGNEE_API_KEY="your-cognee-access-token"
|
COGNEE_API_KEY="your-cognee-access-token"
|
||||||
CLAWDBOT_GATEWAY_TOKEN="your-random-gateway-token"
|
CLAWDBOT_GATEWAY_TOKEN="your-random-gateway-token"
|
||||||
```
|
```
|
||||||
|
|
||||||
### Step 2: Configure Cognee in `~/.clawdbot/clawdbot.json`
|
### Step 2: Configure Cognee in `~/.clawdbot/moltbot.json`
|
||||||
|
|
||||||
```json5
|
```json5
|
||||||
{
|
{
|
||||||
@ -87,27 +89,11 @@ CLAWDBOT_GATEWAY_TOKEN="your-random-gateway-token"
|
|||||||
### Step 3: Start the gateway with env loaded
|
### Step 3: Start the gateway with env loaded
|
||||||
|
|
||||||
```zsh
|
```zsh
|
||||||
set -a; source "examples/.env"; set +a; pnpm clawdbot gateway --port 18789 --token "$CLAWDBOT_GATEWAY_TOKEN" --verbose
|
set -a; source "$HOME/.clawdbot/.env"; set +a
|
||||||
|
pnpm moltbot gateway --port 18789 --token "$CLAWDBOT_GATEWAY_TOKEN" --verbose
|
||||||
```
|
```
|
||||||
|
|
||||||
### Cloud Configuration (tbt)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
agents:
|
|
||||||
defaults:
|
|
||||||
memorySearch:
|
|
||||||
enabled: true
|
|
||||||
provider: cognee
|
|
||||||
sources: [memory, sessions]
|
|
||||||
cognee:
|
|
||||||
baseUrl: https://cognee--cognee-saas-backend-serve.modal.run
|
|
||||||
apiKey: your-api-key-here # Required for cloud
|
|
||||||
datasetName: clawdbot
|
|
||||||
searchType: GRAPH_COMPLETION
|
|
||||||
maxResults: 8
|
|
||||||
autoCognify: true
|
|
||||||
timeoutSeconds: 60
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration Options
|
## Configuration Options
|
||||||
|
|
||||||
@ -124,23 +110,9 @@ agents:
|
|||||||
|
|
||||||
## Search Types
|
## Search Types
|
||||||
|
|
||||||
Cognee offers these modes:
|
Cognee offers these modes. GRAPH_COMPLETION is the default mode.
|
||||||
|
|
||||||
### GRAPH_COMPLETION
|
Learn more from [cognee documentation](https://docs.cognee.ai/core-concepts/main-operations/search)
|
||||||
Best for: **High-level understanding and reasoning**
|
|
||||||
- Returns graph-completion outputs over the knowledge graph
|
|
||||||
- Good for: "What projects am I working on?" or "Summarize my notes about X"
|
|
||||||
|
|
||||||
### Chunks
|
|
||||||
Best for: **Specific text matching**
|
|
||||||
- Returns raw document chunks
|
|
||||||
- Similar to traditional vector search
|
|
||||||
- Good for: Finding exact quotes or specific information
|
|
||||||
|
|
||||||
### Summaries
|
|
||||||
Best for: **Document overviews**
|
|
||||||
- Returns condensed summaries
|
|
||||||
- Good for: Quick scanning of content
|
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
@ -167,8 +139,8 @@ agents:
|
|||||||
### Manual Sync and Status
|
### Manual Sync and Status
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Force sync + cognify for the current agent
|
# Manually sync cognee memory for the current agent
|
||||||
clawdbot memory status --index --json
|
pnpm moltbot memory status --index --json
|
||||||
```
|
```
|
||||||
|
|
||||||
## How It Works
|
## How It Works
|
||||||
@ -188,12 +160,12 @@ clawdbot memory status --index --json
|
|||||||
| Feature | Cognee | SQLite (Default) |
|
| Feature | Cognee | SQLite (Default) |
|
||||||
|---------|--------|------------------|
|
|---------|--------|------------------|
|
||||||
| **Setup** | Requires Docker/cloud | Built-in, no setup |
|
| **Setup** | Requires Docker/cloud | Built-in, no setup |
|
||||||
| **Offline** | No (needs service) | Yes (fully local) |
|
| **Offline** | Yes (Optional cloud) | Yes (fully local) |
|
||||||
| **Search** | Knowledge graph + LLM | Vector + BM25 hybrid |
|
| **Search** | Knowledge graph + LLM | Vector + BM25 hybrid |
|
||||||
| **Entities** | Extracted automatically | Not available |
|
| **Entities** | Extracted automatically | Not available |
|
||||||
| **Relationships** | Yes (graph-based) | No |
|
| **Relationships** | Yes (graph-based) | No |
|
||||||
| **Speed** | Slower (API calls) | Faster (local DB) |
|
| **Speed** | Slower (API calls) | Faster (local DB) |
|
||||||
| **Memory** | Stored externally | SQLite file |
|
| **Memory** | Stored locally (Optional cognee configs) | SQLite file |
|
||||||
| **Best for** | Rich context, reasoning | Fast lookup, privacy |
|
| **Best for** | Rich context, reasoning | Fast lookup, privacy |
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
@ -226,11 +198,11 @@ clawdbot memory status --index --json
|
|||||||
|
|
||||||
### 401 Unauthorized (Cognee auth)
|
### 401 Unauthorized (Cognee auth)
|
||||||
|
|
||||||
**Cause**: Cognee auth is enabled, but Clawdbot is missing/using an invalid `COGNEE_API_KEY`.
|
**Cause**: Cognee auth is enabled, but Moltbot is missing/using an invalid `COGNEE_API_KEY`.
|
||||||
|
|
||||||
**Fix**:
|
**Fix**:
|
||||||
1. Log in to Cognee and get a fresh access token.
|
1. Log in to Cognee and get a fresh access token.
|
||||||
2. Update `COGNEE_API_KEY` in `examples/.env`.
|
2. Update `COGNEE_API_KEY` in `.clawd/.env`.
|
||||||
3. Restart the gateway with env loaded (see Step 3 above).
|
3. Restart the gateway with env loaded (see Step 3 above).
|
||||||
|
|
||||||
## Advanced Configuration
|
## Advanced Configuration
|
||||||
@ -252,12 +224,6 @@ agents:
|
|||||||
maxResults: 10
|
maxResults: 10
|
||||||
```
|
```
|
||||||
|
|
||||||
### Hybrid Setup (Not Yet Supported)
|
|
||||||
|
|
||||||
Future versions may support using both Cognee and SQLite:
|
|
||||||
- Cognee for semantic understanding
|
|
||||||
- SQLite for fast local lookup
|
|
||||||
|
|
||||||
## Docker Production Tips
|
## Docker Production Tips
|
||||||
|
|
||||||
### Health Checks
|
### Health Checks
|
||||||
@ -300,13 +266,6 @@ volumes:
|
|||||||
- ./cognee_logs:/app/logs
|
- ./cognee_logs:/app/logs
|
||||||
```
|
```
|
||||||
|
|
||||||
## Roadmap
|
|
||||||
|
|
||||||
Planned features:
|
|
||||||
- [ ] Hybrid mode (Cognee + SQLite)
|
|
||||||
- [ ] Graph visualization export
|
|
||||||
- [ ] Manual entity management
|
|
||||||
|
|
||||||
## Resources
|
## Resources
|
||||||
|
|
||||||
- [Cognee Documentation](https://docs.cognee.ai/)
|
- [Cognee Documentation](https://docs.cognee.ai/)
|
||||||
|
|||||||
@ -2,9 +2,13 @@ import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
|
|||||||
import { CogneeClient } from "./cognee-client.js";
|
import { CogneeClient } from "./cognee-client.js";
|
||||||
import { request } from "undici";
|
import { request } from "undici";
|
||||||
|
|
||||||
vi.mock("undici", () => ({
|
vi.mock("undici", async () => {
|
||||||
request: vi.fn(),
|
const actual = await vi.importActual<typeof import("undici")>("undici");
|
||||||
}));
|
return {
|
||||||
|
...actual,
|
||||||
|
request: vi.fn(),
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
describe("CogneeClient", () => {
|
describe("CogneeClient", () => {
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
@ -73,7 +77,9 @@ describe("CogneeClient", () => {
|
|||||||
data: "Test data",
|
data: "Test data",
|
||||||
datasetName: "test-dataset",
|
datasetName: "test-dataset",
|
||||||
}),
|
}),
|
||||||
).rejects.toThrow("Cognee add failed with status 500");
|
).rejects.toThrow(
|
||||||
|
"Cognee add request failed: Cognee add failed with status 500: Internal server error",
|
||||||
|
);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@ -295,16 +295,37 @@ export class CogneeClient {
|
|||||||
|
|
||||||
if (item && typeof item === "object") {
|
if (item && typeof item === "object") {
|
||||||
const record = item as Record<string, unknown>;
|
const record = item as Record<string, unknown>;
|
||||||
|
const hasStructuredFields =
|
||||||
|
"id" in record || "text" in record || "score" in record || "metadata" in record;
|
||||||
const raw =
|
const raw =
|
||||||
record.search_result ?? record.result ?? record.context ?? record.text ?? record;
|
record.search_result ?? record.result ?? record.context ?? record.text ?? record;
|
||||||
const text = typeof raw === "string" ? raw : JSON.stringify(raw, null, 2);
|
const text = typeof raw === "string" ? raw : JSON.stringify(raw, null, 2);
|
||||||
const metadata =
|
const datasetMetadata =
|
||||||
record.dataset_name || record.dataset_id
|
record.dataset_name || record.dataset_id
|
||||||
? {
|
? {
|
||||||
datasetName: record.dataset_name,
|
datasetName: record.dataset_name,
|
||||||
datasetId: record.dataset_id,
|
datasetId: record.dataset_id,
|
||||||
}
|
}
|
||||||
: undefined;
|
: undefined;
|
||||||
|
const recordMetadata =
|
||||||
|
record.metadata && typeof record.metadata === "object"
|
||||||
|
? (record.metadata as Record<string, unknown>)
|
||||||
|
: undefined;
|
||||||
|
const metadata = recordMetadata
|
||||||
|
? datasetMetadata
|
||||||
|
? { ...datasetMetadata, ...recordMetadata }
|
||||||
|
: recordMetadata
|
||||||
|
: datasetMetadata;
|
||||||
|
|
||||||
|
if (hasStructuredFields) {
|
||||||
|
return {
|
||||||
|
id: typeof record.id === "string" ? record.id : `result-${index}`,
|
||||||
|
text,
|
||||||
|
score: typeof record.score === "number" ? record.score : 0,
|
||||||
|
metadata,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
return { id: `result-${index}`, text, score: 0, metadata };
|
return { id: `result-${index}`, text, score: 0, metadata };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -2,19 +2,19 @@ import { describe, expect, it, vi, beforeEach } from "vitest";
|
|||||||
import { CogneeMemoryProvider } from "./cognee-provider.js";
|
import { CogneeMemoryProvider } from "./cognee-provider.js";
|
||||||
import type { ClawdbotConfig } from "../config/config.js";
|
import type { ClawdbotConfig } from "../config/config.js";
|
||||||
|
|
||||||
vi.mock("./cognee-client.js", () => ({
|
vi.mock("./cognee-client.js", () => {
|
||||||
CogneeClient: vi.fn().mockImplementation(() => ({
|
class CogneeClient {
|
||||||
healthCheck: vi.fn().mockResolvedValue(true),
|
healthCheck = vi.fn().mockResolvedValue(true);
|
||||||
add: vi.fn().mockResolvedValue({
|
add = vi.fn().mockResolvedValue({
|
||||||
datasetId: "test-dataset-id",
|
datasetId: "test-dataset-id",
|
||||||
datasetName: "test-dataset",
|
datasetName: "test-dataset",
|
||||||
message: "Success",
|
message: "Success",
|
||||||
}),
|
});
|
||||||
cognify: vi.fn().mockResolvedValue({
|
cognify = vi.fn().mockResolvedValue({
|
||||||
status: "success",
|
status: "success",
|
||||||
message: "Cognify completed",
|
message: "Cognify completed",
|
||||||
}),
|
});
|
||||||
search: vi.fn().mockResolvedValue({
|
search = vi.fn().mockResolvedValue({
|
||||||
results: [
|
results: [
|
||||||
{
|
{
|
||||||
id: "result-1",
|
id: "result-1",
|
||||||
@ -28,8 +28,8 @@ vi.mock("./cognee-client.js", () => ({
|
|||||||
],
|
],
|
||||||
query: "test query",
|
query: "test query",
|
||||||
searchType: "GRAPH_COMPLETION",
|
searchType: "GRAPH_COMPLETION",
|
||||||
}),
|
});
|
||||||
status: vi.fn().mockResolvedValue({
|
status = vi.fn().mockResolvedValue({
|
||||||
status: "healthy",
|
status: "healthy",
|
||||||
version: "1.0.0",
|
version: "1.0.0",
|
||||||
datasets: [
|
datasets: [
|
||||||
@ -39,9 +39,11 @@ vi.mock("./cognee-client.js", () => ({
|
|||||||
documentCount: 5,
|
documentCount: 5,
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
}),
|
});
|
||||||
})),
|
}
|
||||||
}));
|
|
||||||
|
return { CogneeClient };
|
||||||
|
});
|
||||||
|
|
||||||
vi.mock("./internal.js", () => ({
|
vi.mock("./internal.js", () => ({
|
||||||
listMemoryFiles: vi.fn().mockResolvedValue([]),
|
listMemoryFiles: vi.fn().mockResolvedValue([]),
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user