openclaw/extensions/memory-ruvector/PR_DESCRIPTION.md

feat(memory): Add ruvector Vector Database Plugin

Summary

This PR introduces @clawdbot/memory-ruvector, a next-generation memory system that brings self-learning AI, graph neural networks, and extreme performance to Clawdbot.

Why This Matters

Capability	memory-ruvector	Current Memory
Self-Learning	SONA learns from user feedback automatically	Static, requires manual tuning
Graph Intelligence	GNN discovers message relationships	No relationship awareness
Query Speed	61μs p50 (16,400 QPS)	10-100ms typical
Memory Efficiency	200MB for 1M vectors	2-4GB for same data
Context Injection	Auto-injects relevant memories	Manual search required
Pattern Recognition	K-means++ with EWC++ consolidation	None

Key Innovations

SONA (Self-Organizing Neural Architecture) - Memory that gets smarter. Every search, every feedback signal improves future results. No retraining, no manual intervention.

Graph Neural Networks - Messages form a knowledge graph. Cypher queries reveal conversation threads, user patterns, and topic clusters that flat vector search can't see.

ruvLLM Adaptive Learning - Three learning loops (instant/background/consolidation) continuously optimize search while EWC++ prevents catastrophic forgetting.

Rust Performance - Native HNSW indexing delivers 100x faster queries with 10-20x less memory.

Production Highlights

• Semantic memory with automatic conversation indexing
• RAG-ready architecture for knowledge base integration
• Multiple embedding providers (OpenAI, Voyage AI, local)
• Graceful degradation and comprehensive error handling
• 275 tests covering all features

Architecture

Configuration

plugins:
  memory-ruvector:
    embedding:
      provider: openai
      apiKey: ${OPENAI_API_KEY}
      model: text-embedding-3-small
    dbPath: ~/.clawdbot/memory/ruvector
    hooks:
      enabled: true

File Structure

extensions/memory-ruvector/
├── index.ts              # Plugin registration and tool setup
├── service.ts            # Lifecycle management (start/stop), SONA + Graph init
├── client.ts             # RuvectorClient wrapper for native API
├── db.ts                 # High-level database abstraction
├── embeddings.ts         # Multi-provider embedding support
├── hooks.ts              # Auto-indexing via message hooks
├── tool.ts               # Agent tools (search, feedback, graph, recall, learn)
├── config.ts             # Configuration schema with validation
├── types.ts              # TypeScript type definitions
├── context-injection.ts  # Context injection for agent prompts
├── sona/
│   ├── trajectory.ts     # Trajectory recording for search patterns
│   ├── patterns.ts       # K-means++ pattern clustering
│   ├── ewc.ts            # EWC++ consolidation (catastrophic forgetting prevention)
│   └── loops/
│       ├── index.ts      # Loop exports
│       ├── instant.ts    # Instant learning (real-time feedback)
│       ├── background.ts # Background learning (pattern clustering)
│       └── consolidation.ts # Deep consolidation (EWC++ integration)
├── graph/
│   ├── index.ts          # Graph exports
│   ├── expansion.ts      # Automatic edge discovery
│   ├── attention.ts      # Multi-head graph attention
│   └── relationships.ts  # Entity extraction & relationship inference
├── index.test.ts         # Vitest test suite (229 tests)
├── p1-ruvllm.test.ts     # ruvLLM P1 feature tests (46 tests)
├── package.json          # Dependencies
└── tsconfig.json         # TypeScript config

Features

1. Automatic Message Indexing

Messages are automatically indexed via clawdbot hooks:

Hook	Purpose
message_received	Index incoming user messages
message_sent	Index outgoing bot responses
agent_end	Index full agent conversation turns

Smart Batching: Messages are batched (default: 10) with debouncing (default: 500ms) to optimize database writes and embedding API calls.

Content Filtering: System markers, commands (/), and very short/long messages are automatically filtered out.

2. Semantic Search Tool

Agents can search conversation history using natural language:

// Tool: ruvector_search
{
  query: "What did the user say about their preferences?",
  limit: 5,
  direction: "inbound",  // Optional: filter by direction
  channel: "telegram"    // Optional: filter by channel
}

3. Manual Indexing Tool

For explicit memory storage:

// Tool: ruvector_index
{
  content: "User prefers dark mode and minimal notifications",
  direction: "outbound",
  channel: "system"
}

4. CLI Commands

# Show memory statistics
clawdbot ruvector stats

# Search indexed messages
clawdbot ruvector search "user preferences" --limit 10 --direction inbound

# Force flush pending batch
clawdbot ruvector flush

5. Multiple Embedding Providers

Provider	Models	Dimensions	Notes
OpenAI	text-embedding-3-small/large	1536/3072	Default
Voyage AI	voyage-3, voyage-3-large, voyage-code-3	1024	Best for RAG
Local	Any OpenAI-compatible API	Configurable	Self-hosted

Auto-dimension detection based on model name.

6. ruvLLM Adaptive Learning

Context Injection

Relevant memories are automatically injected into agent system prompts:

// Enabled via config
ruvllm: {
  enabled: true,
  contextInjection: {
    enabled: true,
    maxTokens: 2000,
    relevanceThreshold: 0.3
  }
}

Trajectory Recording

Search queries and results are recorded for learning:

{
  id: "traj-abc123",
  query: "user preferences",
  queryVector: [...],
  results: [...],
  feedback: 0.85,
  timestamp: 1706123456789
}

Pattern Learning Tools

ruvector_recall - Pattern-aware memory recall:

{
  query: "What are the user's coding preferences?",
  usePatterns: true,    // Apply learned pattern re-ranking
  expandGraph: true,    // Include graph-connected memories
  graphDepth: 2,        // Depth for graph traversal
  patternBoost: 0.2     // Boost factor for pattern matches
}

ruvector_learn - Manual knowledge injection:

{
  content: "User prefers TypeScript over JavaScript",
  category: "preference",
  importance: 0.8,
  relationships: ["msg-123"],
  inferRelationships: true,
  linkSimilar: true
}

Multi-Temporal Learning Loops

Loop	Interval	Purpose
Instant	Immediate	Process feedback in real-time, apply micro-boosts
Background	30s	Cluster recent trajectories, update pattern store
Consolidation	5min	Deep reanalysis, merge patterns, prune stale data

EWC++ Consolidation

Prevents catastrophic forgetting by:

• Tracking pattern importance via Fisher Information Matrix
• Protecting critical patterns during consolidation
• Computing penalties for modifying important patterns

Graph Attention

Multi-head attention aggregates context from graph neighbors:

• Semantic head: Weights by content similarity
• Temporal head: Weights by time proximity
• Causal head: Weights by cause-effect relationships
• Structural head: Weights by graph structure

Pattern Export/Import

clawdbot ruvector export-patterns ./patterns.json
clawdbot ruvector import-patterns ./patterns.json --merge
clawdbot ruvector pattern-stats

Implementation Details

Error Handling

• Connection failures: Graceful fallback to in-memory storage
• Embedding API errors: 30-second timeout, response validation, dimension checking
• Service unavailable: Tools return disabled: true response
• Batch failures: Retry with limits, reject pending on shutdown

Resource Management

• Timer cleanup: All timers cleared on destroy
• Promise handling: Pending promises rejected on shutdown
• Connection lifecycle: Proper connect/disconnect with deduplication
• Batcher shutdown: forceFlush() with 30s timeout and 3 retry limit

Type Safety

• Zero any types
• Custom RuvectorError class with error codes
• Comprehensive TypeScript interfaces
• Runtime validation for API responses

Configuration Validation

• Environment variable resolution (${VAR_NAME} syntax)
• Unknown key detection with helpful error messages
• Required field validation (apiKey for non-local providers)
• Dimension auto-detection from model name

Test Coverage

275 test cases covering:

• RuvectorClient operations (connect, insert, search, delete)
• RuvectorService lifecycle
• Configuration parsing and validation
• EmbeddingProvider API calls
• MessageBatcher batching behavior
• Content filtering logic
• Tool parameter validation
• Error handling paths
• SONA self-learning (enable, feedback recording, pattern finding, stats)
• Graph features (init, edge management, Cypher queries, neighbors, message linking)
• ruvLLM Config - Config parsing with ruvllm options
• TrajectoryRecorder - record(), getRecent(), prune(), findSimilar(), import/export
• ContextInjector - injectContext(), formatContext(), buildContextForMessage()
• PatternStore - addSample(), cluster(), findSimilar(), export/import
• GraphExpander - expandFromSearch(), suggestRelationships()
• BackgroundLoop - start(), stop(), runCycle(), pattern learning
• InstantLoop - processImmediateFeedback(), getBoostForVector(), decay
• RelationshipInferrer - inferFromContent(), linkSimilar(), entity extraction
• EWCConsolidator - consolidate(), protectCritical(), computePenalty()
• ConsolidationLoop - runDeepConsolidation(), exportPatterns(), importPatterns()
• GraphAttention - aggregateContext(), addHead(), multi-head attention
• ruvector_recall tool - pattern-aware recall with graph expansion
• ruvector_learn tool - content indexing with relationships

Dependencies

{
  "dependencies": {
    "@sinclair/typebox": "0.34.47",
    "ruvector": "0.1.96"
  },
  "devDependencies": {
    "clawdbot": "workspace:*"
  },
  "peerDependencies": {
    "clawdbot": "*"
  }
}

Performance Characteristics

Based on ruvector benchmarks:

• Query Latency: p50 61μs, p99 < 1ms
• Throughput: 16,400 QPS (k=10, 1536-dim vectors)
• Memory: 200MB for 1M vectors with compression
• Index Build: O(n log n) with HNSW

Migration Path

For users of memory-lancedb:

1. Both plugins can coexist - different plugin IDs
2. Similar configuration structure
3. Same embedding provider options
4. Compatible tool interface patterns

Breaking Changes

None - this is a new optional plugin.

Checklist

• Plugin follows clawdbot extension patterns
• Comprehensive TypeScript types
• Error handling with graceful degradation
• Test coverage (275 tests)
• CLI commands registered
• Documentation (plugin docs, SONA, Graph queries, ruvLLM)
• Configuration validation
• Resource cleanup on shutdown
• SONA self-learning implementation
• Cypher graph query support
• ruvLLM adaptive learning (trajectory recording, context injection)
• Pattern clustering with K-means++
• Multi-temporal learning loops (instant, background, consolidation)
• EWC++ consolidation for catastrophic forgetting prevention
• Multi-head graph attention
• Pattern export/import CLI commands
• ruvector_recall and ruvector_learn tools

Test Plan

• Run npx vitest run extensions/memory-ruvector (275 tests pass)
• Verify plugin loads: clawdbot config get plugins
• Test local mode with OpenAI embeddings
• Test CLI commands: clawdbot ruvector stats
• Send messages and verify auto-indexing
• Test search tool via agent interaction
• Verify graceful shutdown flushes pending batch
• Test ruvLLM features: clawdbot ruvector ruvllm-status
• Test pattern export/import: clawdbot ruvector export-patterns

Documentation

• Plugin docs: docs/plugins/memory-ruvector.md
• Configuration: See config.ts uiHints for all options

---

Generated with Claude Code