docs(security): add secret detection documentation
This commit is contained in:
Rome Thorstenson
parent
af64c12977
commit
ed698b0256
249
docs/security/secret-handling.md
Normal file
249
docs/security/secret-handling.md
Normal file
@ -0,0 +1,249 @@
|
||||
# Secret Detection & Redaction
|
||||
|
||||
Moltbot includes built-in secret detection to protect API keys, tokens, and other sensitive credentials from being accidentally sent to AI models.
|
||||
|
||||
## Overview
|
||||
|
||||
The secret detection system:
|
||||
- **Detects** high-entropy strings and known secret patterns in messages
|
||||
- **Prompts** users interactively to choose how to handle detected secrets
|
||||
- **Redacts** secrets based on user choice or configured defaults
|
||||
- **Prevents** secrets from reaching the AI model's conversation history
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. Detection Phase
|
||||
|
||||
When you send a message, Moltbot automatically scans for:
|
||||
|
||||
- **Known patterns**: OpenAI API keys, GitHub tokens, AWS keys, JWT tokens, private keys, etc.
|
||||
- **High-entropy strings**: Random-looking sequences likely to be secrets (using Shannon entropy analysis)
|
||||
- **Custom patterns**: Additional regex patterns you configure
|
||||
|
||||
### 2. Interactive Prompt (Default Behavior)
|
||||
|
||||
If secrets are detected, you'll receive a security alert:
|
||||
|
||||
```
|
||||
🔒 Security Alert
|
||||
|
||||
Your message contains what appears to be 1 secret or API key:
|
||||
|
||||
• sk-proj-Ab12...Yz56 (OpenAI API key)
|
||||
|
||||
Options:
|
||||
1️⃣ Redact - Replace with [REDACTED] before processing
|
||||
2️⃣ Cancel - Don't process this message
|
||||
3️⃣ Continue anyway ⚠️ - Send to AI as-is (not recommended)
|
||||
|
||||
Reply with 1, 2, or 3 (timeout in 15s)
|
||||
```
|
||||
|
||||
**Simply reply with your choice** (1-3) and Moltbot will apply the selected action.
|
||||
|
||||
### 3. Actions
|
||||
|
||||
#### 1. Redact (Recommended)
|
||||
- Replaces the secret with `[REDACTED]` in your message
|
||||
- The AI sees `[REDACTED]` instead of the secret value
|
||||
- Protects against accidental exposure in conversation history
|
||||
|
||||
#### 2. Cancel
|
||||
- Blocks the message from being processed
|
||||
- Nothing is sent to the AI
|
||||
- Useful if you sent the message by mistake
|
||||
|
||||
#### 3. Continue Anyway
|
||||
- Sends the message to the AI as-is, including the secret
|
||||
- **Not recommended** - secrets may be stored in conversation history and logs
|
||||
- Only use if you're certain the string is not actually a secret
|
||||
|
||||
### 4. Timeout Behavior
|
||||
|
||||
If you don't respond within **15 seconds** (configurable), the system applies the **default action** (redact by default).
|
||||
|
||||
## Configuration
|
||||
|
||||
### Enable/Disable Detection
|
||||
|
||||
```json5
|
||||
{
|
||||
"security": {
|
||||
"secrets": {
|
||||
"detection": {
|
||||
"enabled": true // Default: true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Interactive vs. Automatic Mode
|
||||
|
||||
```json5
|
||||
{
|
||||
"security": {
|
||||
"secrets": {
|
||||
"handling": {
|
||||
"interactive": true, // Default: true (prompt user)
|
||||
"defaultAction": "redact", // Default: "redact" (options: redact, block, allow)
|
||||
"confirmationTimeoutMs": 15000 // Default: 15 seconds
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Non-interactive mode**: Set `interactive: false` to automatically apply `defaultAction` without prompting.
|
||||
|
||||
### Detection Thresholds
|
||||
|
||||
```json5
|
||||
{
|
||||
"security": {
|
||||
"secrets": {
|
||||
"detection": {
|
||||
"minEntropyThreshold": 4.5, // Default: 4.5 (Shannon entropy)
|
||||
"minLength": 24, // Default: 24 characters
|
||||
"customPatterns": [
|
||||
// Additional regex patterns to detect
|
||||
"MYAPP-[A-Za-z0-9]{32}"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Detected Patterns
|
||||
|
||||
Moltbot recognizes these secret types:
|
||||
|
||||
### API Keys & Tokens
|
||||
- **OpenAI**: `sk-proj-...` (48+ chars), `sk-...` (48 chars legacy)
|
||||
- **Anthropic**: `sk-ant-...` (95+ chars)
|
||||
- **Google**: `AIza...` (39 chars)
|
||||
- **GitHub**: `ghp_...`, `gho_...`, `ghs_...`, `ghr_...` (36 chars)
|
||||
- **AWS**: `AKIA...` (20 chars), `aws_secret_access_key=...`
|
||||
- **Slack**: `xox[baprs]-...`
|
||||
- **Stripe**: `sk_live_...`, `rk_live_...`
|
||||
- **Twilio**: `SK[0-9a-fA-F]{32}`
|
||||
- **SendGrid**: `SG....`
|
||||
- **Mailgun**: `key-...`
|
||||
|
||||
### Other Formats
|
||||
- **Bearer tokens**: `Bearer ...`
|
||||
- **JWT tokens**: `eyJ...eyJ....` (3-part base64)
|
||||
- **Private keys**: `-----BEGIN ... PRIVATE KEY-----`
|
||||
|
||||
### Generic Patterns
|
||||
- `api_key = "..."`
|
||||
- `secret: "..."`
|
||||
- `token = "..."`
|
||||
|
||||
## Security Logging
|
||||
|
||||
Security events are logged to the `security:events` subsystem:
|
||||
|
||||
- `high_entropy_detected` - When secrets are found
|
||||
- `secret_redacted` - When a secret is redacted
|
||||
- `secret_allowed_by_user` - When user chooses "continue anyway"
|
||||
- `interactive_prompt_timeout` - When user doesn't respond in time
|
||||
- `interactive_prompt_cancelled` - When user chooses "cancel"
|
||||
|
||||
View security logs:
|
||||
```bash
|
||||
moltbot logs --subsystem security:events
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Auto-Redact
|
||||
|
||||
**You send:**
|
||||
```
|
||||
Can you help me debug this API call? I'm using key sk-proj-Ab12Cd34Ef56Gh78Ij90Kl12Mn34Op56Qr78St90Uv12Wx34Yz56
|
||||
```
|
||||
|
||||
**Moltbot responds:**
|
||||
```
|
||||
🔒 Security Alert
|
||||
Your message contains what appears to be 1 secret or API key:
|
||||
• sk-proj-Ab12...Yz56 (OpenAI API key)
|
||||
|
||||
Options: (reply with 1, 2, or 3)
|
||||
1️⃣ Redact
|
||||
2️⃣ Cancel
|
||||
3️⃣ Continue anyway ⚠️
|
||||
```
|
||||
|
||||
**You reply:**
|
||||
```
|
||||
1
|
||||
```
|
||||
|
||||
**Moltbot applies action:**
|
||||
```
|
||||
✓ Applying action: redact
|
||||
```
|
||||
|
||||
**AI receives:**
|
||||
```
|
||||
Can you help me debug this API call? I'm using key [REDACTED]
|
||||
```
|
||||
|
||||
### Example 2: Auto-Redact (Non-Interactive Mode)
|
||||
|
||||
**Config:**
|
||||
```json5
|
||||
{
|
||||
"security": {
|
||||
"secrets": {
|
||||
"handling": {
|
||||
"interactive": false,
|
||||
"defaultAction": "redact"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**You send:**
|
||||
```
|
||||
My GitHub token is ghp_Ab12Cd34Ef56Gh78Ij90Kl12Mn34Op5678Qr
|
||||
```
|
||||
|
||||
**AI receives immediately:**
|
||||
```
|
||||
My GitHub token is [REDACTED]
|
||||
```
|
||||
|
||||
## Opting Out
|
||||
|
||||
To completely disable secret detection:
|
||||
|
||||
```json5
|
||||
{
|
||||
"security": {
|
||||
"secrets": {
|
||||
"detection": {
|
||||
"enabled": false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Warning**: Disabling detection means secrets may be sent to AI models and stored in conversation history.
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
**Phase 4 (Planned):**
|
||||
- **Secure storage**: Store detected secrets in OS keychain (macOS Keychain, Windows Credential Manager, Linux Secret Service)
|
||||
- **Secret retrieval**: CLI commands to list/retrieve stored secrets
|
||||
- **Gitleaks integration**: Optional backend for enhanced detection using [Gitleaks](https://gitleaks.io/)
|
||||
|
||||
## Related
|
||||
|
||||
- [Configuration Guide](/configuration)
|
||||
- [Security Audit](/security/audit)
|
||||
Loading…
Reference in New Issue
Block a user