feat: add workspace cloud sync via rclone
Add bidirectional workspace synchronization with cloud storage providers (Dropbox, Google Drive, OneDrive, S3/R2) using rclone. Features: - Config-driven setup via workspace.sync in moltbot.json - Native CLI: moltbot workspace setup/sync/status/authorize/list - Session hooks: onSessionStart/onSessionEnd triggers (no LLM cost) - Background interval sync via gateway (no LLM cost) - Auto-install rclone when missing (Homebrew/official script) - Dropbox app folder support for ringfenced security - Secure defaults: syncs shared/ subdirectory only New files: - src/infra/rclone.ts - rclone wrapper - src/cli/workspace-cli.ts - CLI commands - src/hooks/bundled/workspace-sync/ - session hooks - src/gateway/workspace-sync-manager.ts - background sync - src/config/types.workspace.ts - config types - docs/gateway/workspace-sync.md - full documentation Tests: 23 unit tests for rclone helpers and hook handler
This commit is contained in:
Ash Brener
parent
9688454a30
commit
7e0cd65156
@ -223,6 +223,21 @@ Suggested `.gitignore` starter:
|
|||||||
4. If you need sessions, copy `~/.clawdbot/agents/<agentId>/sessions/` from the
|
4. If you need sessions, copy `~/.clawdbot/agents/<agentId>/sessions/` from the
|
||||||
old machine separately.
|
old machine separately.
|
||||||
|
|
||||||
|
## Cloud sync (remote Gateways)
|
||||||
|
|
||||||
|
If your Gateway runs on a remote server (Fly.io, Hetzner, VPS), you can sync a
|
||||||
|
workspace subfolder with your local machine using cloud storage (Dropbox, Google
|
||||||
|
Drive, OneDrive, S3, etc.):
|
||||||
|
|
||||||
|
```
|
||||||
|
Local Machine Cloud Provider Remote Gateway
|
||||||
|
~/Dropbox/clawd/ ←→ Dropbox/GDrive/etc ←→ <workspace>/shared/
|
||||||
|
```
|
||||||
|
|
||||||
|
This lets you drop files locally and have them appear on the remote Gateway.
|
||||||
|
|
||||||
|
See [Workspace cloud sync](/gateway/workspace-sync) for setup instructions.
|
||||||
|
|
||||||
## Advanced notes
|
## Advanced notes
|
||||||
|
|
||||||
- Multi-agent routing can use different workspaces per agent. See
|
- Multi-agent routing can use different workspaces per agent. See
|
||||||
|
|||||||
@ -965,7 +965,8 @@
|
|||||||
"gateway/remote-gateway-readme",
|
"gateway/remote-gateway-readme",
|
||||||
"gateway/discovery",
|
"gateway/discovery",
|
||||||
"gateway/bonjour",
|
"gateway/bonjour",
|
||||||
"gateway/tailscale"
|
"gateway/tailscale",
|
||||||
|
"gateway/workspace-sync"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
400
docs/gateway/workspace-sync.md
Normal file
400
docs/gateway/workspace-sync.md
Normal file
@ -0,0 +1,400 @@
|
|||||||
|
---
|
||||||
|
summary: "Sync your agent workspace with cloud storage (Dropbox, Google Drive, OneDrive, S3)"
|
||||||
|
read_when:
|
||||||
|
- Setting up workspace sync on a remote/cloud Gateway
|
||||||
|
- Sharing files between local machine and remote Moltbot
|
||||||
|
---
|
||||||
|
|
||||||
|
# Workspace Cloud Sync
|
||||||
|
|
||||||
|
Sync your agent workspace between a remote Gateway (Fly.io, Hetzner, VPS) and your local machine using cloud storage.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
```
|
||||||
|
Local Machine Cloud Provider Remote Gateway
|
||||||
|
~/Dropbox/moltbot/ ←→ Dropbox/GDrive/etc ←→ <workspace>/shared/
|
||||||
|
(native app) (any provider) (rclone bisync)
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Local**: Native cloud app syncs `~/Dropbox/moltbot/` (or equivalent)
|
||||||
|
- **Remote**: rclone bisync keeps `<workspace>/shared/` in sync with the cloud
|
||||||
|
- **Result**: Drop a file locally, it appears on the remote Gateway (and vice versa)
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Interactive setup wizard (recommended)
|
||||||
|
moltbot workspace setup
|
||||||
|
```
|
||||||
|
|
||||||
|
The setup wizard guides you through:
|
||||||
|
1. ✅ Checking rclone installation
|
||||||
|
2. 📦 Selecting cloud provider (Dropbox, Google Drive, OneDrive, S3)
|
||||||
|
3. 🔐 Dropbox app folder option (for scoped access)
|
||||||
|
4. ⏱️ Background sync interval
|
||||||
|
5. 🔑 OAuth authorization
|
||||||
|
6. 🔄 First sync
|
||||||
|
|
||||||
|
Or configure manually:
|
||||||
|
|
||||||
|
**1) Add minimal config:**
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**2) Authorize and sync:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
moltbot workspace authorize # opens browser for OAuth
|
||||||
|
moltbot workspace sync --resync # first sync (establishes baseline)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Add to `~/.clawdbot/moltbot.json`:
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox", // dropbox | gdrive | onedrive | s3 | custom
|
||||||
|
remotePath: "moltbot-share", // folder in cloud storage
|
||||||
|
localPath: "shared", // subfolder in workspace (default: shared)
|
||||||
|
interval: 300, // background sync every 5 minutes (0 = disabled)
|
||||||
|
onSessionStart: true, // sync when session starts
|
||||||
|
onSessionEnd: false, // sync when session ends
|
||||||
|
conflictResolve: "newer", // newer | local | remote
|
||||||
|
exclude: [".git/**", "node_modules/**", "*.log"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
**Zero LLM cost.** The `interval` setting runs pure rclone in the background.
|
||||||
|
It does NOT wake the bot or trigger any LLM calls - it's just file synchronization.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
### Provider-specific options
|
||||||
|
|
||||||
|
**Dropbox with app folder (recommended):**
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "", // empty = app folder root
|
||||||
|
dropbox: {
|
||||||
|
appFolder: true,
|
||||||
|
appKey: "your-app-key",
|
||||||
|
appSecret: "your-app-secret"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**S3/R2/Minio:**
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "s3",
|
||||||
|
remotePath: "moltbot-sync", // path within bucket
|
||||||
|
s3: {
|
||||||
|
// Cloudflare R2: https://<ACCOUNT_ID>.r2.cloudflarestorage.com
|
||||||
|
// AWS S3: https://s3.<REGION>.amazonaws.com
|
||||||
|
// Minio: https://your-minio-host:9000
|
||||||
|
endpoint: "https://abc123def456.r2.cloudflarestorage.com",
|
||||||
|
bucket: "your-bucket",
|
||||||
|
region: "auto" // "auto" for R2, or specific region for AWS
|
||||||
|
// accessKeyId and secretAccessKey via env vars recommended
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## CLI commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Interactive setup wizard
|
||||||
|
moltbot workspace setup
|
||||||
|
|
||||||
|
# Check sync status
|
||||||
|
moltbot workspace status
|
||||||
|
|
||||||
|
# Sync bidirectionally
|
||||||
|
moltbot workspace sync
|
||||||
|
|
||||||
|
# First sync (required to establish baseline)
|
||||||
|
moltbot workspace sync --resync
|
||||||
|
|
||||||
|
# Preview changes without syncing
|
||||||
|
moltbot workspace sync --dry-run
|
||||||
|
|
||||||
|
# One-way sync
|
||||||
|
moltbot workspace sync --direction pull # remote → local
|
||||||
|
moltbot workspace sync --direction push # local → remote
|
||||||
|
|
||||||
|
# Authorize with cloud provider (use 'setup' for guided flow)
|
||||||
|
moltbot workspace authorize
|
||||||
|
moltbot workspace authorize --provider gdrive
|
||||||
|
|
||||||
|
# List remote files
|
||||||
|
moltbot workspace list
|
||||||
|
```
|
||||||
|
|
||||||
|
## Auto-sync hooks
|
||||||
|
|
||||||
|
Enable automatic sync on session start/end. These hooks run during existing agent activity,
|
||||||
|
so they **don't wake the bot** or incur extra LLM costs:
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true, // sync when session starts (no LLM cost)
|
||||||
|
onSessionEnd: false // sync when session ends (no LLM cost)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
hooks: {
|
||||||
|
internal: {
|
||||||
|
entries: {
|
||||||
|
"workspace-sync": { enabled: true }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Periodic background sync
|
||||||
|
|
||||||
|
Set `interval` to enable automatic background sync (in seconds):
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
interval: 300 // sync every 5 minutes (minimum: 60s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The gateway runs rclone bisync in the background at this interval.
|
||||||
|
This is a **pure file operation** - it does NOT wake the bot or incur any LLM costs.
|
||||||
|
|
||||||
|
### Alternative: External cron
|
||||||
|
|
||||||
|
If you prefer external scheduling (e.g., for more control or logging):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add to crontab (crontab -e)
|
||||||
|
*/5 * * * * moltbot workspace sync >> /var/log/moltbot-sync.log 2>&1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Supported providers
|
||||||
|
|
||||||
|
| Provider | Config value | Auth method |
|
||||||
|
|----------|--------------|-------------|
|
||||||
|
| Dropbox | `dropbox` | OAuth token |
|
||||||
|
| Google Drive | `gdrive` | OAuth token |
|
||||||
|
| OneDrive | `onedrive` | OAuth token |
|
||||||
|
| S3/R2/Minio | `s3` | Access keys |
|
||||||
|
| Custom rclone | `custom` | Varies |
|
||||||
|
|
||||||
|
For the full list of 70+ providers, see [rclone overview](https://rclone.org/overview/).
|
||||||
|
|
||||||
|
## Manual setup (without wizard)
|
||||||
|
|
||||||
|
If you prefer manual configuration:
|
||||||
|
|
||||||
|
### 1. Install rclone
|
||||||
|
|
||||||
|
rclone is **auto-installed** when you run `moltbot workspace setup`.
|
||||||
|
|
||||||
|
For manual installation:
|
||||||
|
- **macOS**: `brew install rclone`
|
||||||
|
- **Linux**: `curl -s https://rclone.org/install.sh | sudo bash`
|
||||||
|
- **Docker**: Add to Dockerfile: `RUN curl -s https://rclone.org/install.sh | bash`
|
||||||
|
|
||||||
|
### 2. Authorize rclone (from your local machine)
|
||||||
|
|
||||||
|
Run this on your **local machine** (where you have a browser):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install rclone locally if needed
|
||||||
|
brew install rclone # or: curl -s https://rclone.org/install.sh | bash
|
||||||
|
|
||||||
|
# Authorize with your cloud provider
|
||||||
|
rclone authorize "dropbox" # or: gdrive, onedrive, s3, etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
Copy the JSON token it outputs.
|
||||||
|
|
||||||
|
### 3. Configure rclone on the Gateway
|
||||||
|
|
||||||
|
SSH into your Gateway and create the config:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p /data/workspace/.config/rclone
|
||||||
|
|
||||||
|
cat > /data/workspace/.config/rclone/rclone.conf << 'EOF'
|
||||||
|
[cloud]
|
||||||
|
type = dropbox
|
||||||
|
token = {"access_token":"YOUR_TOKEN_HERE","token_type":"bearer","expiry":"..."}
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
|
For other providers, see [rclone config docs](https://rclone.org/docs/).
|
||||||
|
|
||||||
|
### 4. Create the sync folder
|
||||||
|
|
||||||
|
**On your local machine:**
|
||||||
|
|
||||||
|
Create the folder your cloud app syncs (e.g., `~/Dropbox/moltbot-share/`).
|
||||||
|
|
||||||
|
**On the Gateway:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p /data/workspace/shared
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Run the first sync
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# First sync needs --resync to establish baseline
|
||||||
|
rclone bisync cloud:moltbot-share /data/workspace/shared --resync
|
||||||
|
|
||||||
|
# Subsequent syncs
|
||||||
|
rclone bisync cloud:moltbot-share /data/workspace/shared
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Token expired
|
||||||
|
|
||||||
|
Re-authorize on your local machine and update the config:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
moltbot workspace authorize
|
||||||
|
# Or manually:
|
||||||
|
rclone authorize "dropbox"
|
||||||
|
# Copy new token to Gateway's rclone.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
### Conflicts
|
||||||
|
|
||||||
|
Files modified on both sides get `.conflict` suffix. Check and resolve manually:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
find /data/workspace/shared -name "*.conflict"
|
||||||
|
```
|
||||||
|
|
||||||
|
### First sync fails
|
||||||
|
|
||||||
|
Use `--resync` flag to establish baseline:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
moltbot workspace sync --resync
|
||||||
|
```
|
||||||
|
|
||||||
|
### Permission errors
|
||||||
|
|
||||||
|
Ensure the workspace directory is writable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
chmod -R 755 /data/workspace/shared
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security notes
|
||||||
|
|
||||||
|
- **Token storage**: rclone tokens are stored in `rclone.conf`. Keep this file secure.
|
||||||
|
- **Sensitive files**: Don't sync secrets, API keys, or credentials.
|
||||||
|
- **Encryption**: Consider using rclone's [crypt](https://rclone.org/crypt/) for sensitive data.
|
||||||
|
|
||||||
|
### Dropbox: App folder access (recommended)
|
||||||
|
|
||||||
|
By default, `rclone authorize "dropbox"` requests **full Dropbox access**. For better security, create an app-scoped token that only accesses a single folder:
|
||||||
|
|
||||||
|
**1. Create a Dropbox App**
|
||||||
|
|
||||||
|
1. Go to [Dropbox App Console](https://www.dropbox.com/developers/apps)
|
||||||
|
2. Click **Create app**
|
||||||
|
3. Choose:
|
||||||
|
- **Scoped access** (not "Dropbox Business API")
|
||||||
|
- **App folder** — only access to `Apps/<your-app-name>/`
|
||||||
|
4. Name it (e.g., `moltbot-sync`)
|
||||||
|
5. Click **Create app**
|
||||||
|
|
||||||
|
**2. Configure permissions**
|
||||||
|
|
||||||
|
In your app's **Permissions** tab, enable:
|
||||||
|
- `files.metadata.read`
|
||||||
|
- `files.metadata.write`
|
||||||
|
- `files.content.read`
|
||||||
|
- `files.content.write`
|
||||||
|
|
||||||
|
Click **Submit** to save.
|
||||||
|
|
||||||
|
**3. Generate token**
|
||||||
|
|
||||||
|
In the **Settings** tab:
|
||||||
|
- Note your **App key** and **App secret**
|
||||||
|
- Under **OAuth 2**, click **Generate** to create an access token
|
||||||
|
|
||||||
|
**4. Configure in moltbot.json**
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "", // empty = app folder root
|
||||||
|
dropbox: {
|
||||||
|
appFolder: true,
|
||||||
|
appKey: "your-app-key",
|
||||||
|
appSecret: "your-app-secret"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Then authorize:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
moltbot workspace authorize
|
||||||
|
```
|
||||||
|
|
||||||
|
**Benefits of app folder access:**
|
||||||
|
- 🔒 Token only accesses one folder, not your entire Dropbox
|
||||||
|
- 🛡️ If token is compromised, blast radius is limited
|
||||||
|
- 📁 Clean separation — sync folder lives under `Apps/`
|
||||||
|
|
||||||
|
### Google Drive: Limited folder access
|
||||||
|
|
||||||
|
Similar scoping is possible with Google Drive using a service account + shared folder, but setup is more involved. See [rclone Google Drive docs](https://rclone.org/drive/).
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [Agent workspace](/concepts/agent-workspace) — workspace layout and backup
|
||||||
|
- [Fly.io deployment](/platforms/fly) — Docker-based cloud deployment
|
||||||
|
- [Hetzner deployment](/platforms/hetzner) — VPS deployment
|
||||||
|
- [rclone docs](https://rclone.org/docs/) — full rclone documentation
|
||||||
123
scripts/workspace-sync.sh
Executable file
123
scripts/workspace-sync.sh
Executable file
@ -0,0 +1,123 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Workspace cloud sync using rclone bisync
|
||||||
|
# Docs: https://docs.molt.bot/gateway/workspace-sync
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Configuration (override via environment)
|
||||||
|
RCLONE_CONFIG="${RCLONE_CONFIG:-$HOME/.config/rclone/rclone.conf}"
|
||||||
|
RCLONE_REMOTE="${RCLONE_REMOTE:-cloud}"
|
||||||
|
REMOTE_PATH="${REMOTE_PATH:-moltbot-share}"
|
||||||
|
LOCAL_PATH="${LOCAL_PATH:-${MOLTBOT_STATE_DIR:-${CLAWDBOT_STATE_DIR:-$HOME/.clawdbot}}/workspace/shared}"
|
||||||
|
|
||||||
|
# Flags
|
||||||
|
RESYNC="${RESYNC:-false}"
|
||||||
|
VERBOSE="${VERBOSE:-false}"
|
||||||
|
DRY_RUN="${DRY_RUN:-false}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat << 'EOF'
|
||||||
|
Usage: workspace-sync.sh [OPTIONS]
|
||||||
|
|
||||||
|
Bidirectional sync between cloud storage and workspace.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--resync Force resync (required on first run)
|
||||||
|
--dry-run Show what would be synced without doing it
|
||||||
|
--verbose Show detailed output
|
||||||
|
--help Show this help
|
||||||
|
|
||||||
|
Environment:
|
||||||
|
RCLONE_CONFIG Path to rclone.conf (default: ~/.config/rclone/rclone.conf)
|
||||||
|
RCLONE_REMOTE rclone remote name (default: cloud)
|
||||||
|
REMOTE_PATH Folder in cloud storage (default: moltbot-share)
|
||||||
|
LOCAL_PATH Local folder to sync (default: <state-dir>/workspace/shared)
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
# First sync (establishes baseline)
|
||||||
|
./scripts/workspace-sync.sh --resync
|
||||||
|
|
||||||
|
# Regular sync
|
||||||
|
./scripts/workspace-sync.sh
|
||||||
|
|
||||||
|
# Preview changes
|
||||||
|
./scripts/workspace-sync.sh --dry-run --verbose
|
||||||
|
|
||||||
|
Setup: https://docs.molt.bot/gateway/workspace-sync
|
||||||
|
EOF
|
||||||
|
exit 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Parse arguments
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case $1 in
|
||||||
|
--resync)
|
||||||
|
RESYNC=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--dry-run)
|
||||||
|
DRY_RUN=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--verbose|-v)
|
||||||
|
VERBOSE=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--help|-h)
|
||||||
|
usage
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "Unknown option: $1"
|
||||||
|
usage
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Check rclone is installed
|
||||||
|
if ! command -v rclone &> /dev/null; then
|
||||||
|
echo "Error: rclone not installed"
|
||||||
|
echo "Install: curl -s https://rclone.org/install.sh | bash"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check config exists
|
||||||
|
if [[ ! -f "$RCLONE_CONFIG" ]]; then
|
||||||
|
echo "Error: rclone config not found at $RCLONE_CONFIG"
|
||||||
|
echo "Setup: https://docs.molt.bot/gateway/workspace-sync"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Ensure local directory exists
|
||||||
|
mkdir -p "$LOCAL_PATH"
|
||||||
|
|
||||||
|
# Build rclone command
|
||||||
|
RCLONE_ARGS=(
|
||||||
|
bisync
|
||||||
|
"${RCLONE_REMOTE}:${REMOTE_PATH}"
|
||||||
|
"$LOCAL_PATH"
|
||||||
|
--config "$RCLONE_CONFIG"
|
||||||
|
--conflict-resolve newer
|
||||||
|
--conflict-suffix .conflict
|
||||||
|
--exclude ".git/**"
|
||||||
|
--exclude "node_modules/**"
|
||||||
|
--exclude "*.log"
|
||||||
|
--exclude ".DS_Store"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ "$RESYNC" == "true" ]]; then
|
||||||
|
RCLONE_ARGS+=(--resync)
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$DRY_RUN" == "true" ]]; then
|
||||||
|
RCLONE_ARGS+=(--dry-run)
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$VERBOSE" == "true" ]]; then
|
||||||
|
RCLONE_ARGS+=(--verbose)
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Run sync
|
||||||
|
echo "Syncing: ${RCLONE_REMOTE}:${REMOTE_PATH} <-> $LOCAL_PATH"
|
||||||
|
rclone "${RCLONE_ARGS[@]}"
|
||||||
|
|
||||||
|
echo "Sync complete: $(date)"
|
||||||
@ -9,6 +9,7 @@ import { getFlagValue, getPositiveIntFlagValue, getVerboseFlag, hasFlag } from "
|
|||||||
import { registerBrowserCli } from "../browser-cli.js";
|
import { registerBrowserCli } from "../browser-cli.js";
|
||||||
import { registerConfigCli } from "../config-cli.js";
|
import { registerConfigCli } from "../config-cli.js";
|
||||||
import { registerMemoryCli, runMemoryStatus } from "../memory-cli.js";
|
import { registerMemoryCli, runMemoryStatus } from "../memory-cli.js";
|
||||||
|
import { registerWorkspaceCli } from "../workspace-cli.js";
|
||||||
import { registerAgentCommands } from "./register.agent.js";
|
import { registerAgentCommands } from "./register.agent.js";
|
||||||
import { registerConfigureCommand } from "./register.configure.js";
|
import { registerConfigureCommand } from "./register.configure.js";
|
||||||
import { registerMaintenanceCommands } from "./register.maintenance.js";
|
import { registerMaintenanceCommands } from "./register.maintenance.js";
|
||||||
@ -152,6 +153,10 @@ export const commandRegistry: CommandRegistration[] = [
|
|||||||
id: "browser",
|
id: "browser",
|
||||||
register: ({ program }) => registerBrowserCli(program),
|
register: ({ program }) => registerBrowserCli(program),
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
id: "workspace",
|
||||||
|
register: ({ program }) => registerWorkspaceCli(program),
|
||||||
|
},
|
||||||
];
|
];
|
||||||
|
|
||||||
export function registerProgramCommands(
|
export function registerProgramCommands(
|
||||||
|
|||||||
701
src/cli/workspace-cli.ts
Normal file
701
src/cli/workspace-cli.ts
Normal file
@ -0,0 +1,701 @@
|
|||||||
|
import { homedir } from "node:os";
|
||||||
|
import type { Command } from "commander";
|
||||||
|
import * as clack from "@clack/prompts";
|
||||||
|
|
||||||
|
import { loadConfig, writeConfigFile } from "../config/config.js";
|
||||||
|
import { resolveDefaultAgentId, resolveAgentWorkspaceDir } from "../agents/agent-scope.js";
|
||||||
|
import { resolveStateDir } from "../config/paths.js";
|
||||||
|
import { danger, setVerbose } from "../globals.js";
|
||||||
|
import { defaultRuntime } from "../runtime.js";
|
||||||
|
import { formatCliCommand } from "./command-format.js";
|
||||||
|
import { colorize, isRich, theme } from "../terminal/theme.js";
|
||||||
|
import { formatDocsLink } from "../terminal/links.js";
|
||||||
|
import { shortenHomePath } from "../utils.js";
|
||||||
|
import {
|
||||||
|
isRcloneInstalled,
|
||||||
|
ensureRcloneInstalled,
|
||||||
|
isRcloneConfigured,
|
||||||
|
resolveSyncConfig,
|
||||||
|
runBisync,
|
||||||
|
runSync,
|
||||||
|
checkRemote,
|
||||||
|
listRemote,
|
||||||
|
authorizeRclone,
|
||||||
|
writeRcloneConfig,
|
||||||
|
generateRcloneConfig,
|
||||||
|
type RcloneSyncResult,
|
||||||
|
} from "../infra/rclone.js";
|
||||||
|
import type { WorkspaceSyncProvider } from "../config/types.workspace.js";
|
||||||
|
import type { MoltbotConfig } from "../config/types.clawdbot.js";
|
||||||
|
|
||||||
|
type WorkspaceSyncOptions = {
|
||||||
|
agent?: string;
|
||||||
|
resync?: boolean;
|
||||||
|
dryRun?: boolean;
|
||||||
|
verbose?: boolean;
|
||||||
|
direction?: "pull" | "push";
|
||||||
|
};
|
||||||
|
|
||||||
|
type WorkspaceStatusOptions = {
|
||||||
|
agent?: string;
|
||||||
|
verbose?: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
type WorkspaceAuthorizeOptions = {
|
||||||
|
provider?: string;
|
||||||
|
appKey?: string;
|
||||||
|
appSecret?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
function resolveAgent(cfg: ReturnType<typeof loadConfig>, agent?: string) {
|
||||||
|
const trimmed = agent?.trim();
|
||||||
|
if (trimmed) return trimmed;
|
||||||
|
return resolveDefaultAgentId(cfg);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function registerWorkspaceCli(program: Command): void {
|
||||||
|
const workspace = program
|
||||||
|
.command("workspace")
|
||||||
|
.description("Workspace management and cloud sync")
|
||||||
|
.addHelpText(
|
||||||
|
"after",
|
||||||
|
() =>
|
||||||
|
`\n${theme.muted("Docs:")} ${formatDocsLink(
|
||||||
|
"/gateway/workspace-sync",
|
||||||
|
"docs.molt.bot/gateway/workspace-sync",
|
||||||
|
)}\n`,
|
||||||
|
)
|
||||||
|
.action(() => {
|
||||||
|
workspace.outputHelp();
|
||||||
|
defaultRuntime.error(
|
||||||
|
danger(`Missing subcommand. Try: "${formatCliCommand("moltbot workspace setup")}"`),
|
||||||
|
);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
// moltbot workspace sync
|
||||||
|
workspace
|
||||||
|
.command("sync")
|
||||||
|
.description("Sync workspace with cloud storage")
|
||||||
|
.option("--agent <id>", "Agent ID (default: main)")
|
||||||
|
.option("--resync", "Force resync (required for first sync)")
|
||||||
|
.option("--dry-run", "Preview changes without syncing")
|
||||||
|
.option("--direction <dir>", "One-way sync: pull or push")
|
||||||
|
.option("-v, --verbose", "Verbose output")
|
||||||
|
.action(async (opts: WorkspaceSyncOptions) => {
|
||||||
|
try {
|
||||||
|
if (opts.verbose) setVerbose(true);
|
||||||
|
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const agentId = resolveAgent(cfg, opts.agent);
|
||||||
|
const workspaceDir = resolveAgentWorkspaceDir(cfg, agentId);
|
||||||
|
const stateDir = resolveStateDir(process.env, homedir);
|
||||||
|
const syncConfig = cfg.workspace?.sync;
|
||||||
|
|
||||||
|
const rich = isRich();
|
||||||
|
|
||||||
|
if (!syncConfig?.provider || syncConfig.provider === "off") {
|
||||||
|
defaultRuntime.error(colorize(rich, theme.error, "Workspace sync not configured."));
|
||||||
|
defaultRuntime.error("");
|
||||||
|
defaultRuntime.error(`Run: ${formatCliCommand("moltbot workspace setup")}`);
|
||||||
|
defaultRuntime.error(`Docs: ${formatDocsLink("/gateway/workspace-sync")}`);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check rclone
|
||||||
|
const installed = await isRcloneInstalled();
|
||||||
|
if (!installed) {
|
||||||
|
defaultRuntime.error(colorize(rich, theme.error, "rclone not installed."));
|
||||||
|
defaultRuntime.error("");
|
||||||
|
defaultRuntime.error(`Run: ${formatCliCommand("moltbot workspace setup")}`);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(syncConfig, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
// Check config
|
||||||
|
if (!isRcloneConfigured(resolved.configPath, resolved.remoteName)) {
|
||||||
|
console.error(
|
||||||
|
colorize(
|
||||||
|
rich,
|
||||||
|
theme.error,
|
||||||
|
`rclone not configured for remote "${resolved.remoteName}".`,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
console.error("");
|
||||||
|
console.error("Run: moltbot workspace authorize");
|
||||||
|
console.error(`Or manually configure: ${shortenHomePath(resolved.configPath)}`);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
colorize(rich, theme.info, `Syncing ${resolved.remoteName}:${resolved.remotePath}`),
|
||||||
|
);
|
||||||
|
console.log(colorize(rich, theme.muted, `Local: ${shortenHomePath(resolved.localPath)}`));
|
||||||
|
|
||||||
|
let result: RcloneSyncResult;
|
||||||
|
|
||||||
|
if (opts.direction) {
|
||||||
|
// One-way sync
|
||||||
|
result = await runSync({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
localPath: resolved.localPath,
|
||||||
|
direction: opts.direction,
|
||||||
|
exclude: resolved.exclude,
|
||||||
|
dryRun: opts.dryRun,
|
||||||
|
verbose: opts.verbose,
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
// Bidirectional sync
|
||||||
|
result = await runBisync({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
localPath: resolved.localPath,
|
||||||
|
conflictResolve: resolved.conflictResolve,
|
||||||
|
exclude: resolved.exclude,
|
||||||
|
resync: opts.resync,
|
||||||
|
dryRun: opts.dryRun,
|
||||||
|
verbose: opts.verbose,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
if (result.ok) {
|
||||||
|
defaultRuntime.log(colorize(rich, theme.success, "✓ Sync completed"));
|
||||||
|
if (result.filesTransferred) {
|
||||||
|
defaultRuntime.log(
|
||||||
|
colorize(rich, theme.muted, `Files transferred: ${result.filesTransferred}`),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
defaultRuntime.error(colorize(rich, theme.error, `✗ Sync failed: ${result.error}`));
|
||||||
|
if (result.error?.includes("--resync")) {
|
||||||
|
defaultRuntime.error("");
|
||||||
|
defaultRuntime.error(
|
||||||
|
`First sync requires --resync: ${formatCliCommand("moltbot workspace sync --resync")}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
defaultRuntime.error(
|
||||||
|
`${theme.error("Error:")} ${err instanceof Error ? err.message : String(err)}`,
|
||||||
|
);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// moltbot workspace status
|
||||||
|
workspace
|
||||||
|
.command("status")
|
||||||
|
.description("Show workspace sync status")
|
||||||
|
.option("--agent <id>", "Agent ID (default: main)")
|
||||||
|
.option("-v, --verbose", "Verbose output")
|
||||||
|
.action(async (opts: WorkspaceStatusOptions) => {
|
||||||
|
if (opts.verbose) setVerbose(true);
|
||||||
|
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const agentId = resolveAgent(cfg, opts.agent);
|
||||||
|
const workspaceDir = resolveAgentWorkspaceDir(cfg, agentId);
|
||||||
|
const stateDir = resolveStateDir(process.env, homedir);
|
||||||
|
const syncConfig = cfg.workspace?.sync;
|
||||||
|
|
||||||
|
const rich = isRich();
|
||||||
|
|
||||||
|
console.log(colorize(rich, theme.heading, "Workspace Sync Status"));
|
||||||
|
console.log("");
|
||||||
|
|
||||||
|
// Check config
|
||||||
|
if (!syncConfig?.provider || syncConfig.provider === "off") {
|
||||||
|
console.log(colorize(rich, theme.muted, "Provider: not configured"));
|
||||||
|
console.log("");
|
||||||
|
console.log(`Configure in ~/.clawdbot/moltbot.json`);
|
||||||
|
console.log(`Docs: ${formatDocsLink("/gateway/workspace-sync")}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(syncConfig, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
console.log(`Provider: ${colorize(rich, theme.info, syncConfig.provider)}`);
|
||||||
|
console.log(`Remote: ${resolved.remoteName}:${resolved.remotePath}`);
|
||||||
|
console.log(`Local: ${shortenHomePath(resolved.localPath)}`);
|
||||||
|
console.log(`Config: ${shortenHomePath(resolved.configPath)}`);
|
||||||
|
console.log("");
|
||||||
|
|
||||||
|
// Check rclone
|
||||||
|
const installed = await isRcloneInstalled();
|
||||||
|
if (!installed) {
|
||||||
|
console.log(colorize(rich, theme.error, "✗ rclone not installed"));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
console.log(colorize(rich, theme.success, "✓ rclone installed"));
|
||||||
|
|
||||||
|
// Check config
|
||||||
|
const configured = isRcloneConfigured(resolved.configPath, resolved.remoteName);
|
||||||
|
if (!configured) {
|
||||||
|
console.log(colorize(rich, theme.error, "✗ rclone not configured"));
|
||||||
|
console.log("");
|
||||||
|
console.log("Run: moltbot workspace authorize");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
console.log(colorize(rich, theme.success, "✓ rclone configured"));
|
||||||
|
|
||||||
|
// Check connection
|
||||||
|
const check = await checkRemote({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
});
|
||||||
|
if (!check.ok) {
|
||||||
|
console.log(colorize(rich, theme.error, `✗ Connection failed: ${check.error}`));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
console.log(colorize(rich, theme.success, "✓ Remote connected"));
|
||||||
|
|
||||||
|
// List remote files
|
||||||
|
const list = await listRemote({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
});
|
||||||
|
if (list.ok) {
|
||||||
|
console.log("");
|
||||||
|
console.log(`Remote files: ${list.files.length}`);
|
||||||
|
if (opts.verbose && list.files.length > 0) {
|
||||||
|
for (const file of list.files.slice(0, 10)) {
|
||||||
|
console.log(colorize(rich, theme.muted, ` ${file}`));
|
||||||
|
}
|
||||||
|
if (list.files.length > 10) {
|
||||||
|
console.log(colorize(rich, theme.muted, ` ... and ${list.files.length - 10} more`));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Show auto-sync settings
|
||||||
|
console.log("");
|
||||||
|
console.log("Auto-sync:");
|
||||||
|
console.log(` On session start: ${resolved.onSessionStart ? "yes" : "no"}`);
|
||||||
|
console.log(` On session end: ${resolved.onSessionEnd ? "yes" : "no"}`);
|
||||||
|
if (resolved.interval > 0) {
|
||||||
|
console.log(` Background interval: ${resolved.interval}s (pure rclone, zero LLM cost)`);
|
||||||
|
} else {
|
||||||
|
console.log(` Background interval: disabled`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// moltbot workspace setup - interactive wizard
|
||||||
|
workspace
|
||||||
|
.command("setup")
|
||||||
|
.description("Interactive setup wizard for cloud sync")
|
||||||
|
.action(async () => {
|
||||||
|
const rich = isRich();
|
||||||
|
|
||||||
|
clack.intro(colorize(rich, theme.heading, "Workspace Cloud Sync Setup"));
|
||||||
|
|
||||||
|
// Step 1: Check/install rclone
|
||||||
|
const rcloneInstalled = await isRcloneInstalled();
|
||||||
|
if (!rcloneInstalled) {
|
||||||
|
// Offer to install
|
||||||
|
const installed = await ensureRcloneInstalled(async (message, defaultValue) => {
|
||||||
|
const result = await clack.confirm({ message, initialValue: defaultValue });
|
||||||
|
return !clack.isCancel(result) && result;
|
||||||
|
});
|
||||||
|
if (!installed) {
|
||||||
|
clack.note(
|
||||||
|
"Install rclone manually:\n\n" +
|
||||||
|
" macOS: brew install rclone\n" +
|
||||||
|
" Linux: curl -s https://rclone.org/install.sh | sudo bash\n" +
|
||||||
|
" Docker: Add to Dockerfile: RUN curl -s https://rclone.org/install.sh | bash",
|
||||||
|
"Installation required",
|
||||||
|
);
|
||||||
|
clack.outro("Install rclone and run this command again.");
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
clack.log.success("rclone is installed");
|
||||||
|
|
||||||
|
// Step 2: Select provider
|
||||||
|
const provider = (await clack.select({
|
||||||
|
message: "Select cloud provider",
|
||||||
|
options: [
|
||||||
|
{ value: "dropbox", label: "Dropbox", hint: "Recommended - easy setup" },
|
||||||
|
{
|
||||||
|
value: "gdrive",
|
||||||
|
label: "Google Drive",
|
||||||
|
hint: "Requires service account for scoped access",
|
||||||
|
},
|
||||||
|
{ value: "onedrive", label: "OneDrive", hint: "Microsoft 365" },
|
||||||
|
{ value: "s3", label: "S3 / R2 / Minio", hint: "Access key authentication" },
|
||||||
|
],
|
||||||
|
})) as WorkspaceSyncProvider;
|
||||||
|
|
||||||
|
if (clack.isCancel(provider)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 3: Remote folder name
|
||||||
|
const remotePath = (await clack.text({
|
||||||
|
message: "Remote folder name",
|
||||||
|
placeholder: "moltbot-share",
|
||||||
|
initialValue: "moltbot-share",
|
||||||
|
validate: (value) => {
|
||||||
|
if (!value.trim()) return "Folder name is required";
|
||||||
|
if (value.includes("/")) return "Use a simple folder name, not a path";
|
||||||
|
return undefined;
|
||||||
|
},
|
||||||
|
})) as string;
|
||||||
|
|
||||||
|
if (clack.isCancel(remotePath)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 4: Dropbox-specific: app folder option
|
||||||
|
let useAppFolder = false;
|
||||||
|
let appKey: string | undefined;
|
||||||
|
let appSecret: string | undefined;
|
||||||
|
|
||||||
|
if (provider === "dropbox") {
|
||||||
|
const accessType = (await clack.select({
|
||||||
|
message: "Dropbox access type",
|
||||||
|
options: [
|
||||||
|
{
|
||||||
|
value: "full",
|
||||||
|
label: "Full Dropbox",
|
||||||
|
hint: "Access entire Dropbox (simpler setup)",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
value: "app",
|
||||||
|
label: "App Folder only",
|
||||||
|
hint: "Restricted to Apps/<app-name>/ (more secure)",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
})) as "full" | "app";
|
||||||
|
|
||||||
|
if (clack.isCancel(accessType)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
useAppFolder = accessType === "app";
|
||||||
|
|
||||||
|
if (useAppFolder) {
|
||||||
|
clack.note(
|
||||||
|
"1. Go to https://www.dropbox.com/developers/apps\n" +
|
||||||
|
"2. Click 'Create app'\n" +
|
||||||
|
"3. Choose 'Scoped access' → 'App folder'\n" +
|
||||||
|
"4. Name it (e.g., 'moltbot-sync')\n" +
|
||||||
|
"5. In Permissions tab, enable:\n" +
|
||||||
|
" - files.metadata.read/write\n" +
|
||||||
|
" - files.content.read/write\n" +
|
||||||
|
"6. Copy the App key and App secret from Settings",
|
||||||
|
"Create Dropbox App",
|
||||||
|
);
|
||||||
|
|
||||||
|
appKey = (await clack.text({
|
||||||
|
message: "Dropbox App key",
|
||||||
|
placeholder: "your-app-key",
|
||||||
|
})) as string;
|
||||||
|
|
||||||
|
if (clack.isCancel(appKey)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
appSecret = (await clack.text({
|
||||||
|
message: "Dropbox App secret",
|
||||||
|
placeholder: "your-app-secret",
|
||||||
|
})) as string;
|
||||||
|
|
||||||
|
if (clack.isCancel(appSecret)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 5: Background sync interval
|
||||||
|
const intervalChoice = (await clack.select({
|
||||||
|
message: "Background sync interval",
|
||||||
|
options: [
|
||||||
|
{ value: "0", label: "Manual only", hint: "Run 'moltbot workspace sync' when needed" },
|
||||||
|
{ value: "300", label: "Every 5 minutes", hint: "Recommended" },
|
||||||
|
{ value: "600", label: "Every 10 minutes" },
|
||||||
|
{ value: "1800", label: "Every 30 minutes" },
|
||||||
|
{ value: "3600", label: "Every hour" },
|
||||||
|
],
|
||||||
|
})) as string;
|
||||||
|
|
||||||
|
if (clack.isCancel(intervalChoice)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const interval = parseInt(intervalChoice, 10);
|
||||||
|
|
||||||
|
// Step 6: Session hooks
|
||||||
|
const onSessionStart = (await clack.confirm({
|
||||||
|
message: "Sync when session starts?",
|
||||||
|
initialValue: true,
|
||||||
|
})) as boolean;
|
||||||
|
|
||||||
|
if (clack.isCancel(onSessionStart)) {
|
||||||
|
clack.cancel("Setup cancelled.");
|
||||||
|
defaultRuntime.exit(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 7: Save config
|
||||||
|
const spinner = clack.spinner();
|
||||||
|
spinner.start("Saving configuration...");
|
||||||
|
|
||||||
|
try {
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const newConfig: MoltbotConfig = {
|
||||||
|
...cfg,
|
||||||
|
workspace: {
|
||||||
|
...cfg.workspace,
|
||||||
|
sync: {
|
||||||
|
provider,
|
||||||
|
remotePath: remotePath.trim(),
|
||||||
|
localPath: "shared",
|
||||||
|
interval,
|
||||||
|
onSessionStart,
|
||||||
|
onSessionEnd: false,
|
||||||
|
...(useAppFolder && appKey && appSecret
|
||||||
|
? {
|
||||||
|
dropbox: {
|
||||||
|
appFolder: true,
|
||||||
|
appKey: appKey.trim(),
|
||||||
|
appSecret: appSecret.trim(),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
: {}),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await writeConfigFile(newConfig);
|
||||||
|
spinner.stop("Configuration saved");
|
||||||
|
} catch (err) {
|
||||||
|
spinner.stop("Failed to save configuration");
|
||||||
|
clack.log.error(`Error: ${err instanceof Error ? err.message : String(err)}`);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 8: OAuth authorization
|
||||||
|
clack.log.info("Starting OAuth authorization...");
|
||||||
|
clack.note(
|
||||||
|
"A browser window will open.\n" + "Log in and authorize access, then return here.",
|
||||||
|
"Authorization",
|
||||||
|
);
|
||||||
|
|
||||||
|
const authResult = await authorizeRclone(provider, appKey, appSecret);
|
||||||
|
|
||||||
|
if (!authResult.ok) {
|
||||||
|
clack.log.error(`Authorization failed: ${authResult.error}`);
|
||||||
|
clack.outro("Fix the error and run 'moltbot workspace setup' again.");
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
clack.log.success("Authorization successful");
|
||||||
|
|
||||||
|
// Step 9: Save rclone config
|
||||||
|
const stateDir = resolveStateDir(process.env, homedir);
|
||||||
|
const configPath = `${stateDir}/.config/rclone/rclone.conf`;
|
||||||
|
const remoteName = "cloud";
|
||||||
|
|
||||||
|
const configContent = generateRcloneConfig(provider, remoteName, authResult.token, {
|
||||||
|
dropbox: useAppFolder ? { appKey, appSecret } : undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
writeRcloneConfig(configPath, configContent);
|
||||||
|
clack.log.success(`rclone config saved to ${shortenHomePath(configPath)}`);
|
||||||
|
|
||||||
|
// Step 10: Create local folder info
|
||||||
|
clack.note(
|
||||||
|
`Create this folder on your local machine:\n\n` +
|
||||||
|
` ~/Dropbox/${remotePath}/\n\n` +
|
||||||
|
`Or wherever your cloud app syncs files.`,
|
||||||
|
"Local folder",
|
||||||
|
);
|
||||||
|
|
||||||
|
// Step 11: First sync
|
||||||
|
const runFirstSync = (await clack.confirm({
|
||||||
|
message: "Run first sync now? (--resync)",
|
||||||
|
initialValue: true,
|
||||||
|
})) as boolean;
|
||||||
|
|
||||||
|
if (runFirstSync && !clack.isCancel(runFirstSync)) {
|
||||||
|
const syncSpinner = clack.spinner();
|
||||||
|
syncSpinner.start("Running first sync...");
|
||||||
|
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const agentId = resolveDefaultAgentId(cfg);
|
||||||
|
const workspaceDir = resolveAgentWorkspaceDir(cfg, agentId);
|
||||||
|
const resolved = resolveSyncConfig(cfg.workspace?.sync, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
const syncResult = await runBisync({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
localPath: resolved.localPath,
|
||||||
|
conflictResolve: resolved.conflictResolve,
|
||||||
|
exclude: resolved.exclude,
|
||||||
|
resync: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (syncResult.ok) {
|
||||||
|
syncSpinner.stop("First sync completed");
|
||||||
|
} else {
|
||||||
|
syncSpinner.stop("First sync failed");
|
||||||
|
clack.log.warn(`Error: ${syncResult.error}`);
|
||||||
|
clack.log.info("You can retry with: moltbot workspace sync --resync");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Done!
|
||||||
|
clack.outro(colorize(rich, theme.success, "✓ Workspace sync configured!"));
|
||||||
|
|
||||||
|
console.log("");
|
||||||
|
console.log("Commands:");
|
||||||
|
console.log(" moltbot workspace sync Sync now");
|
||||||
|
console.log(" moltbot workspace status Check status");
|
||||||
|
console.log(" moltbot workspace list List remote files");
|
||||||
|
console.log("");
|
||||||
|
console.log(`Docs: ${formatDocsLink("/gateway/workspace-sync")}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// moltbot workspace authorize
|
||||||
|
workspace
|
||||||
|
.command("authorize")
|
||||||
|
.description("Authorize rclone with cloud provider (use 'setup' for guided flow)")
|
||||||
|
.option("--provider <name>", "Provider: dropbox, gdrive, onedrive, s3")
|
||||||
|
.option("--app-key <key>", "Dropbox app key (for app folder access)")
|
||||||
|
.option("--app-secret <secret>", "Dropbox app secret (for app folder access)")
|
||||||
|
.action(async (opts: WorkspaceAuthorizeOptions) => {
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const syncConfig = cfg.workspace?.sync;
|
||||||
|
const rich = isRich();
|
||||||
|
|
||||||
|
// Determine provider
|
||||||
|
let provider: WorkspaceSyncProvider =
|
||||||
|
(opts.provider as WorkspaceSyncProvider) || syncConfig?.provider || "dropbox";
|
||||||
|
|
||||||
|
if (provider === "off" || provider === "custom") {
|
||||||
|
console.error(colorize(rich, theme.error, "Please specify a provider: --provider dropbox"));
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check rclone
|
||||||
|
const installed = await isRcloneInstalled();
|
||||||
|
if (!installed) {
|
||||||
|
console.error(colorize(rich, theme.error, "rclone not installed."));
|
||||||
|
console.error("");
|
||||||
|
console.error("Install: curl -s https://rclone.org/install.sh | bash");
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(colorize(rich, theme.info, `Authorizing with ${provider}...`));
|
||||||
|
console.log("");
|
||||||
|
console.log("A browser window will open for authentication.");
|
||||||
|
console.log("Complete the OAuth flow and return here.");
|
||||||
|
console.log("");
|
||||||
|
|
||||||
|
const result = await authorizeRclone(
|
||||||
|
provider,
|
||||||
|
opts.appKey || syncConfig?.dropbox?.appKey,
|
||||||
|
opts.appSecret || syncConfig?.dropbox?.appSecret,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(colorize(rich, theme.error, `Authorization failed: ${result.error}`));
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(colorize(rich, theme.success, "✓ Authorization successful"));
|
||||||
|
console.log("");
|
||||||
|
|
||||||
|
// Generate and save config
|
||||||
|
const stateDir = resolveStateDir(process.env, homedir);
|
||||||
|
const remoteName = syncConfig?.remoteName || "cloud";
|
||||||
|
const configPath = syncConfig?.configPath || `${stateDir}/.config/rclone/rclone.conf`;
|
||||||
|
|
||||||
|
const configContent = generateRcloneConfig(provider, remoteName, result.token, {
|
||||||
|
dropbox: syncConfig?.dropbox,
|
||||||
|
s3: syncConfig?.s3,
|
||||||
|
});
|
||||||
|
|
||||||
|
writeRcloneConfig(configPath, configContent);
|
||||||
|
|
||||||
|
console.log(`Config saved to: ${shortenHomePath(configPath)}`);
|
||||||
|
console.log("");
|
||||||
|
console.log("Next steps:");
|
||||||
|
console.log(" 1. Create the remote folder (e.g., ~/Dropbox/moltbot-share/)");
|
||||||
|
console.log(" 2. Run first sync: moltbot workspace sync --resync");
|
||||||
|
});
|
||||||
|
|
||||||
|
// moltbot workspace list
|
||||||
|
workspace
|
||||||
|
.command("list")
|
||||||
|
.description("List files in remote storage")
|
||||||
|
.option("--agent <id>", "Agent ID (default: main)")
|
||||||
|
.action(async (opts: { agent?: string }) => {
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const agentId = resolveAgent(cfg, opts.agent);
|
||||||
|
const workspaceDir = resolveAgentWorkspaceDir(cfg, agentId);
|
||||||
|
const stateDir = resolveStateDir(process.env, homedir);
|
||||||
|
const syncConfig = cfg.workspace?.sync;
|
||||||
|
|
||||||
|
const rich = isRich();
|
||||||
|
|
||||||
|
if (!syncConfig?.provider || syncConfig.provider === "off") {
|
||||||
|
console.error(colorize(rich, theme.error, "Workspace sync not configured."));
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(syncConfig, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
if (!isRcloneConfigured(resolved.configPath, resolved.remoteName)) {
|
||||||
|
console.error(colorize(rich, theme.error, "rclone not configured."));
|
||||||
|
console.error("Run: moltbot workspace authorize");
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await listRemote({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(colorize(rich, theme.error, `Failed to list: ${result.error}`));
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (result.files.length === 0) {
|
||||||
|
console.log(colorize(rich, theme.muted, "No files in remote."));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`${resolved.remoteName}:${resolved.remotePath}/`);
|
||||||
|
for (const file of result.files) {
|
||||||
|
console.log(` ${file}`);
|
||||||
|
}
|
||||||
|
console.log("");
|
||||||
|
console.log(colorize(rich, theme.muted, `${result.files.length} files`));
|
||||||
|
});
|
||||||
|
}
|
||||||
@ -23,6 +23,7 @@ import type { NodeHostConfig } from "./types.node-host.js";
|
|||||||
import type { PluginsConfig } from "./types.plugins.js";
|
import type { PluginsConfig } from "./types.plugins.js";
|
||||||
import type { SkillsConfig } from "./types.skills.js";
|
import type { SkillsConfig } from "./types.skills.js";
|
||||||
import type { ToolsConfig } from "./types.tools.js";
|
import type { ToolsConfig } from "./types.tools.js";
|
||||||
|
import type { WorkspaceConfig } from "./types.workspace.js";
|
||||||
|
|
||||||
export type MoltbotConfig = {
|
export type MoltbotConfig = {
|
||||||
meta?: {
|
meta?: {
|
||||||
@ -95,6 +96,7 @@ export type MoltbotConfig = {
|
|||||||
canvasHost?: CanvasHostConfig;
|
canvasHost?: CanvasHostConfig;
|
||||||
talk?: TalkConfig;
|
talk?: TalkConfig;
|
||||||
gateway?: GatewayConfig;
|
gateway?: GatewayConfig;
|
||||||
|
workspace?: WorkspaceConfig;
|
||||||
};
|
};
|
||||||
|
|
||||||
export type ConfigValidationIssue = {
|
export type ConfigValidationIssue = {
|
||||||
|
|||||||
119
src/config/types.workspace.ts
Normal file
119
src/config/types.workspace.ts
Normal file
@ -0,0 +1,119 @@
|
|||||||
|
/**
|
||||||
|
* Workspace sync provider modes.
|
||||||
|
* - off: no sync
|
||||||
|
* - dropbox: Dropbox via rclone
|
||||||
|
* - gdrive: Google Drive via rclone
|
||||||
|
* - onedrive: OneDrive via rclone
|
||||||
|
* - s3: S3-compatible storage via rclone
|
||||||
|
* - custom: custom rclone remote (user-configured)
|
||||||
|
*/
|
||||||
|
export type WorkspaceSyncProvider = "off" | "dropbox" | "gdrive" | "onedrive" | "s3" | "custom";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Workspace sync configuration.
|
||||||
|
* Enables bidirectional sync between the agent workspace and cloud storage.
|
||||||
|
*/
|
||||||
|
export type WorkspaceSyncConfig = {
|
||||||
|
/**
|
||||||
|
* Sync provider mode.
|
||||||
|
* - off: disabled (default)
|
||||||
|
* - dropbox: Dropbox via rclone
|
||||||
|
* - gdrive: Google Drive via rclone
|
||||||
|
* - onedrive: OneDrive via rclone
|
||||||
|
* - s3: S3-compatible storage via rclone
|
||||||
|
* - custom: custom rclone remote
|
||||||
|
*/
|
||||||
|
provider?: WorkspaceSyncProvider;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Remote path/folder in cloud storage (e.g., "moltbot-share").
|
||||||
|
* For Dropbox App folders, this is relative to the app folder root.
|
||||||
|
*/
|
||||||
|
remotePath?: string;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Local subfolder within workspace to sync (default: "shared").
|
||||||
|
* Files outside this folder are not synced.
|
||||||
|
*/
|
||||||
|
localPath?: string;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sync interval in seconds (0 = manual only, default: 0).
|
||||||
|
* When > 0, the gateway runs rclone bisync in the background at this interval.
|
||||||
|
* This is a pure file operation - it does NOT wake the bot or incur LLM costs.
|
||||||
|
*/
|
||||||
|
interval?: number;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sync on session start (default: false).
|
||||||
|
*/
|
||||||
|
onSessionStart?: boolean;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sync on session end (default: false).
|
||||||
|
*/
|
||||||
|
onSessionEnd?: boolean;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* rclone remote name (default: "cloud").
|
||||||
|
* Used when provider is "custom" or to override the auto-generated name.
|
||||||
|
*/
|
||||||
|
remoteName?: string;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Path to rclone config file.
|
||||||
|
* Default: $CLAWDBOT_STATE_DIR/.config/rclone/rclone.conf
|
||||||
|
*/
|
||||||
|
configPath?: string;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Conflict resolution strategy.
|
||||||
|
* - newer: keep the newer file, rename older with .conflict suffix
|
||||||
|
* - local: local wins, remote gets .conflict suffix
|
||||||
|
* - remote: remote wins, local gets .conflict suffix
|
||||||
|
*/
|
||||||
|
conflictResolve?: "newer" | "local" | "remote";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* File patterns to exclude from sync (glob patterns).
|
||||||
|
*/
|
||||||
|
exclude?: string[];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* S3-specific configuration (when provider is "s3").
|
||||||
|
*/
|
||||||
|
s3?: {
|
||||||
|
/** S3 endpoint URL (for non-AWS S3-compatible services). */
|
||||||
|
endpoint?: string;
|
||||||
|
/** S3 bucket name. */
|
||||||
|
bucket?: string;
|
||||||
|
/** S3 region. */
|
||||||
|
region?: string;
|
||||||
|
/** Access key ID (prefer env var S3_ACCESS_KEY_ID). */
|
||||||
|
accessKeyId?: string;
|
||||||
|
/** Secret access key (prefer env var S3_SECRET_ACCESS_KEY). */
|
||||||
|
secretAccessKey?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Dropbox-specific configuration.
|
||||||
|
*/
|
||||||
|
dropbox?: {
|
||||||
|
/** Use app folder access (more secure, limited to Apps/<app-name>/). */
|
||||||
|
appFolder?: boolean;
|
||||||
|
/** Dropbox app key (for app folder access). */
|
||||||
|
appKey?: string;
|
||||||
|
/** Dropbox app secret (for app folder access). */
|
||||||
|
appSecret?: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Top-level workspace configuration.
|
||||||
|
*/
|
||||||
|
export type WorkspaceConfig = {
|
||||||
|
/**
|
||||||
|
* Cloud sync configuration for bidirectional workspace sync.
|
||||||
|
*/
|
||||||
|
sync?: WorkspaceSyncConfig;
|
||||||
|
};
|
||||||
@ -527,6 +527,55 @@ export const MoltbotSchema = z
|
|||||||
})
|
})
|
||||||
.strict()
|
.strict()
|
||||||
.optional(),
|
.optional(),
|
||||||
|
workspace: z
|
||||||
|
.object({
|
||||||
|
sync: z
|
||||||
|
.object({
|
||||||
|
provider: z
|
||||||
|
.union([
|
||||||
|
z.literal("off"),
|
||||||
|
z.literal("dropbox"),
|
||||||
|
z.literal("gdrive"),
|
||||||
|
z.literal("onedrive"),
|
||||||
|
z.literal("s3"),
|
||||||
|
z.literal("custom"),
|
||||||
|
])
|
||||||
|
.optional(),
|
||||||
|
remotePath: z.string().optional(),
|
||||||
|
localPath: z.string().optional(),
|
||||||
|
interval: z.number().int().nonnegative().optional(),
|
||||||
|
onSessionStart: z.boolean().optional(),
|
||||||
|
onSessionEnd: z.boolean().optional(),
|
||||||
|
remoteName: z.string().optional(),
|
||||||
|
configPath: z.string().optional(),
|
||||||
|
conflictResolve: z
|
||||||
|
.union([z.literal("newer"), z.literal("local"), z.literal("remote")])
|
||||||
|
.optional(),
|
||||||
|
exclude: z.array(z.string()).optional(),
|
||||||
|
s3: z
|
||||||
|
.object({
|
||||||
|
endpoint: z.string().optional(),
|
||||||
|
bucket: z.string().optional(),
|
||||||
|
region: z.string().optional(),
|
||||||
|
accessKeyId: z.string().optional(),
|
||||||
|
secretAccessKey: z.string().optional(),
|
||||||
|
})
|
||||||
|
.strict()
|
||||||
|
.optional(),
|
||||||
|
dropbox: z
|
||||||
|
.object({
|
||||||
|
appFolder: z.boolean().optional(),
|
||||||
|
appKey: z.string().optional(),
|
||||||
|
appSecret: z.string().optional(),
|
||||||
|
})
|
||||||
|
.strict()
|
||||||
|
.optional(),
|
||||||
|
})
|
||||||
|
.strict()
|
||||||
|
.optional(),
|
||||||
|
})
|
||||||
|
.strict()
|
||||||
|
.optional(),
|
||||||
})
|
})
|
||||||
.strict()
|
.strict()
|
||||||
.superRefine((cfg, ctx) => {
|
.superRefine((cfg, ctx) => {
|
||||||
|
|||||||
@ -5,6 +5,7 @@ import { type ChannelId, listChannelPlugins } from "../channels/plugins/index.js
|
|||||||
import { stopGmailWatcher } from "../hooks/gmail-watcher.js";
|
import { stopGmailWatcher } from "../hooks/gmail-watcher.js";
|
||||||
import type { HeartbeatRunner } from "../infra/heartbeat-runner.js";
|
import type { HeartbeatRunner } from "../infra/heartbeat-runner.js";
|
||||||
import type { PluginServicesHandle } from "../plugins/services.js";
|
import type { PluginServicesHandle } from "../plugins/services.js";
|
||||||
|
import { stopWorkspaceSyncManager } from "./workspace-sync-manager.js";
|
||||||
|
|
||||||
export function createGatewayCloseHandler(params: {
|
export function createGatewayCloseHandler(params: {
|
||||||
bonjourStop: (() => Promise<void>) | null;
|
bonjourStop: (() => Promise<void>) | null;
|
||||||
@ -68,6 +69,7 @@ export function createGatewayCloseHandler(params: {
|
|||||||
await params.pluginServices.stop().catch(() => {});
|
await params.pluginServices.stop().catch(() => {});
|
||||||
}
|
}
|
||||||
await stopGmailWatcher();
|
await stopGmailWatcher();
|
||||||
|
stopWorkspaceSyncManager();
|
||||||
params.cron.stop();
|
params.cron.stop();
|
||||||
params.heartbeatRunner.stop();
|
params.heartbeatRunner.stop();
|
||||||
for (const timer of params.nodePresenceTimers.values()) {
|
for (const timer of params.nodePresenceTimers.values()) {
|
||||||
|
|||||||
@ -22,6 +22,7 @@ import {
|
|||||||
scheduleRestartSentinelWake,
|
scheduleRestartSentinelWake,
|
||||||
shouldWakeFromRestartSentinel,
|
shouldWakeFromRestartSentinel,
|
||||||
} from "./server-restart-sentinel.js";
|
} from "./server-restart-sentinel.js";
|
||||||
|
import { startWorkspaceSyncManager } from "./workspace-sync-manager.js";
|
||||||
|
|
||||||
export async function startGatewaySidecars(params: {
|
export async function startGatewaySidecars(params: {
|
||||||
cfg: ReturnType<typeof loadConfig>;
|
cfg: ReturnType<typeof loadConfig>;
|
||||||
@ -139,6 +140,17 @@ export async function startGatewaySidecars(params: {
|
|||||||
}, 250);
|
}, 250);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Start workspace sync manager (runs rclone in background - no LLM cost).
|
||||||
|
try {
|
||||||
|
startWorkspaceSyncManager(params.cfg, {
|
||||||
|
info: (msg) => params.logHooks.info(msg),
|
||||||
|
warn: (msg) => params.logHooks.warn(msg),
|
||||||
|
error: (msg) => params.logHooks.error(msg),
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
params.logHooks.warn(`workspace sync manager failed to start: ${String(err)}`);
|
||||||
|
}
|
||||||
|
|
||||||
let pluginServices: PluginServicesHandle | null = null;
|
let pluginServices: PluginServicesHandle | null = null;
|
||||||
try {
|
try {
|
||||||
pluginServices = await startPluginServices({
|
pluginServices = await startPluginServices({
|
||||||
|
|||||||
190
src/gateway/workspace-sync-manager.ts
Normal file
190
src/gateway/workspace-sync-manager.ts
Normal file
@ -0,0 +1,190 @@
|
|||||||
|
/**
|
||||||
|
* Background workspace sync manager for the gateway.
|
||||||
|
*
|
||||||
|
* Runs rclone bisync at configured intervals WITHOUT involving the agent/LLM.
|
||||||
|
* This is a pure file operation that incurs zero token cost.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { MoltbotConfig } from "../config/config.js";
|
||||||
|
import { resolveAgentWorkspaceDir, resolveDefaultAgentId } from "../agents/agent-scope.js";
|
||||||
|
import {
|
||||||
|
isRcloneInstalled,
|
||||||
|
isRcloneConfigured,
|
||||||
|
resolveSyncConfig,
|
||||||
|
runBisync,
|
||||||
|
} from "../infra/rclone.js";
|
||||||
|
|
||||||
|
type SyncManagerLogger = {
|
||||||
|
info: (msg: string) => void;
|
||||||
|
warn: (msg: string) => void;
|
||||||
|
error: (msg: string) => void;
|
||||||
|
};
|
||||||
|
|
||||||
|
type SyncManagerState = {
|
||||||
|
intervalId: ReturnType<typeof setInterval> | null;
|
||||||
|
lastSyncAt: Date | null;
|
||||||
|
lastSyncOk: boolean | null;
|
||||||
|
syncCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
const state: SyncManagerState = {
|
||||||
|
intervalId: null,
|
||||||
|
lastSyncAt: null,
|
||||||
|
lastSyncOk: null,
|
||||||
|
syncCount: 0,
|
||||||
|
errorCount: 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
let currentConfig: MoltbotConfig | null = null;
|
||||||
|
let currentLogger: SyncManagerLogger | null = null;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run a single sync operation.
|
||||||
|
* This is a pure rclone operation - no agent/LLM involvement.
|
||||||
|
*/
|
||||||
|
async function runSync(): Promise<void> {
|
||||||
|
if (!currentConfig || !currentLogger) return;
|
||||||
|
|
||||||
|
const syncConfig = currentConfig.workspace?.sync;
|
||||||
|
if (!syncConfig?.provider || syncConfig.provider === "off") return;
|
||||||
|
|
||||||
|
const logger = currentLogger;
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Check if rclone is available
|
||||||
|
const installed = await isRcloneInstalled();
|
||||||
|
if (!installed) {
|
||||||
|
logger.warn("[workspace-sync] rclone not installed, skipping periodic sync");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve workspace and config
|
||||||
|
const agentId = resolveDefaultAgentId(currentConfig);
|
||||||
|
const workspaceDir = resolveAgentWorkspaceDir(currentConfig, agentId);
|
||||||
|
const resolved = resolveSyncConfig(syncConfig, workspaceDir);
|
||||||
|
|
||||||
|
// Check if configured
|
||||||
|
if (!isRcloneConfigured(resolved.configPath, resolved.remoteName)) {
|
||||||
|
logger.warn(`[workspace-sync] rclone not configured for "${resolved.remoteName}", skipping`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
`[workspace-sync] Running periodic sync: ${resolved.remoteName}:${resolved.remotePath}`,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Run bisync - pure file operation, no LLM involvement
|
||||||
|
const result = await runBisync({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
localPath: resolved.localPath,
|
||||||
|
conflictResolve: resolved.conflictResolve,
|
||||||
|
exclude: resolved.exclude,
|
||||||
|
});
|
||||||
|
|
||||||
|
state.lastSyncAt = new Date();
|
||||||
|
state.syncCount++;
|
||||||
|
|
||||||
|
if (result.ok) {
|
||||||
|
state.lastSyncOk = true;
|
||||||
|
logger.info("[workspace-sync] Periodic sync completed");
|
||||||
|
} else {
|
||||||
|
state.lastSyncOk = false;
|
||||||
|
state.errorCount++;
|
||||||
|
logger.warn(`[workspace-sync] Periodic sync failed: ${result.error}`);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
state.lastSyncOk = false;
|
||||||
|
state.errorCount++;
|
||||||
|
logger.error(
|
||||||
|
`[workspace-sync] Periodic sync error: ${err instanceof Error ? err.message : String(err)}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Start the background sync manager.
|
||||||
|
* Called when the gateway starts.
|
||||||
|
*/
|
||||||
|
export function startWorkspaceSyncManager(cfg: MoltbotConfig, logger: SyncManagerLogger): void {
|
||||||
|
// Stop any existing interval
|
||||||
|
stopWorkspaceSyncManager();
|
||||||
|
|
||||||
|
currentConfig = cfg;
|
||||||
|
currentLogger = logger;
|
||||||
|
|
||||||
|
const syncConfig = cfg.workspace?.sync;
|
||||||
|
if (!syncConfig?.provider || syncConfig.provider === "off") {
|
||||||
|
logger.info("[workspace-sync] Workspace sync not configured");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const intervalSeconds = syncConfig.interval ?? 0;
|
||||||
|
if (intervalSeconds <= 0) {
|
||||||
|
logger.info("[workspace-sync] Periodic sync disabled (interval=0)");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Minimum interval: 60 seconds to prevent thrashing
|
||||||
|
const effectiveInterval = Math.max(intervalSeconds, 60);
|
||||||
|
if (effectiveInterval !== intervalSeconds) {
|
||||||
|
logger.warn(
|
||||||
|
`[workspace-sync] Interval increased from ${intervalSeconds}s to ${effectiveInterval}s (minimum)`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
`[workspace-sync] Starting periodic sync every ${effectiveInterval}s (pure file sync, zero LLM cost)`,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Run initial sync after a short delay (let gateway fully start)
|
||||||
|
setTimeout(() => {
|
||||||
|
runSync().catch(() => {});
|
||||||
|
}, 5000);
|
||||||
|
|
||||||
|
// Set up periodic sync
|
||||||
|
state.intervalId = setInterval(() => {
|
||||||
|
runSync().catch(() => {});
|
||||||
|
}, effectiveInterval * 1000);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stop the background sync manager.
|
||||||
|
* Called when the gateway stops.
|
||||||
|
*/
|
||||||
|
export function stopWorkspaceSyncManager(): void {
|
||||||
|
if (state.intervalId) {
|
||||||
|
clearInterval(state.intervalId);
|
||||||
|
state.intervalId = null;
|
||||||
|
}
|
||||||
|
currentConfig = null;
|
||||||
|
currentLogger = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get current sync manager status.
|
||||||
|
*/
|
||||||
|
export function getWorkspaceSyncStatus(): {
|
||||||
|
running: boolean;
|
||||||
|
lastSyncAt: Date | null;
|
||||||
|
lastSyncOk: boolean | null;
|
||||||
|
syncCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
running: state.intervalId !== null,
|
||||||
|
lastSyncAt: state.lastSyncAt,
|
||||||
|
lastSyncOk: state.lastSyncOk,
|
||||||
|
syncCount: state.syncCount,
|
||||||
|
errorCount: state.errorCount,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Trigger an immediate sync (for CLI use).
|
||||||
|
*/
|
||||||
|
export async function triggerImmediateSync(): Promise<void> {
|
||||||
|
await runSync();
|
||||||
|
}
|
||||||
64
src/hooks/bundled/workspace-sync/HOOK.md
Normal file
64
src/hooks/bundled/workspace-sync/HOOK.md
Normal file
@ -0,0 +1,64 @@
|
|||||||
|
---
|
||||||
|
name: workspace-sync
|
||||||
|
description: Sync workspace with cloud storage (Dropbox, Google Drive, etc.) on session start/end
|
||||||
|
metadata:
|
||||||
|
events:
|
||||||
|
- session:start
|
||||||
|
- session:end
|
||||||
|
requires:
|
||||||
|
bins:
|
||||||
|
- rclone
|
||||||
|
config:
|
||||||
|
- workspace.sync.provider
|
||||||
|
---
|
||||||
|
|
||||||
|
# Workspace Sync Hook
|
||||||
|
|
||||||
|
Automatically syncs your agent workspace with cloud storage when sessions start or end.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Enable in `~/.clawdbot/moltbot.json`:
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox", // dropbox | gdrive | onedrive | s3 | custom
|
||||||
|
remotePath: "moltbot-share", // folder in cloud storage
|
||||||
|
localPath: "shared", // subfolder in workspace
|
||||||
|
onSessionStart: true, // sync when session starts
|
||||||
|
onSessionEnd: false // sync when session ends
|
||||||
|
}
|
||||||
|
},
|
||||||
|
hooks: {
|
||||||
|
internal: {
|
||||||
|
entries: {
|
||||||
|
"workspace-sync": { enabled: true }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- rclone (auto-installed via `moltbot workspace setup` if missing)
|
||||||
|
- Cloud provider account (Dropbox, Google Drive, OneDrive, or S3)
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Run the interactive setup wizard:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
moltbot workspace setup
|
||||||
|
```
|
||||||
|
|
||||||
|
The wizard will:
|
||||||
|
1. Install rclone (if needed)
|
||||||
|
2. Guide you through provider selection
|
||||||
|
3. Handle OAuth authorization
|
||||||
|
4. Configure sync settings
|
||||||
|
5. Run first sync
|
||||||
|
|
||||||
|
See [Workspace Cloud Sync](/gateway/workspace-sync) for full documentation.
|
||||||
327
src/hooks/bundled/workspace-sync/handler.test.ts
Normal file
327
src/hooks/bundled/workspace-sync/handler.test.ts
Normal file
@ -0,0 +1,327 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
|
||||||
|
import workspaceSyncHandler from "./handler.js";
|
||||||
|
import * as rclone from "../../../infra/rclone.js";
|
||||||
|
|
||||||
|
vi.mock("../../../infra/rclone.js", () => ({
|
||||||
|
isRcloneInstalled: vi.fn(),
|
||||||
|
isRcloneConfigured: vi.fn(),
|
||||||
|
resolveSyncConfig: vi.fn(),
|
||||||
|
runBisync: vi.fn(),
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock("../../../agents/agent-scope.js", () => ({
|
||||||
|
resolveAgentWorkspaceDir: vi.fn(() => "/workspace"),
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock("../../../routing/session-key.js", () => ({
|
||||||
|
resolveAgentIdFromSessionKey: vi.fn(() => "default"),
|
||||||
|
}));
|
||||||
|
|
||||||
|
describe("workspace-sync hook handler", () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.clearAllMocks();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores non-session events", async () => {
|
||||||
|
const event = {
|
||||||
|
type: "command",
|
||||||
|
action: "new",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.isRcloneInstalled).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores session events when sync is not configured", async () => {
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: { cfg: {} },
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.isRcloneInstalled).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores session start when onSessionStart is false", async () => {
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: false,
|
||||||
|
onSessionEnd: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.isRcloneInstalled).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores session end when onSessionEnd is false", async () => {
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "end",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true,
|
||||||
|
onSessionEnd: false,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.isRcloneInstalled).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores provider: off", async () => {
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "off",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.isRcloneInstalled).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns when rclone is not installed", async () => {
|
||||||
|
vi.mocked(rclone.isRcloneInstalled).mockResolvedValue(false);
|
||||||
|
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||||
|
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.isRcloneInstalled).toHaveBeenCalled();
|
||||||
|
expect(warnSpy).toHaveBeenCalledWith("[workspace-sync] rclone not installed, skipping sync");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns when rclone is not configured", async () => {
|
||||||
|
vi.mocked(rclone.isRcloneInstalled).mockResolvedValue(true);
|
||||||
|
vi.mocked(rclone.isRcloneConfigured).mockReturnValue(false);
|
||||||
|
vi.mocked(rclone.resolveSyncConfig).mockReturnValue({
|
||||||
|
provider: "dropbox",
|
||||||
|
remoteName: "cloud",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
localPath: "/workspace/shared",
|
||||||
|
configPath: "/home/.config/rclone/rclone.conf",
|
||||||
|
conflictResolve: "newer",
|
||||||
|
exclude: [],
|
||||||
|
interval: 0,
|
||||||
|
onSessionStart: true,
|
||||||
|
onSessionEnd: false,
|
||||||
|
});
|
||||||
|
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||||
|
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(warnSpy).toHaveBeenCalledWith(
|
||||||
|
expect.stringContaining('rclone not configured for remote "cloud"'),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("runs bisync on session start when configured", async () => {
|
||||||
|
vi.mocked(rclone.isRcloneInstalled).mockResolvedValue(true);
|
||||||
|
vi.mocked(rclone.isRcloneConfigured).mockReturnValue(true);
|
||||||
|
vi.mocked(rclone.resolveSyncConfig).mockReturnValue({
|
||||||
|
provider: "dropbox",
|
||||||
|
remoteName: "cloud",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
localPath: "/workspace/shared",
|
||||||
|
configPath: "/home/.config/rclone/rclone.conf",
|
||||||
|
conflictResolve: "newer",
|
||||||
|
exclude: [".git/**"],
|
||||||
|
interval: 0,
|
||||||
|
onSessionStart: true,
|
||||||
|
onSessionEnd: false,
|
||||||
|
});
|
||||||
|
vi.mocked(rclone.runBisync).mockResolvedValue({ ok: true });
|
||||||
|
vi.spyOn(console, "log").mockImplementation(() => {});
|
||||||
|
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.runBisync).toHaveBeenCalledWith(
|
||||||
|
expect.objectContaining({
|
||||||
|
remoteName: "cloud",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
localPath: "/workspace/shared",
|
||||||
|
conflictResolve: "newer",
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("handles session stop as end event", async () => {
|
||||||
|
vi.mocked(rclone.isRcloneInstalled).mockResolvedValue(true);
|
||||||
|
vi.mocked(rclone.isRcloneConfigured).mockReturnValue(true);
|
||||||
|
vi.mocked(rclone.resolveSyncConfig).mockReturnValue({
|
||||||
|
provider: "dropbox",
|
||||||
|
remoteName: "cloud",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
localPath: "/workspace/shared",
|
||||||
|
configPath: "/home/.config/rclone/rclone.conf",
|
||||||
|
conflictResolve: "newer",
|
||||||
|
exclude: [],
|
||||||
|
interval: 0,
|
||||||
|
onSessionStart: false,
|
||||||
|
onSessionEnd: true,
|
||||||
|
});
|
||||||
|
vi.mocked(rclone.runBisync).mockResolvedValue({ ok: true });
|
||||||
|
vi.spyOn(console, "log").mockImplementation(() => {});
|
||||||
|
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "stop",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionEnd: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(rclone.runBisync).toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns about --resync on first run error", async () => {
|
||||||
|
vi.mocked(rclone.isRcloneInstalled).mockResolvedValue(true);
|
||||||
|
vi.mocked(rclone.isRcloneConfigured).mockReturnValue(true);
|
||||||
|
vi.mocked(rclone.resolveSyncConfig).mockReturnValue({
|
||||||
|
provider: "dropbox",
|
||||||
|
remoteName: "cloud",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
localPath: "/workspace/shared",
|
||||||
|
configPath: "/home/.config/rclone/rclone.conf",
|
||||||
|
conflictResolve: "newer",
|
||||||
|
exclude: [],
|
||||||
|
interval: 0,
|
||||||
|
onSessionStart: true,
|
||||||
|
onSessionEnd: false,
|
||||||
|
});
|
||||||
|
vi.mocked(rclone.runBisync).mockResolvedValue({
|
||||||
|
ok: false,
|
||||||
|
error: "bisync requires --resync on first run",
|
||||||
|
});
|
||||||
|
vi.spyOn(console, "log").mockImplementation(() => {});
|
||||||
|
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||||
|
|
||||||
|
const event = {
|
||||||
|
type: "session",
|
||||||
|
action: "start",
|
||||||
|
sessionKey: "test-session",
|
||||||
|
context: {
|
||||||
|
cfg: {
|
||||||
|
workspace: {
|
||||||
|
sync: {
|
||||||
|
provider: "dropbox",
|
||||||
|
remotePath: "moltbot-share",
|
||||||
|
onSessionStart: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await workspaceSyncHandler(event as never);
|
||||||
|
|
||||||
|
expect(warnSpy).toHaveBeenCalledWith(
|
||||||
|
expect.stringContaining("First sync requires manual --resync"),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
118
src/hooks/bundled/workspace-sync/handler.ts
Normal file
118
src/hooks/bundled/workspace-sync/handler.ts
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
/**
|
||||||
|
* Workspace sync hook handler
|
||||||
|
*
|
||||||
|
* Syncs workspace with cloud storage on session start/end
|
||||||
|
* when workspace.sync is configured.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { MoltbotConfig } from "../../../config/config.js";
|
||||||
|
import { resolveAgentWorkspaceDir } from "../../../agents/agent-scope.js";
|
||||||
|
import { resolveAgentIdFromSessionKey } from "../../../routing/session-key.js";
|
||||||
|
import type { HookHandler } from "../../hooks.js";
|
||||||
|
import {
|
||||||
|
isRcloneInstalled,
|
||||||
|
isRcloneConfigured,
|
||||||
|
resolveSyncConfig,
|
||||||
|
runBisync,
|
||||||
|
} from "../../../infra/rclone.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sync workspace on session start or end
|
||||||
|
*/
|
||||||
|
const workspaceSyncHandler: HookHandler = async (event) => {
|
||||||
|
// Only handle session events
|
||||||
|
if (event.type !== "session") {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const context = event.context || {};
|
||||||
|
const cfg = context.cfg as MoltbotConfig | undefined;
|
||||||
|
|
||||||
|
if (!cfg?.workspace?.sync) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const syncConfig = cfg.workspace.sync;
|
||||||
|
|
||||||
|
// Check if sync is enabled for this event
|
||||||
|
const isStart = event.action === "start";
|
||||||
|
const isEnd = event.action === "end" || event.action === "stop";
|
||||||
|
|
||||||
|
if (isStart && !syncConfig.onSessionStart) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (isEnd && !syncConfig.onSessionEnd) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!isStart && !isEnd) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if provider is configured
|
||||||
|
if (!syncConfig.provider || syncConfig.provider === "off") {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`[workspace-sync] Triggered on session ${event.action}`);
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Check if rclone is installed
|
||||||
|
const installed = await isRcloneInstalled();
|
||||||
|
if (!installed) {
|
||||||
|
console.warn("[workspace-sync] rclone not installed, skipping sync");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve workspace and config
|
||||||
|
const agentId = resolveAgentIdFromSessionKey(event.sessionKey);
|
||||||
|
const workspaceDir = resolveAgentWorkspaceDir(cfg, agentId);
|
||||||
|
const stateDir = context.stateDir as string | undefined;
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(syncConfig, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
// Check if rclone is configured
|
||||||
|
if (!isRcloneConfigured(resolved.configPath, resolved.remoteName)) {
|
||||||
|
console.warn(
|
||||||
|
`[workspace-sync] rclone not configured for remote "${resolved.remoteName}", skipping sync`,
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
`[workspace-sync] Syncing ${resolved.remoteName}:${resolved.remotePath} <-> ${resolved.localPath}`,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Run sync
|
||||||
|
const result = await runBisync({
|
||||||
|
configPath: resolved.configPath,
|
||||||
|
remoteName: resolved.remoteName,
|
||||||
|
remotePath: resolved.remotePath,
|
||||||
|
localPath: resolved.localPath,
|
||||||
|
conflictResolve: resolved.conflictResolve,
|
||||||
|
exclude: resolved.exclude,
|
||||||
|
verbose: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (result.ok) {
|
||||||
|
console.log("[workspace-sync] Sync completed successfully");
|
||||||
|
if (result.filesTransferred) {
|
||||||
|
console.log(`[workspace-sync] Files transferred: ${result.filesTransferred}`);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Check if this is a first-run issue
|
||||||
|
if (result.error?.includes("--resync")) {
|
||||||
|
console.warn(
|
||||||
|
"[workspace-sync] First sync requires manual --resync. Run: moltbot workspace sync --resync",
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
console.error(`[workspace-sync] Sync failed: ${result.error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
console.error("[workspace-sync] Error:", err instanceof Error ? err.message : String(err));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
export default workspaceSyncHandler;
|
||||||
157
src/infra/rclone.test.ts
Normal file
157
src/infra/rclone.test.ts
Normal file
@ -0,0 +1,157 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
|
import { resolveSyncConfig, generateRcloneConfig, isRcloneConfigured } from "./rclone.js";
|
||||||
|
|
||||||
|
describe("rclone helpers", () => {
|
||||||
|
describe("resolveSyncConfig", () => {
|
||||||
|
it("uses defaults when config is minimal", () => {
|
||||||
|
const config = { provider: "dropbox" as const, remotePath: "test-folder" };
|
||||||
|
const workspaceDir = "/home/user/workspace";
|
||||||
|
const stateDir = "/home/user/.moltbot";
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(config, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
expect(resolved.provider).toBe("dropbox");
|
||||||
|
expect(resolved.remotePath).toBe("test-folder");
|
||||||
|
expect(resolved.localPath).toBe("/home/user/workspace/shared");
|
||||||
|
expect(resolved.remoteName).toBe("cloud");
|
||||||
|
expect(resolved.conflictResolve).toBe("newer");
|
||||||
|
expect(resolved.interval).toBe(0);
|
||||||
|
expect(resolved.onSessionStart).toBe(false);
|
||||||
|
expect(resolved.onSessionEnd).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("respects custom localPath", () => {
|
||||||
|
const config = {
|
||||||
|
provider: "dropbox" as const,
|
||||||
|
remotePath: "test-folder",
|
||||||
|
localPath: "sync",
|
||||||
|
};
|
||||||
|
const workspaceDir = "/home/user/workspace";
|
||||||
|
const stateDir = "/home/user/.moltbot";
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(config, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
expect(resolved.localPath).toBe("/home/user/workspace/sync");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("respects custom remoteName", () => {
|
||||||
|
const config = {
|
||||||
|
provider: "dropbox" as const,
|
||||||
|
remotePath: "test-folder",
|
||||||
|
remoteName: "my-dropbox",
|
||||||
|
};
|
||||||
|
const workspaceDir = "/home/user/workspace";
|
||||||
|
const stateDir = "/home/user/.moltbot";
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(config, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
expect(resolved.remoteName).toBe("my-dropbox");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("respects interval and session hooks", () => {
|
||||||
|
const config = {
|
||||||
|
provider: "dropbox" as const,
|
||||||
|
remotePath: "test-folder",
|
||||||
|
interval: 300,
|
||||||
|
onSessionStart: true,
|
||||||
|
onSessionEnd: true,
|
||||||
|
};
|
||||||
|
const workspaceDir = "/home/user/workspace";
|
||||||
|
const stateDir = "/home/user/.moltbot";
|
||||||
|
|
||||||
|
const resolved = resolveSyncConfig(config, workspaceDir, stateDir);
|
||||||
|
|
||||||
|
expect(resolved.interval).toBe(300);
|
||||||
|
expect(resolved.onSessionStart).toBe(true);
|
||||||
|
expect(resolved.onSessionEnd).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("applies default excludes", () => {
|
||||||
|
const config = { provider: "dropbox" as const, remotePath: "test" };
|
||||||
|
const resolved = resolveSyncConfig(config, "/workspace", "/state");
|
||||||
|
|
||||||
|
expect(resolved.exclude).toContain(".git/**");
|
||||||
|
expect(resolved.exclude).toContain("node_modules/**");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("respects custom excludes", () => {
|
||||||
|
const config = {
|
||||||
|
provider: "dropbox" as const,
|
||||||
|
remotePath: "test",
|
||||||
|
exclude: ["*.tmp", "cache/**"],
|
||||||
|
};
|
||||||
|
const resolved = resolveSyncConfig(config, "/workspace", "/state");
|
||||||
|
|
||||||
|
expect(resolved.exclude).toEqual(["*.tmp", "cache/**"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("generateRcloneConfig", () => {
|
||||||
|
it("generates dropbox config with token", () => {
|
||||||
|
const config = generateRcloneConfig("dropbox", "cloud", '{"access_token":"abc123"}');
|
||||||
|
|
||||||
|
expect(config).toContain("[cloud]");
|
||||||
|
expect(config).toContain("type = dropbox");
|
||||||
|
expect(config).toContain('token = {"access_token":"abc123"}');
|
||||||
|
});
|
||||||
|
|
||||||
|
it("generates gdrive config", () => {
|
||||||
|
const config = generateRcloneConfig("gdrive", "drive", '{"access_token":"xyz"}');
|
||||||
|
|
||||||
|
expect(config).toContain("[drive]");
|
||||||
|
expect(config).toContain("type = drive");
|
||||||
|
expect(config).toContain('token = {"access_token":"xyz"}');
|
||||||
|
});
|
||||||
|
|
||||||
|
it("generates onedrive config", () => {
|
||||||
|
const config = generateRcloneConfig("onedrive", "od", '{"access_token":"123"}');
|
||||||
|
|
||||||
|
expect(config).toContain("[od]");
|
||||||
|
expect(config).toContain("type = onedrive");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("includes app key/secret for dropbox app folder", () => {
|
||||||
|
const config = generateRcloneConfig("dropbox", "cloud", '{"access_token":"abc"}', {
|
||||||
|
dropbox: { appKey: "key123", appSecret: "secret456" },
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(config).toContain("client_id = key123");
|
||||||
|
expect(config).toContain("client_secret = secret456");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("generates s3 config with endpoint", () => {
|
||||||
|
const config = generateRcloneConfig("s3", "r2", "", {
|
||||||
|
s3: {
|
||||||
|
endpoint: "https://xxx.r2.cloudflarestorage.com",
|
||||||
|
accessKeyId: "AKID",
|
||||||
|
secretAccessKey: "SECRET",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(config).toContain("[r2]");
|
||||||
|
expect(config).toContain("type = s3");
|
||||||
|
expect(config).toContain("endpoint = https://xxx.r2.cloudflarestorage.com");
|
||||||
|
expect(config).toContain("access_key_id = AKID");
|
||||||
|
expect(config).toContain("secret_access_key = SECRET");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("includes region for s3 when provided", () => {
|
||||||
|
const config = generateRcloneConfig("s3", "aws", "", {
|
||||||
|
s3: {
|
||||||
|
region: "us-east-1",
|
||||||
|
bucket: "my-bucket",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(config).toContain("region = us-east-1");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("isRcloneConfigured", () => {
|
||||||
|
it("returns false when config file does not exist", () => {
|
||||||
|
const result = isRcloneConfigured("/nonexistent/path/rclone.conf", "cloud");
|
||||||
|
expect(result).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
516
src/infra/rclone.ts
Normal file
516
src/infra/rclone.ts
Normal file
@ -0,0 +1,516 @@
|
|||||||
|
import { existsSync, mkdirSync, writeFileSync, readFileSync } from "node:fs";
|
||||||
|
import { dirname, join } from "node:path";
|
||||||
|
import { homedir, platform } from "node:os";
|
||||||
|
import { logVerbose } from "../globals.js";
|
||||||
|
import { runExec } from "../process/exec.js";
|
||||||
|
import type { WorkspaceSyncConfig, WorkspaceSyncProvider } from "../config/types.workspace.js";
|
||||||
|
import type { RuntimeEnv } from "../runtime.js";
|
||||||
|
import { defaultRuntime } from "../runtime.js";
|
||||||
|
|
||||||
|
const DEFAULT_REMOTE_NAME = "cloud";
|
||||||
|
const DEFAULT_LOCAL_PATH = "shared";
|
||||||
|
const DEFAULT_REMOTE_PATH = "moltbot-share";
|
||||||
|
const DEFAULT_CONFLICT_RESOLVE = "newer";
|
||||||
|
const DEFAULT_EXCLUDES = [".git/**", "node_modules/**", "*.log", ".DS_Store"];
|
||||||
|
|
||||||
|
export type RcloneSyncResult = {
|
||||||
|
ok: boolean;
|
||||||
|
error?: string;
|
||||||
|
filesTransferred?: number;
|
||||||
|
bytesTransferred?: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Find rclone binary in PATH or common locations.
|
||||||
|
*/
|
||||||
|
export async function findRcloneBinary(): Promise<string | null> {
|
||||||
|
const checkBinary = async (path: string): Promise<boolean> => {
|
||||||
|
if (!path || (path.startsWith("/") && !existsSync(path))) return false;
|
||||||
|
try {
|
||||||
|
await runExec(path, ["--version"], { timeoutMs: 3000 });
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Strategy 1: which command
|
||||||
|
try {
|
||||||
|
const { stdout } = await runExec("which", ["rclone"]);
|
||||||
|
const fromPath = stdout.trim();
|
||||||
|
if (fromPath && (await checkBinary(fromPath))) {
|
||||||
|
return fromPath;
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// which failed, continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Strategy 2: Common install locations
|
||||||
|
const commonPaths = ["/usr/local/bin/rclone", "/usr/bin/rclone", "/opt/homebrew/bin/rclone"];
|
||||||
|
for (const path of commonPaths) {
|
||||||
|
if (await checkBinary(path)) {
|
||||||
|
return path;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
let cachedRcloneBinary: string | null = null;
|
||||||
|
|
||||||
|
export async function getRcloneBinary(): Promise<string> {
|
||||||
|
if (cachedRcloneBinary) return cachedRcloneBinary;
|
||||||
|
cachedRcloneBinary = await findRcloneBinary();
|
||||||
|
return cachedRcloneBinary ?? "rclone";
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if rclone is installed.
|
||||||
|
*/
|
||||||
|
export async function isRcloneInstalled(): Promise<boolean> {
|
||||||
|
const binary = await findRcloneBinary();
|
||||||
|
return binary !== null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ensure rclone is installed, offering to install if missing.
|
||||||
|
* Returns true if rclone is available, false if user declined install.
|
||||||
|
*/
|
||||||
|
export async function ensureRcloneInstalled(
|
||||||
|
prompt: (message: string, defaultValue: boolean) => Promise<boolean>,
|
||||||
|
exec: typeof runExec = runExec,
|
||||||
|
runtime: RuntimeEnv = defaultRuntime,
|
||||||
|
): Promise<boolean> {
|
||||||
|
const installed = await isRcloneInstalled();
|
||||||
|
if (installed) return true;
|
||||||
|
|
||||||
|
const isMac = platform() === "darwin";
|
||||||
|
const isLinux = platform() === "linux";
|
||||||
|
|
||||||
|
if (isMac) {
|
||||||
|
// Check if Homebrew is available
|
||||||
|
const hasBrew = await exec("which", ["brew"]).then(
|
||||||
|
() => true,
|
||||||
|
() => false,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (hasBrew) {
|
||||||
|
const install = await prompt(
|
||||||
|
"rclone not found. Install via Homebrew (brew install rclone)?",
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
if (!install) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
logVerbose("Installing rclone via Homebrew...");
|
||||||
|
try {
|
||||||
|
await exec("brew", ["install", "rclone"], { timeoutMs: 120_000 });
|
||||||
|
// Clear cached binary so we find the new one
|
||||||
|
cachedRcloneBinary = null;
|
||||||
|
return true;
|
||||||
|
} catch (err) {
|
||||||
|
runtime.error(
|
||||||
|
`Failed to install rclone: ${err instanceof Error ? err.message : String(err)}`,
|
||||||
|
);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (isLinux || isMac) {
|
||||||
|
const install = await prompt(
|
||||||
|
"rclone not found. Install via official script (curl https://rclone.org/install.sh)?",
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
if (!install) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
logVerbose("Installing rclone via official script...");
|
||||||
|
try {
|
||||||
|
// Download and run the install script
|
||||||
|
const { stdout } = await exec("curl", ["-s", "https://rclone.org/install.sh"], {
|
||||||
|
timeoutMs: 30_000,
|
||||||
|
});
|
||||||
|
await exec("sudo", ["bash", "-c", stdout], { timeoutMs: 120_000 });
|
||||||
|
// Clear cached binary so we find the new one
|
||||||
|
cachedRcloneBinary = null;
|
||||||
|
return true;
|
||||||
|
} catch (err) {
|
||||||
|
runtime.error(
|
||||||
|
`Failed to install rclone: ${err instanceof Error ? err.message : String(err)}`,
|
||||||
|
);
|
||||||
|
runtime.error("Try installing manually: https://rclone.org/install/");
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
runtime.error("rclone not found. Please install manually: https://rclone.org/install/");
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get the default rclone config path.
|
||||||
|
*/
|
||||||
|
export function getDefaultRcloneConfigPath(stateDir?: string): string {
|
||||||
|
const base = stateDir ?? process.env.CLAWDBOT_STATE_DIR ?? join(homedir(), ".clawdbot");
|
||||||
|
return join(base, ".config", "rclone", "rclone.conf");
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolve sync config with defaults.
|
||||||
|
*/
|
||||||
|
export function resolveSyncConfig(
|
||||||
|
config: WorkspaceSyncConfig | undefined,
|
||||||
|
workspace: string,
|
||||||
|
stateDir?: string,
|
||||||
|
): {
|
||||||
|
provider: WorkspaceSyncProvider;
|
||||||
|
remoteName: string;
|
||||||
|
remotePath: string;
|
||||||
|
localPath: string;
|
||||||
|
configPath: string;
|
||||||
|
conflictResolve: "newer" | "local" | "remote";
|
||||||
|
exclude: string[];
|
||||||
|
interval: number;
|
||||||
|
onSessionStart: boolean;
|
||||||
|
onSessionEnd: boolean;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
provider: config?.provider ?? "off",
|
||||||
|
remoteName: config?.remoteName ?? DEFAULT_REMOTE_NAME,
|
||||||
|
remotePath: config?.remotePath ?? DEFAULT_REMOTE_PATH,
|
||||||
|
localPath: join(workspace, config?.localPath ?? DEFAULT_LOCAL_PATH),
|
||||||
|
configPath: config?.configPath ?? getDefaultRcloneConfigPath(stateDir),
|
||||||
|
conflictResolve: config?.conflictResolve ?? DEFAULT_CONFLICT_RESOLVE,
|
||||||
|
exclude: config?.exclude ?? DEFAULT_EXCLUDES,
|
||||||
|
interval: config?.interval ?? 0,
|
||||||
|
onSessionStart: config?.onSessionStart ?? false,
|
||||||
|
onSessionEnd: config?.onSessionEnd ?? false,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get rclone type string for a provider.
|
||||||
|
*/
|
||||||
|
function getRcloneType(provider: WorkspaceSyncProvider): string {
|
||||||
|
switch (provider) {
|
||||||
|
case "dropbox":
|
||||||
|
return "dropbox";
|
||||||
|
case "gdrive":
|
||||||
|
return "drive";
|
||||||
|
case "onedrive":
|
||||||
|
return "onedrive";
|
||||||
|
case "s3":
|
||||||
|
return "s3";
|
||||||
|
default:
|
||||||
|
return "unknown";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate rclone config content for a provider.
|
||||||
|
*/
|
||||||
|
export function generateRcloneConfig(
|
||||||
|
provider: WorkspaceSyncProvider,
|
||||||
|
remoteName: string,
|
||||||
|
token: string,
|
||||||
|
options?: {
|
||||||
|
dropbox?: { appKey?: string; appSecret?: string };
|
||||||
|
s3?: {
|
||||||
|
endpoint?: string;
|
||||||
|
bucket?: string;
|
||||||
|
region?: string;
|
||||||
|
accessKeyId?: string;
|
||||||
|
secretAccessKey?: string;
|
||||||
|
};
|
||||||
|
},
|
||||||
|
): string {
|
||||||
|
const type = getRcloneType(provider);
|
||||||
|
let config = `[${remoteName}]\ntype = ${type}\n`;
|
||||||
|
|
||||||
|
if (provider === "dropbox") {
|
||||||
|
config += `token = ${token}\n`;
|
||||||
|
if (options?.dropbox?.appKey) {
|
||||||
|
config += `client_id = ${options.dropbox.appKey}\n`;
|
||||||
|
}
|
||||||
|
if (options?.dropbox?.appSecret) {
|
||||||
|
config += `client_secret = ${options.dropbox.appSecret}\n`;
|
||||||
|
}
|
||||||
|
} else if (provider === "gdrive") {
|
||||||
|
config += `token = ${token}\n`;
|
||||||
|
} else if (provider === "onedrive") {
|
||||||
|
config += `token = ${token}\n`;
|
||||||
|
} else if (provider === "s3") {
|
||||||
|
if (options?.s3?.endpoint) {
|
||||||
|
config += `endpoint = ${options.s3.endpoint}\n`;
|
||||||
|
}
|
||||||
|
if (options?.s3?.region) {
|
||||||
|
config += `region = ${options.s3.region}\n`;
|
||||||
|
}
|
||||||
|
if (options?.s3?.accessKeyId) {
|
||||||
|
config += `access_key_id = ${options.s3.accessKeyId}\n`;
|
||||||
|
}
|
||||||
|
if (options?.s3?.secretAccessKey) {
|
||||||
|
config += `secret_access_key = ${options.s3.secretAccessKey}\n`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return config;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Write rclone config to disk.
|
||||||
|
*/
|
||||||
|
export function writeRcloneConfig(configPath: string, content: string): void {
|
||||||
|
const dir = dirname(configPath);
|
||||||
|
if (!existsSync(dir)) {
|
||||||
|
mkdirSync(dir, { recursive: true });
|
||||||
|
}
|
||||||
|
writeFileSync(configPath, content, { mode: 0o600 });
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if rclone config exists and has the remote configured.
|
||||||
|
*/
|
||||||
|
export function isRcloneConfigured(configPath: string, remoteName: string): boolean {
|
||||||
|
if (!existsSync(configPath)) return false;
|
||||||
|
try {
|
||||||
|
const content = readFileSync(configPath, "utf-8");
|
||||||
|
return content.includes(`[${remoteName}]`);
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run rclone authorize command (returns the token).
|
||||||
|
* This must be run on a machine with a browser.
|
||||||
|
*/
|
||||||
|
export async function authorizeRclone(
|
||||||
|
provider: WorkspaceSyncProvider,
|
||||||
|
appKey?: string,
|
||||||
|
appSecret?: string,
|
||||||
|
): Promise<{ ok: true; token: string } | { ok: false; error: string }> {
|
||||||
|
const rcloneBin = await getRcloneBinary();
|
||||||
|
const type = getRcloneType(provider);
|
||||||
|
|
||||||
|
const args = ["authorize", type];
|
||||||
|
if (appKey && appSecret) {
|
||||||
|
args.push(appKey, appSecret);
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { stdout, stderr } = await runExec(rcloneBin, args, {
|
||||||
|
timeoutMs: 300_000, // 5 minutes for OAuth flow
|
||||||
|
maxBuffer: 1_000_000,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Extract token from output
|
||||||
|
const combined = stdout + stderr;
|
||||||
|
const tokenMatch = combined.match(/\{[^}]*"access_token"[^}]*\}/);
|
||||||
|
if (tokenMatch) {
|
||||||
|
return { ok: true, token: tokenMatch[0] };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Try to find JSON object in output
|
||||||
|
const jsonMatch = combined.match(
|
||||||
|
/Paste the following into your remote machine[\s\S]*?(\{[\s\S]*?\})\s*$/m,
|
||||||
|
);
|
||||||
|
if (jsonMatch) {
|
||||||
|
return { ok: true, token: jsonMatch[1] };
|
||||||
|
}
|
||||||
|
|
||||||
|
return { ok: false, error: "Could not extract token from rclone output" };
|
||||||
|
} catch (err) {
|
||||||
|
const message = err instanceof Error ? err.message : String(err);
|
||||||
|
return { ok: false, error: `Authorization failed: ${message}` };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run bidirectional sync using rclone bisync.
|
||||||
|
*/
|
||||||
|
export async function runBisync(params: {
|
||||||
|
configPath: string;
|
||||||
|
remoteName: string;
|
||||||
|
remotePath: string;
|
||||||
|
localPath: string;
|
||||||
|
conflictResolve: "newer" | "local" | "remote";
|
||||||
|
exclude: string[];
|
||||||
|
resync?: boolean;
|
||||||
|
dryRun?: boolean;
|
||||||
|
verbose?: boolean;
|
||||||
|
}): Promise<RcloneSyncResult> {
|
||||||
|
const rcloneBin = await getRcloneBinary();
|
||||||
|
|
||||||
|
// Ensure local directory exists
|
||||||
|
if (!existsSync(params.localPath)) {
|
||||||
|
mkdirSync(params.localPath, { recursive: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
const args = [
|
||||||
|
"bisync",
|
||||||
|
`${params.remoteName}:${params.remotePath}`,
|
||||||
|
params.localPath,
|
||||||
|
"--config",
|
||||||
|
params.configPath,
|
||||||
|
"--conflict-resolve",
|
||||||
|
params.conflictResolve,
|
||||||
|
"--conflict-suffix",
|
||||||
|
".conflict",
|
||||||
|
];
|
||||||
|
|
||||||
|
// Add excludes
|
||||||
|
for (const pattern of params.exclude) {
|
||||||
|
args.push("--exclude", pattern);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.resync) {
|
||||||
|
args.push("--resync");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.dryRun) {
|
||||||
|
args.push("--dry-run");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.verbose) {
|
||||||
|
args.push("--verbose");
|
||||||
|
}
|
||||||
|
|
||||||
|
logVerbose(`Running: ${rcloneBin} ${args.join(" ")}`);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { stdout, stderr } = await runExec(rcloneBin, args, {
|
||||||
|
timeoutMs: 600_000, // 10 minutes
|
||||||
|
maxBuffer: 10_000_000,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Parse output for stats
|
||||||
|
const combined = stdout + stderr;
|
||||||
|
const transferredMatch = combined.match(/Transferred:\s*(\d+)\s*\/\s*(\d+)/);
|
||||||
|
|
||||||
|
return {
|
||||||
|
ok: true,
|
||||||
|
filesTransferred: transferredMatch ? parseInt(transferredMatch[1], 10) : undefined,
|
||||||
|
};
|
||||||
|
} catch (err) {
|
||||||
|
const errObj = err as { stdout?: string; stderr?: string; message?: string };
|
||||||
|
const message =
|
||||||
|
errObj.stderr?.trim() || errObj.stdout?.trim() || errObj.message || "Unknown sync error";
|
||||||
|
|
||||||
|
// Check for common errors
|
||||||
|
if (message.includes("bisync requires --resync")) {
|
||||||
|
return {
|
||||||
|
ok: false,
|
||||||
|
error: "First sync requires --resync flag to establish baseline",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return { ok: false, error: message };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run one-way sync (copy) from remote to local or vice versa.
|
||||||
|
*/
|
||||||
|
export async function runSync(params: {
|
||||||
|
configPath: string;
|
||||||
|
remoteName: string;
|
||||||
|
remotePath: string;
|
||||||
|
localPath: string;
|
||||||
|
direction: "pull" | "push";
|
||||||
|
exclude: string[];
|
||||||
|
dryRun?: boolean;
|
||||||
|
verbose?: boolean;
|
||||||
|
}): Promise<RcloneSyncResult> {
|
||||||
|
const rcloneBin = await getRcloneBinary();
|
||||||
|
|
||||||
|
// Ensure local directory exists
|
||||||
|
if (!existsSync(params.localPath)) {
|
||||||
|
mkdirSync(params.localPath, { recursive: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
const remote = `${params.remoteName}:${params.remotePath}`;
|
||||||
|
const [source, dest] =
|
||||||
|
params.direction === "pull" ? [remote, params.localPath] : [params.localPath, remote];
|
||||||
|
|
||||||
|
const args = ["sync", source, dest, "--config", params.configPath];
|
||||||
|
|
||||||
|
for (const pattern of params.exclude) {
|
||||||
|
args.push("--exclude", pattern);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.dryRun) {
|
||||||
|
args.push("--dry-run");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.verbose) {
|
||||||
|
args.push("--verbose");
|
||||||
|
}
|
||||||
|
|
||||||
|
logVerbose(`Running: ${rcloneBin} ${args.join(" ")}`);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await runExec(rcloneBin, args, {
|
||||||
|
timeoutMs: 600_000,
|
||||||
|
maxBuffer: 10_000_000,
|
||||||
|
});
|
||||||
|
return { ok: true };
|
||||||
|
} catch (err) {
|
||||||
|
const errObj = err as { stderr?: string; message?: string };
|
||||||
|
const message = errObj.stderr?.trim() || errObj.message || "Unknown sync error";
|
||||||
|
return { ok: false, error: message };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* List files in remote.
|
||||||
|
*/
|
||||||
|
export async function listRemote(params: {
|
||||||
|
configPath: string;
|
||||||
|
remoteName: string;
|
||||||
|
remotePath: string;
|
||||||
|
}): Promise<{ ok: true; files: string[] } | { ok: false; error: string }> {
|
||||||
|
const rcloneBin = await getRcloneBinary();
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { stdout } = await runExec(
|
||||||
|
rcloneBin,
|
||||||
|
["lsf", `${params.remoteName}:${params.remotePath}`, "--config", params.configPath],
|
||||||
|
{ timeoutMs: 30_000, maxBuffer: 1_000_000 },
|
||||||
|
);
|
||||||
|
|
||||||
|
const files = stdout
|
||||||
|
.trim()
|
||||||
|
.split("\n")
|
||||||
|
.filter((f) => f.length > 0);
|
||||||
|
return { ok: true, files };
|
||||||
|
} catch (err) {
|
||||||
|
const errObj = err as { stderr?: string; message?: string };
|
||||||
|
const message = errObj.stderr?.trim() || errObj.message || "Unknown error";
|
||||||
|
return { ok: false, error: message };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check remote connection.
|
||||||
|
*/
|
||||||
|
export async function checkRemote(params: {
|
||||||
|
configPath: string;
|
||||||
|
remoteName: string;
|
||||||
|
}): Promise<{ ok: true } | { ok: false; error: string }> {
|
||||||
|
const rcloneBin = await getRcloneBinary();
|
||||||
|
|
||||||
|
try {
|
||||||
|
await runExec(
|
||||||
|
rcloneBin,
|
||||||
|
["about", `${params.remoteName}:`, "--config", params.configPath, "--json"],
|
||||||
|
{ timeoutMs: 30_000, maxBuffer: 100_000 },
|
||||||
|
);
|
||||||
|
return { ok: true };
|
||||||
|
} catch (err) {
|
||||||
|
const errObj = err as { stderr?: string; message?: string };
|
||||||
|
const message = errObj.stderr?.trim() || errObj.message || "Connection failed";
|
||||||
|
return { ok: false, error: message };
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
Reference in New Issue
Block a user