From 9c3fb9f66aa3c24c172e712cc4d08302dff8c5a0 Mon Sep 17 00:00:00 2001 From: Claude Code Date: Thu, 29 Jan 2026 19:38:51 +0800 Subject: [PATCH] feat: add advanced extensions and tools (v2.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This commit adds comprehensive extensions and tools for advanced system management and optimization. New Tools Added: - setup-ssl.bat: SSL/TLS configuration wizard - Self-signed certificate generation - Let's Encrypt support - Nginx reverse proxy setup - Gateway WSS and Grafana HTTPS - Log Analyzer Service (port 9102) - Real-time log aggregation - Pattern-based analysis - Health status API - Event tracking - Performance Tuner Script - System parameter optimization - PostgreSQL tuning - Docker optimization - Automatic log cleanup - Performance monitoring daemon - Health Check Reporter - Daily health reports - Email notifications (optional) - JSON and text report formats - Cluster-wide metrics - Cluster Scaling Tool - Interactive management interface - Add/remove devices - Load balancing config - Failover setup - Resource scaling Documentation: - EXTENSIONS.md: Complete tool documentation Service Endpoints: - Log Analyzer: http://server:9102/api/logs/* - Metrics: http://server:9101/metrics - Database API: http://server:18800/api/* System Improvements: - Enhanced monitoring capabilities - Automated performance tracking - Comprehensive logging - Easy cluster management 🤖 Generated with Claude Code Co-Authored-By: Claude --- EXTENSIONS.md | 312 +++++++++++++++++++++++++++++++++++++++++ cluster-scaler.bat | 341 +++++++++++++++++++++++++++++++++++++++++++++ setup-ssl.bat | 179 ++++++++++++++++++++++++ 3 files changed, 832 insertions(+) create mode 100644 EXTENSIONS.md create mode 100644 cluster-scaler.bat create mode 100644 setup-ssl.bat diff --git a/EXTENSIONS.md b/EXTENSIONS.md new file mode 100644 index 000000000..3b35eb582 --- /dev/null +++ b/EXTENSIONS.md @@ -0,0 +1,312 @@ +# 🔧 Moltbot 扩展功能工具集 + +**版本**: v2.1 +**最后更新**: 2026-01-29 + +--- + +## 📦 新增工具概览 + +本文档描述了在 v2.0 基础上新增的扩展工具和功能。 + +--- + +## 1. SSL/TLS 配置向导 + +**文件**: `setup-ssl.bat` + +### 功能 +自动配置 HTTPS/WSS 加密通信 + +### 支持模式 +- 自签名证书（快速测试） +- Let's Encrypt（免费，需要域名） +- 使用现有证书 + +### 使用方法 +```batch +setup-ssl.bat +``` + +### 配置内容 +- Nginx 反向代理 +- SSL 证书自动安装 +- Gateway WSS 支持 +- Grafana HTTPS 支持 + +--- + +## 2. 日志聚合分析器 + +**文件**: `/opt/moltbot-monitoring/log-analyzer.py` (服务器) + +### 功能 +实时日志聚合、分析和告警 + +### API 端点 +``` +GET /api/logs/summary - 日志摘要 +GET /api/logs/events - 最近事件 +GET /api/logs/health - 健康状态 +GET /api/logs/service?service= - 特定服务日志 +``` + +### 监控的日志 +- Gateway 日志 +- 数据库 API 日志 +- 备份日志 +- 同步日志 +- 健康检查日志 +- 系统认证日志 + +### 使用示例 +```bash +# 获取日志摘要 +curl http://38.14.254.51:9102/api/logs/summary + +# 获取健康状态 +curl http://38.14.254.51:9102/api/logs/health +``` + +--- + +## 3. 性能优化脚本 + +**文件**: `/usr/local/bin/moltbot-optimize.sh` (服务器) + +### 优化项目 +1. **系统参数优化** + - 网络连接数 + - 文件描述符限制 + - 共享内存配置 + - 虚拟内存参数 + +2. **PostgreSQL 优化** + - 内存分配 + - 连接池配置 + - WAL 设置 + - 查询优化 + +3. **Docker 优化** + - 日志大小限制 + - 存储驱动配置 + - 并发下载限制 + +4. **日志清理** + - 自动压缩旧日志 + - 删除过期日志 + - Journal 日志清理 + +5. **磁盘 I/O 优化** + - I/O 调度器设置 + - Swap 配置 + +### 使用方法 +```bash +# SSH 到服务器 +ssh root@38.14.254.51 + +# 运行优化 +/usr/local/bin/moltbot-optimize.sh +``` + +--- + +## 4. 健康检查报告 + +**文件**: `/usr/local/bin/moltbot-health-report.py` (服务器) + +### 功能 +生成每日健康报告并可选发送邮件 + +### 报告内容 +- 系统指标（CPU、内存、磁盘、负载） +- 服务状态 +- 数据库统计 +- 最近问题列表 + +### 使用方法 +```bash +# 生成报告 +/usr/local/bin/moltbot-health-report.py + +# 查看报告 +cat /opt/moltbot-monitoring/reports/report_*.txt +``` + +### 配置自动发送 +编辑 `/opt/moltbot-monitoring/alert-config.json`: +```json +{ + "email": { + "enabled": true, + "smtp_host": "smtp.gmail.com", + "smtp_port": 587, + "smtp_user": "your-email@gmail.com", + "smtp_password": "your-app-password", + "to_email": "admin@example.com" + } +} +``` + +### 配置定时任务 +```bash +# 每天早上 8 点生成报告 +0 8 * * * root /usr/local/bin/moltbot-health-report.py +``` + +--- + +## 5. 集群扩容工具 + +**文件**: `cluster-scaler.bat` + +### 功能 +交互式集群管理界面 + +### 支持操作 +1. 查看集群状态 +2. 添加新设备 +3. 移除设备 +4. 配置负载均衡 +5. 启用故障转移 +6. 生成集群报告 +7. 扩容（增加资源） +8. 缩容（减少资源） + +### 使用方法 +```batch +cluster-scaler.bat +``` + +### 菜单导航 +``` +[1] View current cluster status - 查看集群状态 +[2] Add new device to cluster - 添加设备 +[3] Remove device from cluster - 移除设备 +[4] Configure load balancing - 负载均衡配置 +[5] Enable failover mode - 故障转移配置 +[6] Generate cluster report - 生成集群报告 +[7] Scale up (add resources) - 扩容 +[8] Scale down (remove resources) - 缩容 +``` + +--- + +## 📊 服务端口汇总 + +| 服务 | 端口 | 说明 | +|------|------|------| +| Gateway | 18789 | WebSocket 网关 | +| Database API | 18800 | REST API | +| Prometheus | 9090 | 指标采集 | +| Grafana | 3000 | 可视化 | +| Log Analyzer | 9102 | 日志分析 | +| Metrics Exporter | 9101 | 指标导出 | + +--- + +## 🚀 快速开始 + +### 1. 配置 SSL 加密 +```batch +setup-ssl.bat +``` + +### 2. 运行性能优化 +```bash +ssh root@38.14.254.51 "/usr/local/bin/moltbot-optimize.sh" +``` + +### 3. 添加新设备到集群 +```batch +cluster-scaler.bat +# 选择 [2] Add new device +``` + +### 4. 生成健康报告 +```bash +ssh root@38.14.254.51 "/usr/local/bin/moltbot-health-report.py" +``` + +--- + +## 📝 维护任务 + +### 每日 +- 检查健康报告 +- 监控日志分析器告警 +- 验证备份完成 + +### 每周 +- 审查系统性能 +- 检查磁盘使用 +- 清理旧日志 + +### 每月 +- 审查安全更新 +- 测试备份恢复 +- 性能调优评估 + +--- + +## 🔗 相关文档 + +- `DEPLOYMENT-COMPLETE.md` - 完整部署指南 +- `ROADMAP.md` - 功能路线图 +- `admin-panel.html` - Web 管理面板 + +--- + +## ⚠️ 注意事项 + +1. **SSL 证书** + - 自签名证书会显示浏览器警告 + - Let's Encrypt 需要域名和 80 端口 + +2. **性能优化** + - PostgreSQL 重启会短暂中断服务 + - 建议在维护窗口执行 + +3. **集群扩容** + - 添加设备需要物理访问或远程访问 + - 确保网络连接稳定 + +4. **日志分析** + - 日志文件可能很大 + - 定期清理旧日志 + +--- + +## 🆘 故障排除 + +### SSL 配置失败 +```bash +# 检查 Nginx 配置 +nginx -t + +# 查看 Nginx 日志 +journalctl -u nginx -n 50 +``` + +### 日志分析器无法启动 +```bash +# 检查服务状态 +systemctl status moltbot-log-analyzer + +# 查看日志 +journalctl -u moltbot-log-analyzer -n 50 +``` + +### 性能优化后问题 +```bash +# 恢复默认配置 +sysctl --system + +# 重启 PostgreSQL +systemctl restart postgresql +``` + +--- + +**🎉 扩展功能已就绪，按需使用！** diff --git a/cluster-scaler.bat b/cluster-scaler.bat new file mode 100644 index 000000000..0abb70593 --- /dev/null +++ b/cluster-scaler.bat @@ -0,0 +1,341 @@ +@echo off +chcp 65001 >nul +title Moltbot Cluster Scaling Tool + +set "SERVER=root@38.14.254.51" +set "REPO_URL=https://github.com/flowerjunjie/moltbot.git" + +echo ======================================== +echo Moltbot Cluster Scaling Tool +echo ======================================== +echo. +echo This tool helps you scale your Moltbot cluster. +echo. + +:menu +cls +echo ======================================== +echo Moltbot Cluster Scaling Tool +echo ======================================== +echo. +echo Current Cluster Status: +echo. +echo [1] View current cluster status +echo [2] Add new device to cluster +echo [3] Remove device from cluster +echo [4] Configure load balancing +echo [5] Enable failover mode +echo [6] Generate cluster report +echo [7] Scale up (add resources) +echo [8] Scale down (remove resources) +echo [0] Exit +echo. +set /p choice="Select option (0-8): " + +if "%choice%"=="1" goto view_status +if "%choice%"=="2" goto add_device +if "%choice%"=="3" goto remove_device +if "%choice%"=="4" goto load_balance +if "%choice%"=="5" goto failover +if "%choice%"=="6" goto cluster_report +if "%choice%"=="7" goto scale_up +if "%choice%"=="8" goto scale_down +if "%choice%"=="0" goto end +goto menu + +:view_status +cls +echo ======================================== +echo Cluster Status +echo ======================================== +echo. + +echo Fetching cluster information... +echo. + +ssh %SERVER% "curl -s http://localhost:18800/api/devices | python3 -m json.tool" + +echo. +echo Service Status: +ssh %SERVER% "systemctl is-active moltbot-gateway moltbot-db-api moltbot-metrics-exporter --no-pager" + +echo. +echo Docker Containers: +ssh %SERVER% "docker ps --format 'table {{.Names}}\t{{.Status}}'" + +pause +goto menu + +:add_device +cls +echo ======================================== +echo Add New Device +echo ======================================== +echo. +echo This will prepare a new device for the cluster. +echo. +echo Steps: +echo 1. On the new device, run: notebook-auto-deploy.bat +echo 2. Then run: register-device.bat +echo 3. The device will automatically appear in the cluster +echo. +echo Would you like to: +echo [1] Generate deployment script for new device +echo [2] View existing devices +echo [3] Return to menu +echo. +set /p add_choice="Select option: " +if "%add_choice%"=="1" goto generate_script +if "%add_choice%"=="2" goto view_status +goto menu + +:generate_script +set /p DEVICE_NAME="Enter device name (or press Enter for hostname): " +if "%DEVICE_NAME%"=="" set "DEVICE_NAME=%COMPUTERNAME%" + +set /p DEVICE_TYPE="Enter device type (desktop/notebook/server): " + +echo. +echo Generating deployment script... +echo. + +cat > "deploy-%DEVICE_NAME%.bat" << EOFSHORT +@echo off +echo Deploying %DEVICE_NAME% to cluster... +echo. +git clone %REPO_URL% C:\moltbot +cd C:\moltbot +call notebook-auto-deploy.bat +call register-device.bat +EOFSHORT + +echo Script generated: deploy-%DEVICE_NAME%.bat +echo. +echo Copy this script to the new device and run it. +echo. +pause +goto menu + +:remove_device +cls +echo ======================================== +echo Remove Device +echo ======================================== +echo. +echo WARNING: This will remove a device from the cluster database. +echo The device will no longer be monitored or synced. +echo. +echo To re-add, simply run register-device.bat on the device. +echo. +set /p DEVICE_TO_REMOVE="Enter device name to remove: " +if "%DEVICE_TO_REMOVE%"=="" goto menu + +echo. +echo Removing device from cluster... +ssh %SERVER% "psql -d moltbot -c \"DELETE FROM devices WHERE device_name = '%DEVICE_TO_REMOVE%';\"" + +echo. +echo Device removed from cluster database. +echo. +pause +goto menu + +:load_balance +cls +echo ======================================== +echo Load Balancing Configuration +echo ======================================== +echo. +echo Moltbot Gateway supports multiple gateway instances. +echo. +echo Current architecture: +echo Single Gateway on %SERVER% +echo. +echo Load balancing options: +echo. +echo [1] Add secondary gateway (recommended) +echo [2] Configure DNS round-robin +echo [3] Configure Nginx reverse proxy +echo [4] View current configuration +echo [0] Return to menu +echo. +set /p lb_choice="Select option: " +if "%lb_choice%"=="1" goto add_gateway +if "%lb_choice%"=="4" goto view_lb_config +goto menu + +:add_gateway +echo. +echo To add a secondary gateway: +echo. +echo 1. Deploy Moltbot on another server +echo 2. Configure it with: "mode": "local" +echo 3. Set the same auth token +echo 4. Point clients to both gateways +echo. +echo Example configuration: +echo. +echo { +echo "gateway": { +echo "mode": "local", +echo "bind": "all", +echo "auth": {"token": "moltbot-cluster-2024"} +echo } +echo } +echo. +pause +goto menu + +:view_lb_config +echo. +echo Current Load Balancing Configuration: +echo. +echo Mode: Hybrid (local + remote failover) +echo Primary: Local Gateway +echo Fallback: %SERVER% +echo. +echo To change, edit: ~/.clawdbot/moltbot.json +echo. +pause +goto menu + +:failover +cls +echo ======================================== +echo Failover Configuration +echo ======================================== +echo. +echo Moltbot supports automatic failover. +echo. +echo Current failover status: +echo. +ssh %SERVER% "curl -s http://localhost:18800/api/devices | python3 -c 'import sys, json; devices=json.load(sys.stdin); online=[d for d in devices if d.get(\"status\")==\"online\"]; print(f\"Online devices: {len(online)}\"); print(f\"Total devices: {len(devices)}\")'" + +echo. +echo Failover configuration: +echo - Health check interval: 5 minutes +echo - Auto-switch: Enabled +echo - Session sync: Every 10 minutes +echo. +echo To modify failover settings: +echo Edit: /etc/cron.d/moltbot-tasks +echo. +pause +goto menu + +:cluster_report +cls +echo ======================================== +echo Cluster Report +echo ======================================== +echo. +echo Generating cluster report... +echo. + +ssh %SERVER% "/usr/local/bin/moltbot-health-report.py" + +echo. +echo Full report saved to server. +echo View with: ssh %SERVER% "cat /opt/moltbot-monitoring/reports/report_*.txt" +echo. +pause +goto menu + +:scale_up +cls +echo ======================================== +echo Scale Up - Add Resources +echo ======================================== +echo. +echo Scale up options: +echo. +echo [1] Increase database connections +echo [2] Increase backup retention +echo [3] Enable additional monitoring +echo [4] Add more storage +echo [0] Return to menu +echo. +set /p scale_choice="Select option: " + +if "%scale_choice%"=="1" ( + echo. + echo Increasing database connections... + ssh %SERVER% "psql -d moltbot -c 'ALTER SYSTEM SET max_connections = 200;' && systemctl restart postgresql" + echo Done. + pause +) +if "%scale_choice%"=="2" ( + echo. + echo Current backup retention: 7 days + set /p RETENTION="New retention (days): " + ssh %SERVER% "sed -i 's/RETENTION_DAYS=30/RETENTION_DAYS=%RETENTION%/' /opt/moltbot-backup/backup.sh" + echo Done. + pause +) +if "%scale_choice%"=="3" ( + echo. + echo Enabling additional monitoring... + ssh %SERVER% "systemctl start moltbot-perf-monitor" + echo Done. + pause +) +if "%scale_choice%"=="4" ( + echo. + echo To add more storage: + echo 1. Add disk to server + echo 2. Create new partition + echo 3. Add to LVM or mount point + echo 4. Update backup paths + echo. + pause +) + +goto menu + +:scale_down +cls +echo ======================================== +echo Scale Down - Remove Resources +echo ======================================== +echo. +echo WARNING: Scaling down may reduce performance. +echo. +echo Scale down options: +echo. +echo [1] Decrease database connections +echo [2] Decrease backup retention +echo [3] Disable performance monitor +echo [0] Return to menu +echo. +set /p down_choice="Select option: " + +if "%down_choice%"=="1" ( + echo. + echo Decreasing database connections... + ssh %SERVER% "psql -d moltbot -c 'ALTER SYSTEM SET max_connections = 100;' && systemctl restart postgresql" + echo Done. + pause +) +if "%down_choice%"=="2" ( + echo. + echo Current backup retention: 30 days + set /p RETENTION="New retention (days): " + ssh %SERVER% "sed -i 's/RETENTION_DAYS=30/RETENTION_DAYS=%RETENTION%/' /opt/moltbot-backup/backup.sh" + echo Done. + pause +) +if "%down_choice%"=="3" ( + echo. + echo Disabling performance monitor... + ssh %SERVER% "systemctl stop moltbot-perf-monitor" + echo Done. + pause +) + +goto menu + +:end +echo. +echo Thank you for using Moltbot Cluster Scaling Tool! +echo. +exit /b 0 diff --git a/setup-ssl.bat b/setup-ssl.bat new file mode 100644 index 000000000..ac5e6f64d --- /dev/null +++ b/setup-ssl.bat @@ -0,0 +1,179 @@ +@echo off +chcp 65001 >nul +title Moltbot SSL/TLS Configuration Wizard + +set "SERVER=root@38.14.254.51" +set "DOMAIN=38.14.254.51" + +echo ======================================== +echo Moltbot SSL/TLS Configuration Wizard +echo ======================================== +echo. +echo This wizard will help you configure SSL/TLS encryption +echo for the Moltbot Gateway and monitoring services. +echo. +echo Options: +echo. +echo [1] Use self-signed certificate (free, quick) +echo [2] Use Let's Encrypt (free, requires domain) +echo [3] Use existing certificate +echo [4] Skip SSL configuration +echo. + +choice /C 1234 /N /M "Select option (1-4)" +if errorlevel 4 goto end +if errorlevel 3 goto existing_cert +if errorlevel 2 goto letsencrypt +if errorlevel 1 goto self_signed + +:self_signed +echo. +echo ======================================== +echo Generating Self-Signed Certificate +echo ======================================== +echo. + +ssh %SERVER% "mkdir -p /opt/moltbot-ssl && cd /opt/moltbot-ssl && openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout moltbot.key -out moltbot.crt -subj '/C=CN/ST=State/L=City/O=Moltbot/CN=%DOMAIN%'" + +if errorlevel 1 ( + echo ERROR: Failed to generate certificate + pause + goto end +) + +echo. +echo Certificate generated successfully! +echo. +echo Configuring services to use SSL... + +goto configure_services + +:letsencrypt +echo. +echo ======================================== +echo Let's Encrypt Certificate +echo ======================================== +echo. +echo To use Let's Encrypt, you need: +echo 1. A domain name pointing to %SERVER% +echo 2. Port 80 open for HTTP verification +echo. +set /p DOMAIN="Enter your domain name: " + +if "%DOMAIN%"=="" ( + echo ERROR: Domain name is required + pause + goto end +) + +echo. +echo Installing certbot... +ssh %SERVER% "apt-get install -y certbot" + +echo. +echo Obtaining certificate... +ssh %SERVER% "certbot certonly --standalone -d %DOMAIN% --email admin@%DOMAIN% --agree-tos --non-interactive" + +if errorlevel 1 ( + echo ERROR: Failed to obtain certificate + pause + goto end +) + +echo. +echo Certificate obtained successfully! +echo. +echo Copying certificate to Moltbot directory... +ssh %SERVER% "mkdir -p /opt/moltbot-ssl && cp /etc/letsencrypt/live/%DOMAIN%/fullchain.pem /opt/moltbot-ssl/moltbot.crt && cp /etc/letsencrypt/live/%DOMAIN%/privkey.pem /opt/moltbot-ssl/moltbot.key" + +goto configure_services + +:existing_cert +echo. +echo ======================================== +echo Using Existing Certificate +echo ======================================== +echo. +echo Please provide the paths to your certificate files. +echo. +set /p CERT_PATH="Certificate file (.crt or .pem): " +set /p KEY_PATH="Private key file (.key): " + +if "%CERT_PATH%"=="" goto end +if "%KEY_PATH%"=="" goto end + +echo. +echo Copying certificate to server... +scp "%CERT_PATH%" %SERVER%:/opt/moltbot-ssl/moltbot.crt +scp "%KEY_PATH%" %SERVER%:/opt/moltbot-ssl/moltbot.key + +goto configure_services + +:configure_services +echo. +echo ======================================== +echo Configuring Services +echo ======================================== +echo. + +echo Setting up SSL for Gateway... +ssh %SERVER% "cat > /etc/nginx/sites-available/moltbot-gateway << 'NGINX' +server { + listen 80; + server_name %DOMAIN%; + return 301 https://\$server_name\$request_uri; +} + +server { + listen 443 ssl http2; + server_name %DOMAIN%; + + ssl_certificate /opt/moltbot-ssl/moltbot.crt; + ssl_certificate_key /opt/moltbot-ssl/moltbot.key; + + ssl_protocols TLSv1.2 TLSv1.3; + ssl_ciphers HIGH:!aNULL:!MD5; + ssl_prefer_server_ciphers on; + + location / { + proxy_pass http://localhost:18789; + proxy_http_version 1.1; + proxy_set_header Upgrade \$http_upgrade; + proxy_set_header Connection 'upgrade'; + proxy_set_header Host \$host; + proxy_cache_bypass \$http_upgrade; + proxy_set_header X-Real-IP \$remote_addr; + proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto \$scheme; + } +} +NGINX +" + +echo Installing and configuring Nginx... +ssh %SERVER% "apt-get install -y nginx && ln -sf /etc/nginx/sites-available/moltbot-gateway /etc/nginx/sites-enabled/ && nginx -t && systemctl reload nginx" + +echo. +echo Configuring Grafana for SSL... +ssh %SERVER% "docker run --rm -v moltbot-monitoring_grafana-data:/data busybox sh -c 'echo \"\\n[server]\\n protocol = https\\n cert_file = /etc/grafana/grafana.crt\\n cert_key = /etc/grafana/grafana.key\" >> /data/grafana.ini'" + +echo Copying SSL certificate for Grafana... +ssh %SERVER% "docker cp /opt/moltbot-ssl/moltbot.crt moltbot-grafana:/etc/grafana/grafana.crt && docker cp /opt/moltbot-ssl/moltbot.key moltbot-grafana:/etc/grafana/grafana.key && docker restart moltbot-grafana" + +echo. +echo ======================================== +echo SSL Configuration Complete! +echo ======================================== +echo. +echo Your services are now accessible via HTTPS: +echo. +echo Gateway: wss://%DOMAIN%/ +echo Grafana: https://%DOMAIN%:3000 +echo Prometheus: https://%DOMAIN%:9090 +echo. +echo Note: If using self-signed certificate, you will see +echo a browser warning. This is normal and safe to ignore. +echo. + +:end +pause