BucketBackup is a high-performance, intelligent orchestration platform designed to secure and synchronize your data across the world's leading cloud storage providers. Built for reliability, it ensures your enterprise data is always protected, versioned, and recoverable.
BucketBackup isn't just a sync tool; it's a complete data management ecosystem.
- 🛡️ Multi-Cloud Synergy: Seamlessly bridge AWS S3, Google Cloud Storage, and Azure Blob Storage.
- ⚡ Real-Time Sync: High-concurrency engine for instantaneous data replication.
- 🤖 AI-Powered Monitoring: Integrated anomaly detection to identify potential data corruption or security threats.
- 🔄 One-Click Recovery: Intuitive disaster recovery workflows to restore business continuity in minutes.
- 🔐 Zero-Trust Security: End-to-end AES-256 encryption with granular Role-Based Access Control (RBAC).
- 📊 Live Analytics: A stunning executive dashboard providing real-time visibility into your global storage footprint.
BucketBackup uses a decoupled, microservices-ready architecture designed for horizontal scalability.
graph TD
User((User)) -->|HTTPS| Dashboard[Next.js Dashboard]
Dashboard -->|API| API[Node.js / Express API]
API -->|Jobs| Scheduler[Cron / Event Scheduler]
Scheduler -->|Task| SyncEngine[Sync & Recovery Engine]
subgraph "Storage Layer"
SyncEngine -->|S3 Protocol| AWS[AWS S3]
SyncEngine -->|GCS API| GCP[Google Cloud Storage]
SyncEngine -->|Blob API| Azure[Azure Blob Storage]
end
API -->|Query/Command| DB[(PostgreSQL / Prisma)]
SyncEngine -->|Logs/Status| API
| Layer | Technology | Purpose |
|---|---|---|
| Frontend | Next.js 14, Tailwind CSS, Lucide Icons | Modern, responsive dashboard UI |
| Backend | Node.js, TypeScript, Express | High-performance API and business logic |
| Database | PostgreSQL, Prisma ORM | Relational data and schema management |
| Infrastructure | Terraform, HCL | Multi-cloud resource provisioning |
| Orchestration | Kubernetes, Docker, Helm | Containerized deployment and scaling |
| Cloud | AWS, GCP, Azure | Distributed object storage providers |
- Node.js (v20+) and npm
- Docker & Kubernetes (with Ingress NGINX enabled for cluster deployments)
- Terraform (v1.5+) (for multi-cloud bucket provisioning)
- PostgreSQL database instance (or run via Kubernetes manifests)
Create a .env file in the server folder with the following variables:
PORT=4000
DATABASE_URL="postgresql://backupuser:supersecurepassword@localhost:5432/bucketbackup?schema=public"
JWT_SECRET="generate-a-secure-jwt-random-token-secret-key"
BACKEND_ENCRYPTION_KEY="your-aes-256-bit-key-for-credentials-at-rest"
SLACK_WEBHOOK_URL="https://hooks.slack.com/services/..."Create a .env.local file in the client folder:
NEXT_PUBLIC_API_URL="http://localhost:4000/api"To sync data across AWS, GCP, and Azure, you must provision access keys with the following least-privilege permissions:
- Policy Action:
s3:ListBucket,s3:GetObject,s3:PutObject,s3:DeleteObject - Onboarding Fields: Bucket Name, Region, Access Key ID, Secret Access Key.
- IAM Role:
Storage Object Admin - Setup: Create a Service Account, generate a JSON key, and copy the entire JSON content into the onboarding key field.
- Onboarding Fields: Container Name, Storage Account Connection String.
- Format:
DefaultEndpointsProtocol=https;AccountName=<name>;AccountKey=<key>;EndpointSuffix=core.windows.net
Apply the Prisma schemas and generate local Prisma Clients:
cd server
npm install
npx prisma generatenpm run devcd ../client
npm install
npm run devVisit http://localhost:3000 to open the dashboard interface.
Run the complete Jest test suite (unit tests and integration tests) using the package runner:
cd server
npm run testProvision the necessary S3, GCS, and Azure Blob storage structures on your active accounts:
cd terraform
terraform init
terraform plan
terraform apply -auto-approveTo deploy the production-ready stack (PostgreSQL state, api server, client server, HPA scaling policies, Ingress rules) to your cluster:
- Build Container Images:
docker build -t bucketbackup-server:latest ./server docker build -t bucketbackup-client:latest ./client
- Apply Kubernetes Manifests:
kubectl apply -f k8s/
- Verify Deployment Services:
kubectl get pods -w kubectl get ingress
All endpoints (except Authentication and Health check) require a Bearer <token> authentication header.
POST /api/auth/register- Create user account and organization workspace.POST /api/auth/login- Sign in operator. Returns JWT bearer token.GET /api/auth/me- Fetch details on active operator profile.
POST /api/storage- Register a new cloud storage target (AWS/GCP/Azure).GET /api/storage- List safe-masked configurations for organization.POST /api/storage/:id/verify- Trigger connection check to verify cloud keys.GET /api/storage/:id/inventory- Fetch list of files and folders inside bucket.
POST /api/backups- Create a sync job schedule (cron, one-way/bidirectional, source, destination).GET /api/backups- List backup job schedules.PUT /api/backups/:id- Edit parameters of a job rule.DELETE /api/backups/:id- Delete job rule and unschedule cron tasks.POST /api/backups/:id/trigger- Force immediate manual backup run.POST /api/backups/:id/pause- Disable scheduler for a job.POST /api/backups/:id/resume- Re-enable cron scheduling.GET /api/backups/:id/logs- Retrieve execution logs for job.
POST /api/history/restores- Initiate point-in-time file recovery.GET /api/history/restores- List DR restore runs.GET /api/history/restores/:id- Check progress of a restore job.
GET /api/monitoring/audits- Fetch administrative action audit logs.GET /api/monitoring/alerts- List active data anomalies or duration spikes.POST /api/monitoring/alerts/:id/resolve- Dismiss warning alerts.
GET /api/system/health- Liveness probe endpoint.GET /api/system/metrics- Fetch real-time host CPU, memory, and database counters.
Distributed under the MIT License. See LICENSE for more information.