Sensorix is a full-stack IoT monitoring system that collects, processes, and visualizes real-time sensor data for traffic, air pollution, and street lighting.
The system consists of three containerized components:
- Backend — Java Spring Boot (port 8080)
- Frontend — Angular 21 served via Nginx (port 80)
- Database — MySQL 8.0 (port 3306)
All three components are orchestrated via Docker Compose and deployed automatically through a Jenkins CI/CD pipeline.
This repo is one of three sibling repositories. Clone all three at the same level:
parent/
├── iot-devops/ ← this repo — Dockerfiles, Compose, Jenkins, scripts
├── iot-backend/ ← Java Spring Boot source + Dockerfile
└── iot-frontend/ ← Angular source + Dockerfile
git clone https://github.com/faridakhaled05/iot-devops.git
git clone https://github.com/Yosra-01/iot-backend.git
git clone https://github.com/SalmaaKhaledd/iot-frontend.git| File | Purpose |
|---|---|
README.md |
Full setup and recreation instructions |
Dockerfile.db |
MySQL 8.0 custom image with database name baked in |
Dockerfile.jenkins |
Custom Jenkins image with Docker, Maven, Node, Compose installed |
docker-compose.yml |
Orchestrates all three containers with secrets and healthchecks |
docker-entrypoint.sh |
Backend entrypoint — reads Docker secret files and exports as env vars |
build-and-push.sh |
Manual script to build and push all three images to Docker Hub |
Jenkinsfile |
CI/CD pipeline definition — all stages from build to deploy |
.env.example |
Template for required environment variables — copy to .env and fill in |
- Docker Engine 20.10+ or Docker Desktop 4.x+ — Install Docker
- Git
- A Docker Hub account — hub.docker.com
- Java 21 and Maven (only needed if running backend locally outside Docker)
- Node.js 20 (only needed if running frontend locally outside Docker)
This setup was originally developed on Apple Silicon. There are two things in the configuration that are Mac-specific. If you are on Linux or Windows, read this section before proceeding.
The Dockerfile.jenkins downloads the ARM64 build of Docker Compose:
RUN curl -L "https://github.com/docker/compose/releases/download/v2.27.0/docker-compose-linux-aarch64" \
-o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-composeIf you are on Linux or Windows (x86_64): replace aarch64 with x86_64:
RUN curl -L "https://github.com/docker/compose/releases/download/v2.27.0/docker-compose-linux-x86_64" \
-o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-composeUsing the wrong architecture binary causes exec format error at runtime.
The Jenkinsfile uses a custom buildx builder named native and forces --platform linux/amd64.
This was necessary on Mac because Docker Desktop's default builder on Apple Silicon produces
multi-platform manifest lists that cannot be pushed to Docker Hub as standard images.
If you are on Linux or Windows: you likely do not need buildx at all. You can simplify
the Build & Push stages in the Jenkinsfile to use plain docker build:
// Replace this (Mac-specific):
sh '''
docker buildx build \
--builder $BUILDER \
--platform linux/amd64 \
--load \
-t $BACKEND_IMAGE \
-f iot-backend/Dockerfile \
iot-backend/
docker push $BACKEND_IMAGE
'''
// With this (Linux/Windows):
sh '''
docker build \
-t $BACKEND_IMAGE \
-f iot-backend/Dockerfile \
iot-backend/
docker push $BACKEND_IMAGE
'''Also remove or skip Step 7 (Create the buildx Builder) in Part 2 below.
You can also remove the BUILDER = 'native' line from the environment block
in the Jenkinsfile since it will no longer be referenced.
Use this path to verify the stack works before setting up CI/CD.
Inside iot-devops/, create a .env file. This file is gitignored — never commit it.
cd iot-devops
cp .env.example .envFill in the values with your own Docker Hub username and credentials:
DOCKER_USERNAME=your_dockerhub_username
DOCKER_PASSWORD=your_dockerhub_password
IMAGE_TAG=v3.0
SECRETS_PATH=./secretsDocker Compose reads secrets from files, not plain environment variables. Create the secrets directory and files manually:
mkdir -p secrets
printf '%s' 'your_mysql_root_password' > secrets/db_password.txt
printf '%s' 'your_jwt_secret_minimum_32_characters' > secrets/jwt_secret.txt
⚠️ Thesecrets/directory is gitignored. Never commit secret files. Useprintf '%s'rather thanechoto avoid writing a trailing newline into the secret file — a trailing newline causes silent auth failures.
Run the provided script from inside iot-devops/:
chmod +x build-and-push.sh
./build-and-push.shThis script:
- Validates that
.envexists and all required variables are set - Validates that
iot-backend/andiot-frontend/sibling directories exist - Logs into Docker Hub using your credentials from
.env - Builds and pushes
iot-backend:v3.0,iot-frontend:v3.0,iot-db:v3.0
To build images manually without the script (run from the parent directory):
docker build -f iot-backend/Dockerfile -t your_dockerhub_username/iot-backend:v3.0 iot-backend/
docker build -f iot-frontend/Dockerfile -t your_dockerhub_username/iot-frontend:v3.0 iot-frontend/
docker build -f iot-devops/Dockerfile.db -t your_dockerhub_username/iot-db:v3.0 iot-devops/From inside iot-devops/:
DOCKER_USERNAME=your_dockerhub_username \
IMAGE_TAG=v3.0 \
SECRETS_PATH=./secrets \
docker-compose up -dDocker Compose starts containers in this order automatically:
db— waits until MySQL healthcheck passes (can take 20–30 seconds)backend— starts only after db reports healthyfrontend— starts after backend
Check all three containers are running:
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"Expected output — all three showing Up, db showing (healthy):
iot-devops-db-1 Up 2 minutes (healthy) 0.0.0.0:3306->3306/tcp
iot-devops-backend-1 Up 1 minute 0.0.0.0:8080->8080/tcp
iot-devops-frontend-1 Up 1 minute 0.0.0.0:80->80/tcp
Verify the backend started correctly:
docker logs iot-devops-backend-1 --tail 20Look for:
Started [ApplicationName] in X.XXX seconds
Verify the backend API is responding:
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:8080/api/auth/login \
-H "Content-Type: application/json" \
-d '{}'Expected: HTTP 400 — server is up and input validation is working.
Open the frontend in your browser:
http://localhost
docker-compose downTo also remove the database volume (wipes all data):
docker-compose down -vThis is the automated path. The pipeline builds, tests, pushes, and deploys the full stack automatically.
Before building, check the Docker Compose binary URL in Dockerfile.jenkins
and make sure it matches your CPU architecture:
- Apple Silicon (Mac M1/M2/M3): use
docker-compose-linux-aarch64← already in the file - Linux / Windows x86_64: change to
docker-compose-linux-x86_64
See the Apple Silicon section at the top of this README for the exact line to change.
From inside iot-devops/:
docker build -f Dockerfile.jenkins -t sensorix-jenkins .This builds a custom Jenkins image that includes:
- Docker CLI (to run docker commands inside pipelines)
- Docker Compose v2.27.0 (architecture-specific binary)
- Maven (to build and test the Java backend)
- Node.js 20 + npm (to build and test the Angular frontend)
This directory is mounted into the Jenkins container so that Docker Compose can access secret files written by Jenkins during the Deploy stage.
mkdir -p /tmp/jenkins-workspace
⚠️ On Linux and Windows,/tmpmay or may not be cleared on reboot depending on your OS configuration. If Jenkins loses its workspace directory, recreate it with the same command and restart the Jenkins container.
docker run -d \
--name sensorix-jenkins \
-p 9090:8080 \
-p 50000:50000 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v jenkins_home:/var/jenkins_home \
-v /tmp/jenkins-workspace:/var/jenkins_home/workspace \
sensorix-jenkinsFlag explanation:
-p 9090:8080— Jenkins UI is atlocalhost:9090(internal Jenkins port is 8080)-p 50000:50000— Jenkins agent port for connecting future build agents-v /var/run/docker.sock:/var/run/docker.sock— mounts host Docker socket so Jenkins can run docker commands via the host daemon-v jenkins_home:/var/jenkins_home— named volume that persists Jenkins config, credentials, plugins, and job history across container restarts-v /tmp/jenkins-workspace:/var/jenkins_home/workspace— exposes the Jenkins workspace to the host filesystem so docker-compose can resolve secret file paths
docker exec -u root sensorix-jenkins chmod 666 /var/run/docker.sockWithout this, Jenkins gets permission denied when trying to run docker commands.
On Linux this is typically needed once after container creation. On Mac with Docker Desktop, the socket is recreated on every Docker restart so this command must be re-run after every restart.
docker exec sensorix-jenkins cat /var/jenkins_home/secrets/initialAdminPassword- Open
http://localhost:9090in your browser - Paste the initial admin password
- Select Install suggested plugins and wait for installation to complete
- Create your admin user
- Accept the default Jenkins URL (
http://localhost:9090/)
Linux/Windows users: skip this step entirely and update the Jenkinsfile as described in the Apple Silicon section at the top of this README.
On Mac, Jenkins uses a custom buildx builder named native to produce
images that can be pushed to Docker Hub. Create it once on the host:
docker buildx create --name native --driver docker-container --use
docker buildx inspect native --bootstrapVerify it was created:
docker buildx lsYou should see native listed with driver docker-container.
Jenkins needs to know the host-side path of the workspace so docker-compose can resolve secret file paths correctly.
- Go to Manage Jenkins → System
- Scroll to Global properties
- Check Environment variables
- Click Add and enter:
- Name:
HOST_WORKSPACE_ROOT - Value:
/tmp/jenkins-workspace
- Name:
- Click Save
Go to Manage Jenkins → Credentials → System → Global credentials → Add Credential.
Add these three credentials. The IDs must match exactly — the Jenkinsfile references them by ID.
Docker Hub credentials:
- Kind:
Username with password - Username: your Docker Hub username
- Password: your Docker Hub password
- ID:
dockerhub-credentials
Database password:
- Kind:
Secret text - Secret: your MySQL root password (choose any strong password)
- ID:
db-password
JWT secret:
- Kind:
Secret text - Secret: your JWT signing secret (minimum 32 characters)
- ID:
jwt-secret
Open Jenkinsfile in iot-devops/ and set your Docker Hub username:
environment {
DOCKERHUB_USER = 'your_dockerhub_username' ← change this
IMAGE_TAG = 'v3.0'
BUILDER = 'native'
...
}Commit and push the change to the iot-devops repo before creating the pipeline job.
- Click New Item
- Enter name:
iot-devops-pipeline - Select Pipeline and click OK
- Under Pipeline, set Definition to
Pipeline script from SCM - Set SCM to
Git - Repository URL:
https://github.com/faridakhaled05/iot-devops.git - Branch:
*/main - Script Path:
Jenkinsfile - Click Save
Click Build Now on the pipeline page.
The pipeline runs these stages in order:
| Stage | What It Does |
|---|---|
| Checkout Backend | Clones iot-backend repo into workspace |
| Checkout Frontend | Clones iot-frontend repo into workspace |
| Test (parallel) | Runs mvn test and npm test simultaneously |
| Build & Push Backend | Builds backend image, pushes to Docker Hub |
| Build & Push Frontend | Builds frontend image, pushes to Docker Hub |
| Deploy | Writes secret files, runs docker-compose up --force-recreate |
| Smoke Test | Verifies all containers are running and backend started |
After the pipeline goes green:
# All three containers should be Up
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
# Backend log should contain "Started"
docker logs iot-devops-backend-1 --tail 20
# Verify backend API
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:8080/api/auth/login \
-H "Content-Type: application/json" -d '{}'Open the frontend: http://localhost
The Docker socket permissions reset when Docker restarts. Run this before running the pipeline again:
docker exec -u root sensorix-jenkins chmod 666 /var/run/docker.sockIf the Jenkins container itself was stopped, start it first:
docker start sensorix-jenkins
docker exec -u root sensorix-jenkins chmod 666 /var/run/docker.sockOn Mac, also recreate the workspace directory if needed:
mkdir -p /tmp/jenkins-workspaceImages are pushed to Docker Hub under the account configured in .env and the Jenkinsfile.
Each team member uses their own Docker Hub account. The image names follow this pattern:
your_dockerhub_username/iot-backend:v3.0
your_dockerhub_username/iot-frontend:v3.0
your_dockerhub_username/iot-db:v3.0
To pull and run without building from source:
mkdir -p secrets
printf '%s' 'your_mysql_password' > secrets/db_password.txt
printf '%s' 'your_jwt_secret_32chars_minimum' > secrets/jwt_secret.txt
DOCKER_USERNAME=your_dockerhub_username \
IMAGE_TAG=v3.0 \
SECRETS_PATH=./secrets \
docker-compose up -dBackend container exits immediately:
docker logs iot-devops-backend-1Most common cause: database not ready, or secret files empty/missing. Check secret files:
cat secrets/db_password.txt
cat secrets/jwt_secret.txtpermission denied on Docker socket in Jenkins:
docker exec -u root sensorix-jenkins chmod 666 /var/run/docker.sockexec format error for docker-compose inside Jenkins:
Wrong CPU architecture binary in Dockerfile.jenkins. Rebuild the Jenkins image
after correcting the URL — use aarch64 for Mac, x86_64 for Linux/Windows.
docker-compose: command not found inside Jenkins:
The compose download failed during image build. Rebuild Jenkins image and check
the build output for curl errors.
Deploy stage fails — secret file not found:
HOST_WORKSPACE_ROOT is not set or incorrect. Verify it is set to /tmp/jenkins-workspace
in Manage Jenkins → System → Global properties → Environment variables.
Also verify the directory exists on the host:
ls /tmp/jenkins-workspacebuildx builder native not found (Mac only):
docker buildx create --name native --driver docker-container --use
docker buildx inspect native --bootstrapContainers start but frontend shows blank page or API errors:
docker logs iot-devops-backend-1 --tail 50
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:8080/api/auth/login \
-H "Content-Type: application/json" -d '{}'Expected: HTTP 400. Anything else means the backend did not start correctly.
This path runs the full stack on a local Kubernetes cluster via minikube, instead of
Docker Compose. All manifests referenced below already exist in this repo under k8s/.
This section assumes the following are already complete. If any of these aren't true yet, go do them first — Part 3 will not work otherwise.
- Part 1 has been completed at least once — specifically, the three images
(
iot-backend,iot-frontend,iot-db) have been built and pushed to your Docker Hub account under a tag (e.g.v4.0). Kubernetes pulls images from Docker Hub (or loads them from your local Docker cache into minikube) — it does not build them itself. - The sibling directory structure exists, i.e.
iot-devops,iot-backend, andiot-frontendare all cloned at the same level under a common parent (see Repository Structure above). Referred to below as~/sensorix/for consistency, but any parent folder name works as long as all three repos sit alongside each other. - All commands below are run from inside
iot-devops/(e.g.~/sensorix/iot-devops), since that's where thek8s/directory lives.cdthere first if you're not already. - Docker is installed and working (already a prerequisite above).
- If working on WSL2/Windows: all commands run inside the WSL Linux shell, never CMD
or PowerShell —
kubectl,minikube, anddockerin this setup are installed inside WSL and are not reachable from a native Windows terminal.
kubectlminikube- Docker (already required above)
The following commands are for Linux OS. If you're using a different OS, check kubernetes guide for the installation
# kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
# minikube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikubeVerify both:
kubectl version --client
minikube versionminikube start --driver=dockerThis starts a single-node cluster using your existing Docker installation. First run downloads a base image and may take a few minutes.
Verify:
minikube status
kubectl get nodesYou should see one node with status Ready.
The K8s Secret holding the database password and JWT secret is gitignored — only a template is committed. Copy the template and fill in real values:
cp k8s/secret.yaml.example k8s/secret.yaml
nano k8s/secret.yamlReplace the placeholder values with your actual MySQL root password and JWT secret —
ideally the same values already in secrets/db_password.txt and secrets/jwt_secret.txt,
so Compose and Kubernetes use identical credentials.
If you've already built and pushed the images per Part 1, load them directly from your local Docker cache into minikube (faster and more reliable than pulling from Docker Hub again):
minikube image load your_dockerhub_username/iot-backend:v4.0
minikube image load your_dockerhub_username/iot-frontend:v4.0
minikube image load your_dockerhub_username/iot-db:v4.0If your image names in
k8s/*.yamlreference a different Docker Hub username, update theimage:field ink8s/backend-deployment.yaml,k8s/frontend-deployment.yaml, andk8s/db-deployment.yamlaccordingly before applying.
Apply everything in the k8s/ directory:
kubectl apply -f k8s/This creates, in dependency order:
- The Secret (
sensorix-secrets) - The database PersistentVolumeClaim, Deployment, and Service
- The backend Deployment and Service (
iot-backend-svc) - The frontend Deployment and Service
kubectl get pods
kubectl get svc
kubectl get pvcAll Pods should reach 1/1 Ready / Running. The database PVC should show Bound.
Check backend startup:
kubectl logs -l app=iot-backend-svc --tail 30Look for Started IotmonitorApplication with no errors following it.
The frontend Service is exposed via NodePort. Get the reachable URL:
minikube service frontend --urlOpen the printed URL in your browser — this serves the full application, with the frontend proxying API calls through to the backend and database running inside the cluster.
This demonstrates zero-downtime deployment — a new version replaces the old one without
any dropped requests, governed by the RollingUpdate strategy in k8s/backend-deployment.yaml
(maxUnavailable: 1, maxSurge: 1).
Create a new tag to roll out. No code change is required to demonstrate the mechanism — retagging an existing image is sufficient, since Kubernetes reacts to the tag/template changing, not to what's actually inside the image:
docker tag your_dockerhub_username/iot-backend:v4.0 your_dockerhub_username/iot-backend:v4.1
docker push your_dockerhub_username/iot-backend:v4.1
minikube image load your_dockerhub_username/iot-backend:v4.1Update the manifest — open k8s/backend-deployment.yaml and change the image: line
to the new tag (v4.1).
Apply and watch it live. In one terminal, start watching before applying:
kubectl get pods -wIn a second terminal:
kubectl apply -f k8s/backend-deployment.yamlYou should see a new Pod (different ReplicaSet hash) come up, pass its readiness probe, and only then does an old Pod begin terminating — repeating until both replicas are on the new version. At no point does the Pod count drop below 1 or exceed 3.
Confirm the rollout completed cleanly:
kubectl rollout status deployment/iot-backend-svcTo undo a rollout (roll back to the previous version):
kubectl rollout undo deployment/iot-backend-svc
kubectl rollout status deployment/iot-backend-svcThis demonstrates independently scaling one stateless service without affecting the rest of the stack.
Scale the backend up:
kubectl scale deployment/iot-backend-svc --replicas=4
kubectl get pods
kubectl get deployment iot-backend-svcYou should see two additional Pods created on the same ReplicaSet (same image, same
config — just more copies), and the Deployment showing 4/4 ready.
Scale back down to steady state:
kubectl scale deployment/iot-backend-svc --replicas=2
kubectl get podsNote on scaling the database:
dbis intentionally kept atreplicas: 1and should not be scaled up. Its PersistentVolumeClaim usesReadWriteOncePod, which Kubernetes enforces at the scheduler level — a second Pod attempting to use the same volume will be rejected and stuck inPending(confirm viakubectl describe pod <pending-db-pod>if ever tested). This is expected and correct: a single MySQL instance without configured replication cannot safely run as multiple writers against the same data.
To remove all Kubernetes objects (keeps the database's persistent volume claim by default):
kubectl delete -f k8s/To also delete the persisted database volume (wipes all data):
kubectl delete pvc db-pvcTo stop the entire minikube cluster:
minikube stopTo delete it completely (removes the cluster, not just stops it):
minikube delete- Never commit
.env,secrets/, or any file containing passwords or tokens .env,secrets/, andk8s/secret.yamlare all listed in.gitignore- Jenkins credentials are encrypted at rest using the master key stored in
jenkins_home - Secret values are masked as
****in all Jenkins build logs - Docker Compose secrets are mounted as read-only files at
/run/secrets/inside containers — never visible viadocker inspect - Kubernetes Secrets are base64-encoded and access-controlled via RBAC — not encrypted at rest by default
- The backend runs as a non-root
springuser inside its container