Real-time Sensor Monitoring with Kubernetes, WebSockets & Machine Learning
Modern IoT Platform for Real-time Sensor Data Processing with Enterprise-Grade Observability
- โจ Features
- ๐๏ธ System Architecture
- ๐ Quick Start
- ๐ ๏ธ Tech Stack
- ๐ Project Structure
- ๐ API Endpoints
- ๐ Live Dashboard
- ๐ Monitoring & Observability
- ๐ฎ Roadmap
- ๐จโ๐ป Author
| ๐ฏ Frontend | ๐ง Backend | ๐ Database | ๐ก๏ธ Infrastructure |
|---|---|---|---|
| Angular 17 + TypeScript | FastAPI + Python | PostgreSQL 13 | Kubernetes + Istio |
| Real-time WebSocket Dashboard | RESTful API | Time-series Data | Docker Containers |
| Responsive Material UI | WebSocket Server | Automated Backups | Service Mesh |
| Live Data Visualization | Sensor Data Processing | High Performance | GitOps (ArgoCD) |
| ๐ก Real-time Features | ๐ Monitoring | ๐ DevOps |
|---|---|---|
| Live Sensor Data Stream | Grafana Dashboards | CI/CD Pipeline |
| WebSocket Connections | Prometheus Metrics | Helm Charts |
| Instant Data Persistence | Kiali Service Mesh | Health Checks |
| Auto-scaling | Jaeger Distributed Tracing | Rolling Updates |
graph TB
A[๐ Web Browser] --> B[๐ฅ๏ธ Frontend<br/>Angular]
B --> C[๐ WebSocket<br/>Real-time]
B --> D[๐ก REST API<br/>HTTP]
C --> E[๐ Backend<br/>FastAPI]
D --> E
E --> F[๐๏ธ PostgreSQL<br/>Sensor Data]
E --> G[๐ Prometheus<br/>Metrics]
H[๐ฑ IoT Sensors] --> I[๐ก Message Queue]
I --> E
E --> J[๐ Istio<br/>Service Mesh]
J --> K[๐ Grafana<br/>Monitoring]
J --> L[๐บ๏ธ Kiali<br/>Visualization]
M[โ ArgoCD] --> N[โธ๏ธ Kubernetes<br/>Orchestration]
style A fill:#ff6b6b
style B fill:#4ecdc4
style E fill:#45b7d1
style F fill:#96ceb4
style K fill:#feca57
style L fill:#ff9ff3
- Docker & Docker Compose
- Minikube (for local Kubernetes)
- kubectl (Kubernetes CLI)
# Clone the repository
git clone https://github.com/pashitox/skysense-iot-platform.git
cd skysense-iot-platform
# Start with Docker Compose (Development)
docker-compose up -d
# Or deploy to Kubernetes (Production)
./deploy-to-k8s.sh# Create namespace and deploy
kubectl create namespace skysense
kubectl apply -f k8s/ -n skysense
# Check deployment status
kubectl get all -n skysense
# Access the application
echo "๐ Frontend: http://192.168.49.2:32323"
echo "๐ง Backend API: http://192.168.49.2:30080"
echo "๐ API Docs: http://192.168.49.2:30080/docs"# Database Configuration
POSTGRES_DB=skysense_db
POSTGRES_USER=skysense_user
POSTGRES_PASSWORD=secure_password
# Backend Configuration
DATABASE_URL=postgresql://user:password@postgresql:5432/skysense_db
WEB_CONCURRENCY=4
# Frontend Configuration
API_URL=http://localhost:30080/api
WS_URL=ws://localhost:30080/ws- Angular 17 - Modern TypeScript framework
- WebSocket API - Real-time data streaming
- Material Design - Professional UI components
- RxJS - Reactive programming library
- FastAPI - High-performance Python API
- WebSocket - Bi-directional real-time communication
- SQLAlchemy - Database ORM
- Uvicorn - ASGI web server
- PostgreSQL 13 - Relational database
- Time-series Data - Optimized for sensor data
- Automated Backups - Data persistence
- Kubernetes - Container orchestration
- Istio Service Mesh - Traffic management & security
- Docker - Containerization
- Prometheus/Grafana - Monitoring stack
skysense-iot-platform/
โโโ ๐จ frontend/ # Angular 17 Application
โ โโโ src/
โ โ โโโ app/
โ โ โ โโโ components/
โ โ โ โ โโโ dashboard/
โ โ โ โ โ โโโ dashboard.component.ts
โ โ โ โ โ โโโ dashboard.component.html
โ โ โ โ โ โโโ dashboard.component.css
โ โ โ โ โโโ sensor-charts/
โ โ โ โ โโโ charts.component.ts
โ โ โ โ โโโ charts.component.html
โ โ โ โโโ services/
โ โ โ โ โโโ websocket.service.ts
โ โ โ โโโ models/
โ โ โ โโโ sensor-data.ts
โ โ โโโ environments/
โ โ โ โโโ environment.ts
โ โ โ โโโ environment.prod.ts
โ โ โโโ assets/
โ โ โโโ env.js
โ โโโ package.json
โ โโโ angular.json
โ โโโ Dockerfile
โ
โโโ ๐ backend/ # FastAPI Application
โ โโโ app/
โ โ โโโ main.py # FastAPI application
โ โ โโโ models.py # SQLAlchemy models
โ โ โโโ database.py # Database configuration
โ โโโ requirements.txt
โ โโโ Dockerfile
โ
โโโ โธ๏ธ k8s/ # Kubernetes Manifests
โ โโโ frontend/
โ โ โโโ deployment.yaml
โ โ โโโ service.yaml
โ โ โโโ configmap.yaml
โ โโโ backend/
โ โ โโโ deployment.yaml
โ โ โโโ service.yaml
โ โ โโโ configmap.yaml
โ โโโ postgresql.yaml
โ โโโ ingress.yaml
โ
โโโ ๐ monitoring/ # Observability Stack
โ โโโ grafana-dashboards/
โ โโโ prometheus-alerts/
โ โโโ kiali-config/
โ
โโโ ๐ณ docker-compose.yml # Development environment
โโโ ๐ง deploy-to-k8s.sh # Production deployment script
โโโ ๐ README.md # This file
# Real-time sensor data stream
ws://localhost:30080/ws/sensors
# Example WebSocket message
{
"sensor_id": "sensor_1",
"temperature": 23.5,
"humidity": 65.2,
"pressure": 1013.25,
"timestamp": "2024-01-15T10:30:00Z"
}| Method | Endpoint | Description |
|---|---|---|
GET |
/api/health |
System health check |
GET |
/api/sensors |
Get sensor data (with pagination) |
GET |
/api/sensors/{sensor_id} |
Get data for specific sensor |
GET |
/api/metrics |
System performance metrics |
POST |
/api/sensors |
Submit sensor data (batch) |
# Health check
curl http://localhost:30080/api/health
# Get latest sensor data
curl http://localhost:30080/api/sensors?limit=10
# API Documentation
open http://localhost:30080/docsReal-time Features:
- ๐ Live Sensor Metrics - Temperature, humidity, pressure
- ๐ WebSocket Connection - Real-time data streaming
- ๐ฑ Responsive Design - Mobile-first approach
- ๐ฏ Data Visualization - Charts and graphs
- โก Instant Updates - Sub-second latency
# Kiali - Service Mesh Visualization
kubectl port-forward -n istio-system svc/kiali 20001:20001
# http://localhost:20001 (admin/admin)
# Grafana - Metrics Dashboard
kubectl port-forward -n monitoring svc/monitoring-grafana 3001:80
# http://localhost:3001
# ArgoCD - GitOps Dashboard
kubectl port-forward -n argocd svc/argocd-server 8081:443
# https://localhost:8081- ๐ Kubernetes Cluster Metrics - Resource usage and performance
- ๐ SkySense Application Metrics - Custom application monitoring
- ๐๏ธ PostgreSQL Performance - Database health and queries
- ๐ก๏ธ Istio Service Mesh - Traffic flow and security
- Microservices architecture with Kubernetes
- Real-time WebSocket communication
- PostgreSQL database with time-series data
- Angular frontend with live dashboard
- Docker containerization
- Service Mesh with Istio
- Monitoring with Prometheus/Grafana
- GitOps with ArgoCD
- Machine Learning anomaly detection
- Advanced data visualization
- Multi-tenant support
- Advanced alerting system
- Mobile application (React Native)
- Edge computing capabilities
- Advanced analytics dashboard
- IoT device management
- Predictive maintenance features
Pashitox - Full Stack Developer & IoT Specialist
"Building the future of IoT, one sensor at a time. Transforming real-world data into actionable insights through cutting-edge technology and scalable cloud infrastructure."
This project is licensed under the MIT License - see the LICENSE file for details.