Infrastructure Automation

Overview

This repository demonstrates an enterprise-grade production ready CI/CD pipeline and kubernetes infrastructure designed for deploying and managing containerized microservices at scale. The architecture is based on real-world experience operation AI/ML platforms handling thousands of concurrent requests across 50+ microservices.

Note: All configurations have been generalized and sanitized to remove company-specific and sensitive infromation while preserving architectural patterns, best practices, and operational insights gained from production environments.

Architecture

CI/CD Pipeline

Automated build, test, and deployment workflow with GitLab webhooks, Jenkins pipelines, Harbor registry, and Kubernetes orchestration.

Pipeline Stages:

1. Source Control: GitLab repository with webhook triggers based on GitFlow branch strategies.
2. Build & Test: Jenkins automated pipeline

• Static analysis (Eslint, Checkstyle)
• Gradle build
• Docker image creation

3. Registry: Harbor private container registry
4. Deployment: Kubernetes rolling updates
5. Notification: Mattermost/Slack alerts for success/failure

Kubernetes

Microservices-based application architecture deployed on Kubernetes with service mesh patters, job-based training workflows, and persistend data storages. [img src=./diagrams/k8s_cluster.png]

Key Components:

• Ingress Controller: External traffic routing
• API Gateway: Central entry point for all backend services
• Frontend: Web application
• Microservices
  • Model Mngmt: ML model lifecycle management
  • System Mngmt: Platform administration
  • Agent: Task execution and orchestration
  • LLM: Large Language Model inference in air-gapped on promise network
• Training Jobs: Kuberenetes job-based model training workflows
• Data Statefulsets
  • RDB: Relational database with persistent volumes
  • Vector DB: Vector database for similarity search

Traffic Flow

• User -> Ingress -> Frontend/API Gateway -> Backend Services -> Databases (synchronous)
• Model Mngmt -> Training Jobs (asynchronous job creations)
• Agent -> LLM (inference requests)

Tech Stack

CI/CD

• Source Control: GitLab
• CI/CD Engine: Jenkins
• Build Tools: Gradle, Docker
• Container Registry: Harbor
• Notification: Mattermost, Slack

Infrastructure

• Orchestration: Kubernetes
• Ingress: Nginx Ingress Controller
• Service Mesh: API Gateway pattern (Spring Cloud API Gateway)
• Storage: Persistent Volumes (Statefulset)

Monitoring

• Metrics: Prometheus
• Visualization: Grafana

Infrastructure Setup

For Kubernetes cluster installation and configuration using Ansible: k8s-ansible-installer