Distributed Task Scheduler

Overview

The Distributed Task Scheduler is a scalable system designed to handle large-scale task execution with dynamic load balancing and priority-based scheduling. It leverages gRPC for efficient task orchestration and PostgreSQL for reliable task storage.

Features

• Scalable Task Scheduling: Efficiently manages 5,000+ concurrent tasks.
• Dynamic Load Balancing: Reduces average job execution latency by 40%.
• Priority-Based Scheduling: Ensures optimal resource utilization.
• gRPC for Task Execution: Reduces communication latency by 30% compared to REST.
• Dockerized Deployment: Simplifies setup and scalability.

Tech Stack

• Go – Core application logic
• gRPC – Efficient remote procedure calls for task execution
• PostgreSQL – Persistent task storage
• Docker – Containerized deployment

Installation

Prerequisites

Install Docker before running the application.

Steps

1. Clone the Repository:

   git clone https://github.com/yourusername/distributed-task-scheduler.git
   cd src

2. Start and Run Application (Docker):

   docker-compose up --build 

Usage

1. Initialize Database Connection: The application connects to PostgreSQL and creates the necessary tables.
2. Task Submission via gRPC: Tasks can be scheduled and executed through gRPC service calls.
3. Monitor Execution: Tasks transition through scheduled, picked, started, completed, or failed states.

Checking Task Entries

To verify if a task has been added, run:

docker exec -it my_postgres psql -U youruser -d tasksdb -c "SELECT * FROM tasks;"