Improve Scalability and Performance of the Server

[ajaynegi45]

Problem Description:

Currently, the server is designed using a thread-per-request model (HttpConnectionWorkerThread), which handles each incoming request in a separate thread. While this is functional for small workloads, this model does not scale well for high-traffic environments due to the limited number of threads that can be efficiently handled by the system. Moreover, the current design lacks mechanisms for load balancing and connection pooling, which are crucial for ensuring high scalability and performance in modern distributed systems.

Objective:

The goal is to significantly improve the server's scalability and performance by:

1. Implementing asynchronous and non-blocking I/O.
2. Introducing load balancing strategies.
3. Setting up connection pooling to manage and optimize external connections efficiently.

These changes will make the server capable of handling a much larger number of concurrent requests, with improved responsiveness and resource management.

---

Task 1: Implement Asynchronous and Non-Blocking I/O

To replace the current thread-per-request model, we need to introduce asynchronous and non-blocking I/O. This will allow the server to handle multiple connections concurrently without spawning a thread for each request.

Questions for Contributors:

• Which would be more effective for our project, NIO (Java's built-in non-blocking I/O) or a high-performance library like Netty? Please provide a detailed explanation of the pros and cons of each option.
• How do you plan to implement asynchronous I/O in the project? Provide a step-by-step breakdown, including code structure changes and how you'd handle concurrent connections.

Suggested Approach:

• Investigate the current server workflow and analyze how the request-handling process can be decoupled from thread management.
• Evaluate NIO and Netty, and propose the best option for this specific use case.
• Implement a new I/O model based on the choice, ensuring backward compatibility where possible.

---

Task 2: Load Balancing

To handle high-traffic environments and distribute incoming requests efficiently, we need to introduce custom load balancing strategies such as round-robin, least connections, or even hash-based methods.

Questions for Contributors:

• Which load-balancing strategy do you propose? Please describe in detail how the load balancer would distribute requests among workers or instances.
• How would you implement the load balancing logic into the server? Provide a detailed technical plan, including the architecture and how requests will be routed.

Suggested Approach:

• Evaluate various load balancing algorithms (e.g., round-robin, least connections) and propose the one that fits our server’s architecture.
• Design and implement the load balancing mechanism, ensuring it is modular and easily customizable.
• Ensure the load balancer can distribute requests to multiple server instances (or threads) while keeping track of connections in an efficient manner.

---

Task 3: Connection Pooling

External resources (e.g., databases, APIs) can be bottlenecks if every connection is created and destroyed per request. We need to implement connection pooling to reuse and manage connections efficiently.

Questions for Contributors:

• How will you implement connection pooling in the server? Please provide details on how the connection pool will manage resources (e.g., lifecycle of connections, pool size limits).
• What considerations will you take to ensure that the connection pool is thread-safe and efficient?

Suggested Approach:

• Design a connection pooling mechanism that fits the server’s architecture.
• Ensure proper resource management, including connection timeout handling, pool size limits, and thread-safety.

---

Contributor Guidelines:

1. If you are interested in working on this, please comment below with a detailed description of how you plan to implement each task. Make sure your plan includes:

   • The technology/libraries you plan to use.
   • Code structure and architecture changes.

2. Before making any contributions, please carefully read the README.md file to better understand the project's goals and purpose. This will give you clarity on our mission.

3. Make sure to review the CONTRIBUTING.md file for important guidelines. This file contains critical information to make the contribution process smoother, especially for newcomers.

4. Adhere to the project’s code style, documentation, and testing guidelines as outlined in the CONTRIBUTING.md file.

5. Once you have a clear implementation plan and have discussed it with the project maintainers, proceed with your changes. Be prepared to explain your choices and respond to feedback during code review.