Refactor(Refactoring): Refactor Services Folder

[Mehak-Conrad]

Description

This PR corresponds to the following Refactor Services Folder

Key Changes

• Restructured LLM clients for OpenAI, Anthropic, DeepSeek, Google AI, and DashScope with a standardized interface.
• Added utility handlers:
  • APIKeyManager for rotating and managing multiple API keys.
  • LLMCallRetryHandler for centralising retry/backoff logic.
• Moved validate_llm_and_model to a more semantically correct location: validators/model_validator.py.
• Introduced chat_formatter module to handle multi-turn formatting across providers (Anthropic, OpenAI, Google AI).
• Centralised constants related to services into constants/services/.
• DashScopeClient and DeepSeekClient now inherit from OpenAIClient instead of BaseClient, as they share similar interfaces.
• Removed unused batch logic from OpenAIClient as it's no longer required and was leading to unnecessary complexity.

Radon Analysis after refactoring:

services/clients/dashscope_client.py
    C 22:0 DashScopeClient - A
    M 31:4 DashScopeClient.__init__ - A
    M 41:4 DashScopeClient.is_o_series_model - A
    M 50:4 DashScopeClient.get_chat_completion_params - A
services/clients/base_client.py
    C 16:0 Client - A
    M 25:4 Client.__init__ - A
    M 37:4 Client.execute_prompt - A
    M 49:4 Client.excecute_chat - A
services/clients/deepseek_client.py
    C 22:0 DeepSeekClient - A
    M 31:4 DeepSeekClient.__init__ - A
    M 41:4 DeepSeekClient.is_o_series_model - A
    M 50:4 DeepSeekClient.get_chat_completion_params - A
services/clients/openai_client.py
    C 29:0 OpenAIClient - A
    M 79:4 OpenAIClient.execute_prompt - A
    M 100:4 OpenAIClient.execute_chat - A
    M 38:4 OpenAIClient.__init__ - A
    M 142:4 OpenAIClient.get_chat_completion_params - A
    M 64:4 OpenAIClient.is_o_series_model - A
    M 124:4 OpenAIClient._configure_client - A
    M 129:4 OpenAIClient.__create_completion - A
services/clients/google_ai_client.py
    M 63:4 GoogleAIClient.execute_prompt - A
    C 29:0 GoogleAIClient - A
    M 83:4 GoogleAIClient.execute_chat - A
    M 47:4 GoogleAIClient.__init__ - A
    M 106:4 GoogleAIClient._configure_genai - A
    M 111:4 GoogleAIClient._get_model - A
    M 123:4 GoogleAIClient._send_prompt - A
    M 144:4 GoogleAIClient._send_chat - A
services/clients/anthropic_client.py
    C 28:0 AnthropicClient - A
    M 53:4 AnthropicClient.execute_prompt - A
    M 75:4 AnthropicClient.execute_chat - A
    M 102:4 AnthropicClient._create_completion - A
    M 36:4 AnthropicClient.__init__ - A
    M 97:4 AnthropicClient._configure_client - A
services/clients/client_factory.py
    C 26:0 ClientFactory - A
    M 35:4 ClientFactory.get_client - A
services/utils/api_key_manager.py
    C 7:0 APIKeyManager - A
    M 16:4 APIKeyManager.__init__ - A
    M 26:4 APIKeyManager.get_current_key - A
    M 35:4 APIKeyManager.get_num_of_keys - A
    M 44:4 APIKeyManager.rotate_api_key - A
services/utils/call_retry_handler.py
    C 29:0 LLMCallRetryHandler - A
    M 61:4 LLMCallRetryHandler.execute_with_retries - A
    M 80:4 LLMCallRetryHandler.__handle_llm_call_exception - A
    M 94:4 LLMCallRetryHandler.__handle_quota_exceeded - A
    M 119:4 LLMCallRetryHandler.is_quota_exceeded_error - A
    M 40:4 LLMCallRetryHandler.__init__ - A
    M 110:4 LLMCallRetryHandler.__backoff_delay - A
services/chat_formatter/anthropic_chat_formatter.py
    C 13:0 AnthropicChatFormatter - A
    M 64:4 AnthropicChatFormatter._extract_system_message - A
    M 36:4 AnthropicChatFormatter._format_messages - A
    M 16:4 AnthropicChatFormatter.format - A
services/chat_formatter/openai_chat_formatter.py
    C 13:0 OpenAIChatFormatter - A
    M 16:4 OpenAIChatFormatter.format - A
services/chat_formatter/base_chat_formatter.py
    C 11:0 LLMChatFormatter - A
    M 14:4 LLMChatFormatter.__init__ - A
    M 24:4 LLMChatFormatter.format - A
services/chat_formatter/google_ai_chat_formatter.py
    C 13:0 GoogleAIChatFormatter - A
    M 73:4 GoogleAIChatFormatter._extract_system_message - A
    M 91:4 GoogleAIChatFormatter._extract_latest_user_message - A
    M 109:4 GoogleAIChatFormatter._should_include_chat_message - A
    M 43:4 GoogleAIChatFormatter._format_history - A
    M 21:4 GoogleAIChatFormatter.format - A
services/validators/model_validator.py
    F 9:0 validate_llm_and_model - A

64 blocks (classes, functions, methods) analyzed.
Average complexity: A (1.96875)

For Reviewers (Optional)

• Note: This PR involves a large number of file changes, including many import path updates. To ease review, we recommend reviewing the PR commit by commit rather than via the full file diff.

[Copilot]

Pull Request Overview

This PR refactors the Services folder to standardize LLM client interfaces, centralize utility functionality, and clean up unused code.

• Standardizes and restructures LLM clients (OpenAI, Anthropic, DeepSeek, Google AI, DashScope) with a unified interface.
• Introduces APIKeyManager and LLMCallRetryHandler for managing API key rotation and retry/backoff logic.
• Moves validation logic and consolidates chat formatters while removing redundant batch-related code.

Reviewed Changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
server/services/utils/call_retry_handler.py	Adds retry/backoff logic with API key rotation support.
server/services/utils/api_key_manager.py	Implements API key rotation functionality.
server/services/clients/*	Refactors client implementations for various LLM providers, standardizing the interface.
server/services/chat_formatter/*	Updates chat formatting modules for consistency with provider-specific formats.
server/services/clients/client_factory.py	Integrates client instantiation via a factory pattern.

Comments suppressed due to low confidence (1)

server/services/chat_formatter/google_ai_chat_formatter.py:109

• [nitpick] The function '_should_include_chat_message' returns True when a message should be skipped, which conflicts with its name; consider renaming it to something like '_should_skip_chat_message' for clarity.

def _should_include_chat_message(self, role: ChatRole, message: str, last_user_msg: Optional[str]) -> bool:

[Copilot]

Pull Request Overview

This PR refactors the Services folder by standardizing the interfaces for various LLM clients, consolidating common utilities like APIKeyManager and LLMCallRetryHandler, and reorganizing module locations for improved clarity.

• Refactored individual LLM clients (Anthropic, OpenAI, Google AI, etc.) with a consistent interface
• Updated client factory and configuration handling to use a unified LLMConfig structure
• Adjusted import paths and removed unused code from legacy modules

Reviewed Changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
server/services/clients/anthropic_client.py	Implemented new AnthropicClient with standardized retry and formatting logic
server/services/client_factory.py	Removed old factory logic in favor of updated client instantiation using LLMConfig
server/services/chat_formatter/*.py	Introduced new chat formatter implementations for OpenAI, Anthropic, and Google AI using the common base formatter
server/scripts/*	Updated client instantiation and import paths to use the new LLMConfig
server/app/main.py	Updated API endpoints to validate and instantiate clients via LLMConfig

[fi5421]

n