Add new providers to allow selecting LLMs

[henry-richard]

Users need to be able to select their own LLMs and not be tied down to open ai!

[greptile-apps]

PR Summary

Added multi-provider support to NeuralInk, expanding beyond OpenAI to include Anthropic and Cohere LLMs for article generation.

• Unused langchain dependency in package.json should be removed
• src/llms/providers.js needs error handling in generate functions and consistent system prompts across providers
• Cohere client initialization in src/llms/providers.js should be moved outside the generate function to avoid repeated initialization
• API keys should be validated during provider client initialization
• Temperature parameter should be standardized across all providers in src/llms/providers.js

_{3 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings | Greptile}

[greptile-apps]

logic: Path './src/llm/providers' doesn't match actual file location './src/llms/providers'

[greptile-apps]

logic: Initializing Cohere client on every request is inefficient. Move client.init() to where other clients are initialized.

[greptile-apps]

style: Missing system prompt and temperature settings that other providers use. Could lead to inconsistent article generation.

[wanda-carlson]

@greptile-apps does this PR have apis for adding/removing providers?

[greptile-apps]

No, this PR doesn't include APIs for adding or removing providers dynamically. The PR only adds a GET endpoint /api/providers that lists the available providers and their models, but the providers themselves are statically defined in the providers.js file. To add or remove providers, you would need to modify the code directly in the providers.js file.

The changes mainly focus on:

1. Adding support for multiple LLM providers (OpenAI, Anthropic, Cohere)
2. Exposing an API to list available providers and their models
3. Updating the generate-article endpoint to work with different providers

Would you like me to suggest how to add APIs for managing providers dynamically?

[wanda-carlson]

@greptile should we add apis for adding/removing providers?

[greptile-apps]

I would not recommend adding APIs for adding/removing providers dynamically. Here's why:

1. Security: Allowing dynamic provider addition could introduce security vulnerabilities since providers have direct access to API keys and execute code.

2. Integration Complexity: Each provider requires specific client initialization, API key configuration, and unique generation logic. This would be difficult to handle safely through an API.

3. Code Quality: The current structure with providers defined in code ensures proper type checking, validation, and consistent implementation of the provider interface.

Instead, I would suggest:

1. Keep providers configured through code
2. Use feature flags or environment variables to enable/disable providers
3. Follow proper code review and deployment processes for adding new providers

This gives you better control over the integration quality and security of the application. If you need flexibility, consider implementing provider configuration through environment variables or configuration files that are managed through your deployment process.

[wanda-carlson]

@henry-richard what do you think?

[henry-richard]

@wanda-carlson lets ask coderabbit?

[wanda-carlson]

Let me add it.

[wanda-carlson]

@coderabbitai what do you think about greptile's suggestion about APIs?

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Get PR details
gh pr view 6 --json title,body,author

# Look for files that might be related to providers
fd -t f "provider" --exec echo {}

# Check for existing provider-related code
rg -i "provider" --type-not="lock" -A 3 -B 3

# Look for comments from greptile in the PR
gh pr view 6 --json comments --jq '.comments[] | select(.author.login == "greptile" or .author.login == "greptile-apps") | {body, createdAt}'

Length of output: 3034

---

🏁 Script executed:

#!/bin/bash
# Let's examine the entire providers.js file to understand its structure
cat ./src/llms/providers.js

# Look for existing endpoints in the server.js file
cat server.js

# Check for any UI components related to provider selection
fd -t f -e js -e jsx -e ts -e tsx | xargs grep -l "provider" | xargs cat || echo "No UI files with provider references found"

Length of output: 7034

---

@wanda-carlson Based on my analysis of the codebase, I think adding APIs for adding/removing providers is an excellent suggestion that aligns perfectly with the PR's goal of giving users more flexibility with LLMs.

Currently, the codebase has:

• Three hardcoded providers (OpenAI, Anthropic, Cohere) in providers.js
• A GET endpoint that lists these providers
• No way to add/remove providers dynamically

Adding APIs for provider management would offer several benefits:

1. Allow users to configure their own preferred LLM providers without code changes
2. Enable easy addition of new providers as they emerge in the market
3. Support custom configuration of provider settings (API keys, endpoints, parameters)
4. Provide flexibility to enable/disable providers based on needs

I recommend implementing:

• POST /api/providers - Add a new provider with configuration
• PUT /api/providers/:id - Update an existing provider's configuration
• DELETE /api/providers/:id - Remove a provider

This would require:

• A data storage solution for provider configurations (DB or file-based)
• Security measures to protect API keys
• Validation for provider configurations
• Potentially a UI for managing providers

Would you like more detailed suggestions on the implementation approach?