[ENHANCEMENT] Support LiteLLM as a first-class provider in CLI (flags/env/config)

[farazoman]

Problem (one or two sentences)

The CLI does not allow users to configure LiteLLM (litellm) as a provider using built-in flags or environment variables, even though backend engine support exists. Users cannot set LiteLLM endpoint, API key, or model from the CLI directly.

Context (who is affected and when)

Anyone running Roo Code CLI who wants to use an OpenAI-compatible LiteLLM endpoint, including for enterprise/private LLM proxy setups or local inference, is blocked. Only some providers are directly configurable from CLI options; litellm is not.

Desired behavior (conceptual, not technical)

CLI users should be able to run Roo Code CLI with --provider litellm and supply required LiteLLM-specific parameters (endpoint/baseURL, API key, model) via flags and/or standard environment variables, with parity to how OpenAI and OpenRouter are supported.

Constraints / preferences (optional)

Flag+env var mapping should use clear and consistent names. It should be possible to run LiteLLM locally without code changes, and ideally allow fallback defaults for local use (e.g., http://localhost:4000).

Request checklist

• I've searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear context and impact

Roo Code Task Links (optional)

No response

Acceptance criteria (optional)

• CLI can launch using --provider litellm with correct model/apiKey/baseUrl flags or env vars
• Equivalent support for LiteLLM in CLI as is available for OpenAI/OpenRouter
• Good documentation for new flags/env vars in README and CLI help

Proposed approach (optional)

Introduce support in CLI for provider litellm, mapping CLI flags/env vars for litellmBaseUrl, litellmApiKey, litellmModelId. Default to http://localhost:4000 when not specified.

Trade-offs / risks (optional)

None anticipated; would improve flexibility for self-hosted/model-proxy scenarios without affecting existing providers.

[roomote]

I have analyzed this issue and here is my proposed implementation plan:

Summary

LiteLLM already has backend support in the core types package (packages/types), but the CLI (apps/cli) does not yet expose it as a first-class provider. The implementation requires updating several files in the CLI package.

Implementation Plan

1. Update apps/cli/src/types/types.ts

• Add "litellm" to the supportedProviders array
• Add litellmBaseUrl?: string to the FlagOptions type

2. Update apps/cli/src/lib/utils/provider.ts

• Add litellm: "LITELLM_API_KEY" to the envVarMap record
• Add a new case for litellm in the getProviderSettings function:

  case "litellm":
    if (apiKey) config.litellmApiKey = apiKey
    if (model) config.litellmModelId = model
    if (baseUrl) config.litellmBaseUrl = baseUrl
    break

• Update the function signature to accept an optional baseUrl parameter

3. Update apps/cli/src/index.ts

• Add new CLI flags:
  • --litellm-base-url <url> for specifying the LiteLLM endpoint (default: http://localhost:4000 when using litellm provider)

4. Update apps/cli/src/commands/cli/run.ts

• Pass the litellmBaseUrl flag to getProviderSettings
• Apply default base URL (http://localhost:4000) when provider is litellm and no base URL is specified

5. Add tests in apps/cli/src/lib/utils/__tests__/provider.test.ts

• Test getApiKeyFromEnv with "litellm" provider
• Test getProviderSettings with LiteLLM-specific settings

6. Update apps/cli/README.md

• Document the new --provider litellm option
• Document --litellm-base-url flag and LITELLM_API_KEY / LITELLM_BASE_URL environment variables

Environment Variables

• LITELLM_API_KEY - API key for LiteLLM
• LITELLM_BASE_URL - Base URL for LiteLLM endpoint (optional, defaults to http://localhost:4000)

CLI Usage Examples

# Using environment variables
export LITELLM_API_KEY="your-key"
export LITELLM_BASE_URL="http://localhost:4000"
roo --provider litellm -m claude-3-sonnet "your prompt"

# Using flags
roo --provider litellm --api-key "your-key" --litellm-base-url "http://localhost:4000" -m claude-3-sonnet "your prompt"

# Using default local LiteLLM (http://localhost:4000)
roo --provider litellm --api-key "your-key" "your prompt"

This approach maintains consistency with how other providers (OpenRouter, OpenAI, etc.) are supported in the CLI while leveraging the existing LiteLLM backend infrastructure.

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[mrubens]

@farazoman does this seem like this would address your ask?

[farazoman]

@mrubens yup, I would say so. Want me to tag roomote as well?

[mrubens]

@roomote go for it!

[roomote]

Thanks for the approval @mrubens! I'll start implementing LiteLLM as a first-class provider in the CLI based on the proposed plan. I'll open a PR shortly to share progress.

[roomote]

Opened a draft PR with my implementation: #10840

This adds LiteLLM as a first-class provider in the CLI with support for:

• --provider litellm flag
• --litellm-base-url <url> flag (defaults to http://localhost:4000)
• LITELLM_API_KEY and LITELLM_BASE_URL environment variables

Feedback welcome!