Skip to content

Support litellm (or OpenRouter) as embedding provider #847

@BasuruK

Description

@BasuruK

Feature Request: Support litellm / OpenRouter for Embedding Models

Problem

Currently, OpenViking's embedding configuration only supports three providers:

  • volcengine (Doubao)
  • openai (text-embedding-3-large, etc.)
  • jina

The VLM layer supports litellm which can bridge to OpenRouter, Ollama, Anthropic, DeepSeek, Gemini, and many other providers. However, the embedding layer does not support litellm, forcing users to either:

  1. Use Volcengine Doubao (requires separate account/API)
  2. Use OpenAI embeddings (separate API key + cost)
  3. Use Jina (free, but limited model selection)

This is a significant blocker for users who want to centralize all model calls through OpenRouter for its simplicity.

Use Case

Many developers prefer OpenRouter due to its simplicity — unified API, single dashboard, single billing for dozens of providers. OpenViking's VLM already supports litellm → OpenRouter, but the embedding layer is a hard gap that breaks the unified workflow.

Proposed Solution

Extend the embedding provider list to include litellm, similar to how VLM already works:

"embedding": {
  "dense": {
    "provider": "litellm",
    "model": "openai/text-embedding-3-small",
    "api_base": "https://openrouter.ai/api/v1",
    "api_key": "<openrouter-key>",
    "dimension": 1024
  }
}

This would allow:

  • OpenRouter embeddings
  • Ollama local embeddings (via litellm bridge)
  • vLLM served embeddings
  • Any OpenAI-compatible embedding endpoint

Why litellm?

litellm already handles OpenAI-compatible embedding endpoints cleanly. Since VLM already uses litellm successfully, the infrastructure is already in place. Extending it to embedding would be a natural addition.

Alternatives Considered

  1. Jina embeddings — Works today, but model selection is limited and requires outbound API call to jina.ai
  2. OpenAI embeddings — Works but adds cost and separate API dependency
  3. Volcengine Doubao — Not free, requires separate account

None of these match the flexibility of litellm for users with existing OpenRouter/Ollama setups.

Additional Context

  • This would bring embedding parity with how VLM already works via litellm
  • Many AI agents are standardizing on OpenRouter for all LLM/embedding needs
  • Embedding model format is standardized (OpenAI-compatible), so litellm should work with minimal changes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    In progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions