feat(embedding): add litellm as embedding provider#853
feat(embedding): add litellm as embedding provider#853stubbi wants to merge 1 commit intovolcengine:mainfrom
Conversation
Adds LiteLLM as a new embedding provider, bringing embedding parity with the VLM layer which already supports litellm. This enables users to route embedding requests through OpenRouter, Ollama, vLLM, and any other OpenAI-compatible endpoint via litellm's unified interface. Closes volcengine#847 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
|
Failed to generate code suggestions for PR |
ZaynJarvis
left a comment
There was a problem hiding this comment.
Critical Issues Found
1. Module-level side effect on import (litellm_embedders.py:13)
os.environ.setdefault("LITELLM_LOCAL_MODEL_COST_MAP", "True")This mutates the process environment at import time — not when the embedder is used. Since __init__.py imports this module (behind try/except), this fires as soon as the embedder package is loaded, even if the user never uses litellm. This can interfere with other litellm usage in the same process that intentionally sets this env var differently.
Fix: Move this inside LiteLLMDenseEmbedder.__init__ or into _build_kwargs.
2. Probe API call during __init__ / silent fallback (litellm_embedders.py:89-93)
def _detect_dimension(self) -> int:
try:
result = self.embed("test")
return len(result.dense_vector) if result.dense_vector else 1536
except Exception:
return 1536When dimension=None, the constructor calls _detect_dimension() which calls self.embed("test"). This means:
- A billable API call the user didn't ask for, fired during object construction
- A network side effect inside
__init__— surprising and hard to test - Silent fallback to 1536 on any exception (auth failure, network error, wrong model) — the user gets no indication their config is wrong
Fix: Require dimension as a mandatory parameter (other providers do this), or at minimum make the probe call lazy and log a warning on fallback instead of silently swallowing errors.
3. Missing None guard for LiteLLMDenseEmbedder in factory
The factory in embedding_config.py imports LiteLLMDenseEmbedder but doesn't check if it's None (which happens when litellm isn't installed). Other providers like Gemini have this guard. If a user configures provider: "litellm" without litellm installed, they'll get a confusing TypeError: 'NoneType' object is not callable instead of a clear error message.
Summary
Adds LiteLLM as a new embedding provider, resolving the gap between VLM (which already supports litellm) and the embedding layer. This enables users to route embedding requests through OpenRouter, Ollama, vLLM, and any OpenAI-compatible endpoint via litellm's unified interface.
openviking/models/embedder/litellm_embedders.py—LiteLLMDenseEmbedderclass extendingDenseEmbedderBaseembedding_config.py— added"litellm"to the provider validation list, factory registry, and docstringsembedder/__init__.py— added conditional import (graceful fallback if litellm not installed)tests/unit/test_litellm_embedder.py— 15 tests covering embed, batch embed, non-symmetric mode, factory integration, and config validationUsage example
Closes #847
Test plan
🤖 Generated with Claude Code