Whether only LLM like ChatGPT is available because of embedding model?

[Sia2010]

❓ Questions and Help

We sincerely suggest you to carefully read the documentation. After that, if you still feel puzzled, please describe the question clearly under this issue.

Hi, I wonder whether only chatgpt this kind of llm with embedding model is availble for this program? Because LLM like deepseek does not have embedding model so it will raise error in "RAG" process?

If only LLM like deepseek is available for me, what change can I make to adjust the program?

Thank you.

[onestardao]

Hey @Sia2010 — you're absolutely right to ask this. A lot of folks hit this silent trap in RAG pipelines. Let me break it down for you, one espresso shot at a time ☕:

---

🚧 The Root Problem:

Not all LLMs come with embedding models built in.

• ChatGPT (like gpt-3.5/4) – doesn't embed by itself.
• Deepseek, LLaMA, etc – many also don’t embed unless wrapped with an external embedding model (e.g. text-embedding-ada-002, nomic-embed, bge-small…).

So when you run a RAG process and it says “embedding not found” — it’s not because the LLM is broken, it’s because the system expected embeddings but didn’t get them.

🍷 How to fix it:

If your LLM (like Deepseek) does not provide embeddings, here’s what you do:

1. Explicitly add an embedding model in your config:

   embed_model: sentence-transformers/all-MiniLM-L6-v2

   or

   embed_model: huggingface/nomic-embed-text-v1

2. Keep your LLM separate — just use it for generation, not embedding.

   That means:

   • Use embed_model for vectorization
   • Use llm_model for response generation

---

🍷Bonus Tip (Drunk Mode activated):

Think of your system like a bar:

• The embedding model is the bartender — takes your question, and mixes it into a "flavor profile" (vector)
• The LLM is the poet at the end of the bar — reads that flavor and tells you a story from it

If you ask the poet to mix your drink? Chaos. Spilled gin. Broken vectors. Error 500.

---

Let me know what framework you’re using (LangChain? RAGFlow? Raw API?), and I can show exact config lines.

You’re close. Just missing that bartender. 😎

---

[Sia2010]

I see!! I've already figured out this problem, however, thanks so much for your explicit explanation!!

[shadowleaves]

Hi, @onestardao , I'm running into perhaps the similar issue as well. still at the early stage of trying to implement exactly as you described in the readme. I'm using deepseek API and keep getting the following error:

2025-07-31 10:09:38.022 | INFO | alphaagent.oai.llm_utils:_create_chat_completion_inner_function:752 - Response:

2025-07-31 10:12:43.409 | WARNING | alphaagent.oai.llm_utils:_try_create_chat_completion_or_embedding:580 - Expecting value: line 1 column 1 (char 0)
2025-07-31 10:12:43.412 | WARNING | alphaagent.oai.llm_utils:_try_create_chat_completion_or_embedding:581 - Retrying 1th time...
2025-07-31 10:12:48.426 | INFO | alphaagent.oai.llm_utils:_create_chat_completion_inner_function:659 -

my .env setting is as below:

Global configs:

USE_AZURE=False
CHAT_USE_AZURE_TOKEN_PROVIDER=False
EMBEDDING_USE_AZURE_TOKEN_PROVIDER=False
MAX_RETRY=5
RETRY_WAIT_SECONDS=5
FACTOR_MINING_TIMEOUT=10800
USE_LOCAL=True

LLM API Setting:

OPENAI_BASE_URL="https://api.deepseek.com"
OPENAI_API_KEY=######
CHAT_MODEL=deepseek-v3 # e.g., "deepseek-v3"
REASONING_MODEL=deepseek-reasoner # e.g. "deepseek-reasoner"
EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2 #text-embedding-3-small # RAG

^^here for the embedding model I've tried both the "text-embedding-3-small" and what you have recommended above but neither works. my apology if sounded stupid here but is the embedding model something I need to install separately?

thanks!

Hey @Sia2010 — you're absolutely right to ask this. A lot of folks hit this silent trap in RAG pipelines. Let me break it down for you, one espresso shot at a time ☕:

🚧 The Root Problem:

Not all LLMs come with embedding models built in.

• ChatGPT (like gpt-3.5/4) – doesn't embed by itself.
• Deepseek, LLaMA, etc – many also don’t embed unless wrapped with an external embedding model (e.g. text-embedding-ada-002, nomic-embed, bge-small…).

So when you run a RAG process and it says “embedding not found” — it’s not because the LLM is broken, it’s because the system expected embeddings but didn’t get them.

🍷 How to fix it:

If your LLM (like Deepseek) does not provide embeddings, here’s what you do:

1. Explicitly add an embedding model in your config:
   embed_model: sentence-transformers/all-MiniLM-L6-v2

   or
   embed_model: huggingface/nomic-embed-text-v1

2. Keep your LLM separate — just use it for generation, not embedding.
   That means:

   • Use embed_model for vectorization
   • Use llm_model for response generation

🍷Bonus Tip (Drunk Mode activated):

Think of your system like a bar:

• The embedding model is the bartender — takes your question, and mixes it into a "flavor profile" (vector)
• The LLM is the poet at the end of the bar — reads that flavor and tells you a story from it

If you ask the poet to mix your drink? Chaos. Spilled gin. Broken vectors. Error 500.

Let me know what framework you’re using (LangChain? RAGFlow? Raw API?), and I can show exact config lines.

You’re close. Just missing that bartender. 😎

[onestardao]

yo you’re super close — but what you’re seeing is actually a different class of failure
deepseek-v3 + sentence-transformers sometimes throws ghost vectors (empty or misaligned) even if config looks right

this one’s on the “pre-deploy mismatch” list — system expects embed-ready output, but infra isn’t bootstrapped cleanly

i mapped this + 15 other quiet failure modes in a doc, mostly for ppl running into stuff like this over and over
lemme know if you want it, can save you hours of headbanging

[shadowleaves]

That sounds great ! and yes please @onestardao I'd really appreciate if you could kindly send the doc to me. Thanks!

[onestardao]

yo that exact error combo you’re getting — where config looks fine but the system still throws a missing-value or broken-line crash during embedding init — is actually a classic case of what we call predeploy collapse (No.15 on the WFGY problem map).

what’s really happening is:

the infra assumes embedding output is "ready to go" during pipeline boot

but deepseek-v3 + sentence-transformers combo sometimes delays or returns a malformed placeholder vector (esp if model loading isn't 100% synced or warmed up)

result? downstream calls hit an uninitialized embed blob = silent death or ghost retry loop

it’s part of a broader class of failures where the system logic is fine on paper, but breaks silently unless infra is fully bootstrapped in advance.

fix is to either:

add a cold-start warmup cycle with dummy embeddings (can just call embed() once manually pre-pipeline),

or patch init to await embedding layer readiness if using async loaders.

if you’re mapping symptoms to root cause, this one matches No.15 – Deployment Bootstrap Collapse
doc ref (all mapped failure types):
https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md

just ctrl+F “predeploy” or check out the full map, you’ll probably find a few more traps hiding in your setup too. welcome to the trenches.

[shadowleaves]

ok thanks again I'll take a look