[BUG] Qdrant - Creation of initial rule - conversion btw. multi & regular vectors failed

[tomthebuzz]

Describe the bug

Thanks for the latest and greatest update. Deployed everything and restarted my tests. However with the local qdrant instance I seem to still run in the same issue. I have a single PDF document which contains variable country names and the doc ingests without error. This is verified from the backend response and from the qdrant console.

To Reproduce

I have a single extraction request without any detailed rules: "Extract all the countries in the text"

Expected behavior

All (or at least) most of the countries listed in the doc / vector chunks should show up in the column

Logs and Error Reports

python3 -m uvicorn app.main:app
WARNING:dotenv.main:Python-dotenv could not parse statement starting at line 1
INFO:app.api.v1.endpoints.query:Query router initialized
INFO:     Loading config settings from the environment...
INFO:     Started server process [90890]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:app.services.llm.factory:Creating LLM service for provider: openai
INFO:app.services.vector_db.factory:Creating vector database service with provider: qdrant
INFO:app.api.v1.endpoints.document:Endpoint received file: RAStrat_SAKURA_ReLaunch_MC_LadyPad_0_5Drops_1gout_Confiance_20240822.pdf, content type: application/pdf
INFO:app.services.document_service:Created document_id: d40c8e7710dd4663875d1862bc14bcd9
INFO:app.services.loaders.factory:Creating loader of type: pypdf
INFO:app.services.loaders.factory:Using PyPDFLoader
INFO:app.services.document_service:Document split into 97 chunks
INFO:app.services.vector_db.base:Preparing 97 chunks
INFO:app.services.vector_db.base:Generating embeddings.
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:app.services.vector_db.qdrant_service:Upserting 97 chunks
INFO:httpx:HTTP Request: GET http://localhost:6333/collections/knowledge_table/exists "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: PUT http://localhost:6333/collections/knowledge_table/points?wait=true "HTTP/1.1 200 OK"
INFO:     127.0.0.1:54462 - "POST /api/v1/document HTTP/1.1" 201 Created
INFO:app.services.llm.factory:Creating LLM service for provider: openai
INFO:app.services.vector_db.factory:Creating vector database service with provider: qdrant
INFO:app.api.v1.endpoints.query:Received query request: {'document_id': 'd40c8e7710dd4663875d1862bc14bcd9', 'prompt': {'id': 'cm33710ez00013b6i54nrt03z', 'entity_type': 'country', 'query': 'Extract all the countries found in the text. Use full country names or ISO-3 country codes.', 'type': 'str', 'rules': []}}
INFO:app.services.vector_db.qdrant_service:Retrieving vectors for 1 queries.
INFO:app.services.vector_db.qdrant_service:Generating embedding.
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:app.services.vector_db.qdrant_service:Searching...
INFO:httpx:HTTP Request: POST http://localhost:6333/collections/knowledge_table/points/query "HTTP/1.1 400 Bad Request"
ERROR:app.api.v1.endpoints.query:Error processing query: Unexpected Response: 400 (Bad Request)
Raw response content:
b'{"status":{"error":"Wrong input: Conversion between multi and regular vectors failed"},"time":0.007499875}'
INFO:     127.0.0.1:54496 - "POST /api/v1/query HTTP/1.1" 500 Internal Server Error

Would appreciate your support to get this working with the local qdrant instance. Happy to explain more details if I can support in debugging.

[tomsmoker]

Thanks @tomthebuzz!

re: the above, I can't recreate it locally, and I added some tests for Qdrant too. Is this on an existing DB? Or are you starting from fresh? I only used new or local DBs

[tomthebuzz]

Using an existing instance but refresh all data. Will try with an empty qdrant instance next.

[tomthebuzz]

Have now completely re-installed local qdrant and used a new collection with the same setting. The embedding works but only for very select PDFs and not across all PDFs I have tried. The test to execute a first query however still fails with the same error of not being able to convert btw. multi and regular vectors:

INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:app.services.llm.factory:Creating LLM service for provider: openai
INFO:app.services.vector_db.factory:Creating vector database service with provider: qdrant
INFO:app.api.v1.endpoints.document:Endpoint received file: RAStrat_SAKURA_ReLaunch_MC_LadyPad_0_5Drops_1gout_Confiance_20240822.pdf, content type: application/pdf
INFO:app.services.document_service:Created document_id: 14adebbb399f4c0aaa5a488070b74a04
INFO:app.services.loaders.factory:Creating loader of type: pypdf
INFO:app.services.loaders.factory:Using PyPDFLoader
INFO:app.services.document_service:Document split into 97 chunks
INFO:app.services.vector_db.base:Preparing 97 chunks
INFO:app.services.vector_db.base:Generating embeddings.
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:app.services.vector_db.qdrant_service:Upserting 97 chunks
INFO:httpx:HTTP Request: GET http://localhost:6333/collections/knowledge_table2/exists "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: PUT http://localhost:6333/collections/knowledge_table2 "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: PUT http://localhost:6333/collections/knowledge_table2/points?wait=true "HTTP/1.1 200 OK"
INFO:     127.0.0.1:61172 - "POST /api/v1/document HTTP/1.1" 201 Created
INFO:     127.0.0.1:61342 - "OPTIONS /api/v1/query HTTP/1.1" 200 OK
INFO:app.services.llm.factory:Creating LLM service for provider: openai
INFO:app.services.vector_db.factory:Creating vector database service with provider: qdrant
INFO:app.api.v1.endpoints.query:Received query request: {'document_id': '14adebbb399f4c0aaa5a488070b74a04', 'prompt': {'id': 'cm349meum00343b6izvxdd2sq', 'entity_type': 'country', 'query': 'which countries are mentioned in the document', 'type': 'str_array', 'rules': []}}
INFO:app.services.vector_db.qdrant_service:Retrieving vectors for 1 queries.
INFO:app.services.vector_db.qdrant_service:Generating embedding.
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
INFO:app.services.vector_db.qdrant_service:Searching...
INFO:httpx:HTTP Request: POST http://localhost:6333/collections/knowledge_table2/points/query "HTTP/1.1 400 Bad Request"
ERROR:app.api.v1.endpoints.query:Error processing query: Unexpected Response: 400 (Bad Request)
Raw response content:
b'{"status":{"error":"Wrong input: Conversion between multi and regular vectors failed"},"time":0.006248125}'
INFO:     127.0.0.1:61342 - "POST /api/v1/query HTTP/1.1" 500 Internal Server Error

Maybe you have an idea on how to test with an even more limited set of test data (e.g. txt only and no PDF / though the embeddings seem fine in the qdrant console).

[tomsmoker]

I can run it with a variety of docs, so I'm trying to reverse engineer. How are you running qdrant? With docker?

[tomthebuzz]

Yes, running with docker and the default config. With the one working PDF from the example above it is creating the new collection and creating qdrant points for every document chunk. I also can query and traverse the points in the collection as expected. Qdrant runs in local docker on standard port.

[tomsmoker]

So it's additional documents of the same type that cause the problems?

[tomthebuzz]

Yes, it is PDF text (not OCR) documents. Also tried with plan vanilla .txt and .csv That piece seems to be a frontend issue as neither the frontend nor the backend logs any issue when I try to add a new document - neither activity nor error or warning. With the one document it works without any issue for both front & backend. Will try with a new arc browser install tomorrow to see if I can get out more details that way. Cheers -Tom

…

_______________________________________ [cid:CB4A45798F0346E3B0AD4A3C2ABFA321] Ferris Labs AG Fabrikstrasse 5 6330 Cham Switzerland Tom Debus Founder & CTO +41 79 335 38 42 ***@***.******@***.***> On 8 Nov 2024 at 19:19 +0100, Tom Smoker ***@***.***>, wrote: So it's additional documents of the same type that cause the problems? — Reply to this email directly, view it on GitHub<#64 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADDGZKXHOG374NEMWRJL3A3Z7T6BLAVCNFSM6AAAAABRESMRDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVGQ3TSNZQGM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[tomsmoker]

Thanks Tom. I really want to get this resolved but still working for me. Can you share the docs with me so I can try? Ok if not, just want to have all the details so I can dig in.

[recchris]

Hey @tomthebuzz checking in here. Were you able to get this working? If not, are you able to share a sample pdf/txt doc with us so we can try to recreate the error on our side? Thx!

[tomthebuzz]

Hi @recchris, thanks for your response. No, I was not able to get this working and it is now a situation where no file whatsoever loads. So you can literally try any pdf or txt file. Or tell me where to look for additional debugging input. Do you expect the backend to provide a valid and working API endpoint and what would be the correct URL for this endpoint? Because with all the obvious ones I tried I get a 404 API response. But all the tests have completed successfully and the startup log shows no errors. Cheers -Tom On 17 Dec 2024 at 06:12 +0100, recchris ***@***.***>, wrote: Hey @tomthebuzz<https://github.com/tomthebuzz> checking in here. Were you able to get this working? If not, are you able to share a sample pdf/txt doc with us so we can try to recreate the error on our side? Thx! — Reply to this email directly, view it on GitHub<#64 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADDGZKWHFSA6RQWU4NIBAYT2F6XCPAVCNFSM6AAAAABRESMRDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBXGUZDENBVGI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[recchris]

@tomthebuzz is this only happening for Qdrant? Can you verify whether or not this is working using the default Milvus config? Tagging @Anush008 who may be able to provide some input on the Qdrant errors you've pasted above. Any thoughts on what Tom is posting here?

Wrt API endpoints, you can see the most up to date APIs at http://localhost:8000/docs when you run the app. Which 404s are you seeing? Can you please paste here?

Could you also please confirm that you have added the following to your .env file:

QDRANT_LOCATION=
QDRANT_PORT=
QDRANT_GRPC_PORT=
QDRANT_PREFER_GRPC=
QDRANT_URL=
QDRANT_HTTPS=
QDRANT_API_KEY=
QDRANT_PREFIX=
QDRANT_TIMEOUT=
QDRANT_HOST=
QDRANT_PATH=