Does the query item works well for all dimension size embeddings?

[OnlinePage]

Hi , first of all a big thanks for Vectra 🥺!

Now context for the question is that , i tried vectra with general embeddings from openai embedding model like text-embedding-3-small and its dimension is 1536 and the result from the query are really good!!

Now i tried to switching to other embedding model with dimension 768, now this time results from the query weren't of good quality.

As i see , the queryItems uses cosine similarity. So does it affects the results🤔.
Or there's any better way to get better results from varying dimension embedding.

[Stevenic]

Thanks! There are a ton of alternatives to Vectra these days but it's still pretty good at what it does.

As for your question, it's likely going to depend on the quality of the embeddings used... The ability to specify the dimensions of your embeddings is a feature I recently added so I haven't used it a lot yet. I'm using it with nomic-embed-text-v1.5 via llama.cpp to get my embedding size down to 128 dimensions and the results seem ok from my testing. But my testing has been very adhoc...

Cosine similarity is the algorithm used to do the matching and is independent of the actual embeddings used. You're quality results are going to be driven largely by the embeddings model you use and the number of dimensions you ask for. There's a feature I could add called re-ranking which would help improve the results across the board but it would involve adding a keyword index into the mix which both complicates things and somewhat goes against the spirit of Vectra (simple, lightning fast, and free.)

[Stevenic]

So we just just switched to just switching to using a more modern tokenizer library that supports multiple dimension embeddings It would be great to either extend the current tokenizer to let you specify both the algorithm and number of dimensions.

If someone want to tackle that I have some other tips for how to dramatically improve the text splitting speed.