How to do Hybrid search : keyword search + semantic search + reranking ?

[snbhanja]

Is your feature request related to a problem? Please describe.
How do I do a keyword search? I can see there is a full-text search, but it doesn't work for a partial search.

Describe the solution you'd like
There is an article that explains how to hybrid search, keyword search from meilisearch + semantic search from Qdrant + reranking using the cross-encoder model.

This is fine, I am able to implement this. But, is there a way if it's already available, or there is a plan to do this using Qdrant only?

Describe alternatives you've considered

1. meilisearch + qdrant + Sbert crossencoder model.
2. Combining results from BM25 and Qdrant.

[kacperlukawski]

@snbhanja It's impossible to be done with Qdrant. Each problem should be solved separately, requiring different tools, the best for a specific scenario. We may consider making some code snippets public so it's easier to start. We have planned to support sparse vectors in our roadmap, but that won't be an end-to-end BM25-like tool.

If you want to have BM25, Meilisearch is not an option, as they have their algorithm instead. You may consider some other tools, like lnx.

[nickprock]

Hi @kacperlukawski,
maybe I'm off topic but is possible to apply only rerank step? (Qdrant Search + Cross Encoder).
Can you recommend any resources where this is discussed?
Thanks in advance.

[kacperlukawski]

Hi @nickprock, Here are some notes on the hybrid search: https://qdrant.tech/articles/hybrid-search/. You can skip the BM25 and only apply the reranking on the vector search results. However, we haven't compared that option. That sounds interesting, though :)

[ogencoglu]

Are your previous responses still valid @kacperlukawski ?

I guess LlamaIndex tutorial talks about final reranking tricks instead of native hybrid search: https://docs.llamaindex.ai/en/stable/examples/vector_stores/qdrant_hybrid.html