Question regarding query word matching

[hassanacto]

Hi all. I read the docs and saw that there's a warning that says that "every word must be present in the returned document."

Just wanted to confirm, is it possible to still return results where most of the words are present? And have that sorted?

My use case right now is: take a search query and return the most relevant documents. It's not necessarily based on exact keyword matching.

(Sorry if this is the wrong place to post. Didn't know where else to)

[tpayet]

Hello,
This is the place for these question! The engine is currently returning documents that contains all the words in the query (except stop-words) because we built the search for search-as-you-type.
The user will begin to type its query with the beginning of its word and MeiliSearch should be able to give him the best results as soon as possible.

Are you building more than a simple search bar?

We thought about adding some settings to allow the search to return documents that does not return all words, but it is quite complicated since the engine is built for search engine.

[hassanacto]

Ah that makes sense. I don't think it would fit my use case then.

That being said, I loved how quickly I could get set up for basic search, so I'll definitely incorporate this as part of my other projects.

Awesome work with the product!

[thijndehaas]

We are using Meili for a system and it works great! Now I want people to find tutorials on another website but the problem is that everybody types human questions like "how to upload a plugin". It would be nice to match just partially, in our case on "upload "and "plugin".

[thijndehaas]

Yes this could be an option. My next thing would be a support ticket field that automatically searches for tutorials. With the current meilisearch functionality I can just send the words that actually exists in Tutorial keywords to Meilisearch to create a decent sorting.

[tpayet]

I am not sure I completely understand what you want to build, what is a support ticket field?

[thijndehaas]

When users create a new support ticket in our system we would like to show possible relative tutorials depending on their support ticket message. Many of the support tickets we receive now are litteraly described in a tutorial.

[tpayet]

Oh okay I get it, this is more complexe and maybe it would look like a more like this feature that was being discussed on #466
The difficulty in your case would be to extract the keyword from the tickets 🤔

[hassanacto]

@thijndehaas you can implement your own version of a TFIDF search to show relevant tutorials. The data can be stored relationally and is very easy to query. The process I'd recommend is this:

1. Extract all unique words from your tutorial
2. Remove stop-words from that list
3. Store these keywords in a table (id, keyword)
4. Generate a TF-IDF for that keyword and particular document combination (the higher the score, the more relevant the document) -> table structure can be something like (id, keyword_id, tf_idf_score, tutorial_id)
5. Whenever a user asks a question, you take the keywords from that question, remove stop words, and search your table to get the most relevant documents for those keywords.

If that's your use case, it doesn't make sense to overcomplicate it by adding in another search service. I know elasticsearch does TFIDF and some other mechanisms for relavancy sorting too, but TF-IDF by itself is pretty good too.

More reading here: http://www.tfidf.com/