Is MeiliSearch suitable for 1.5million documents?

[rohitagarwal88]

We have around 1.5million documents (full HTML page). I can extract data in json format and would have around 150 key, value pairs.

Our DB will grow over a period of time.

Would MeiliSearch be a suitable option?

[tpayet]

Yup, MeiliSearch should works well :)
You can read more about storage on our documentation.

But do you really need to have your 150 fields searchable? Because the more attributes you have, the more difficult it will be to tune relevancy ^^

[devrck]

I would be curious to see how good can it be with 1.5m documents 😄

[rohitagarwal88]

@tpayet : I have indexed around 7.5k documents on 1gb, 1core server. The search time was 750ms but the time reduced drastically to 250ms after adding 5 synonyms for each document.
Still, I feel the time is too high. How could I optimize this? any suggestions.

[devrck]

Any idea of adding more RAM, CPU can boost performance?

https://meili-react-demo.netlify.app/

In this demo i think they have loaded ~7k products(https://github.com/Giorat/meili_react_demo/blob/master/backend/decathlon.json) and response time is around 50ms

[rohitagarwal88]

I have own documents and document size is much much bigger than that in demo app.
Increasing Ram or CPU is not the solution. That can always be done.

[devrck]

Maybe tune:

• which attributes to be displayed when you get back the response
• trim attributes down when you get back the response
• implement stop words for your language
• configure which attributes you want to be searchable

It also depends how you do the search? (only using ?q=search term or you do some filtering/faceting).

Also maybe worth looking at meilisearch/documentation#328.

I was just curious if CPU/RAM/SSD is a solution in your case and then start tuning the settings since to me it's not clear 😄 what is the issue.

From https://docs.meilisearch.com/resources/about_storage.html

You should have the same amount of RAM than the space taken on disk by MeiliSearch for optimal performances.

More in depth https://docs.meilisearch.com/resources/about_storage.html#memory-usage

Disclairement: I'm just a person who started using it and want to see more use cases and best practices, sorry if I'm not that helpful

[eskombro]

Thanks @devrck, your answer is quite complete!

@rohitagarwal88, out of curiosity, those times include the network delay (the whole search request flow) or just the MeiliSearch processing time?