What's the best way to index millions of data records?

[liquidpeachy]

If I have a dataset with millions of records, what would be the best way to index in Meilisearch?

[curquiza]

Hello @liquidpeachy
Since you are indexing millions of documents, I recommend reading this guide to ensure you optimize the indexation time: https://docs.meilisearch.com/learn/advanced/indexation.html#improving-indexation-performance 🙂

[qumberrizvi]

Returns 404.

[Kerollmops]

Sorry for that. Here is a blog post explaining how to index a lot of documents.

Change your settings beforehand to avoid reindexing the whole dataset when you change them. Send the documents in the biggest batches you can (default limit of 150MiB). Do not delete before inserting the newest version of your documents but rather send updates directly. Use the latest version if you want to see the best performances.