New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: docker crash when inserts more data #32716
Comments
/assign @tadinhkien99 |
|
I did not see any critical errors when the milvus crash, I guess there is a OOM with the container. could you please double check that? @tadinhkien99 /assign @congqixia |
I'm aware that Docker can encounter Out of Memory errors, but in this instance, I was merely adding entities into the system without conducting any searches. What could be causing the OOM issue under these circumstances? Additionally, I need advice on the most effective index type for handling larger datasets. I have approximately 50 million entities to insert. I initially used IVF_PQ, but encountered an OOM error after inserting only 10 million entities. What would you recommend? |
|
Now I use gpu cagra index rtx 4090 24gb ram. And it's fine for 4M entities. |
what part you need top optimize with milvus.yaml? |
I want to use gpu index to save more and more entities. Around > 10M entities. |
you can use more GPU devices on single machine. I think the document already cover multi device use case https://milvus.io/docs/install_standalone-helm-gpu.md @Presburger can help if you hit any issue |
Is there an existing issue for this?
Environment
Current Behavior
When I insert upto 10M entities, docker crash then milvus disconnect. As I check because of cpu usage 100% and there no available RAM.
I use IVF_SQ8 index, each vectors 768 dimension.
I install milvus docker gpu version.
I use batchsize insert 10000 entities one time.
I think cpu and ram won't increase when we insert data?
Expected Behavior
Cpu and ram shouldn't OOM because only 10M entities
Steps To Reproduce
Milvus Log
...
Anything else?
...
The text was updated successfully, but these errors were encountered: