-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
使用milvus数据库,初始化数据库的时候,显存爆炸。 #3971
Comments
The CUDA out of memory error you're experiencing is likely due to the high computational demands of indexing and searching with Milvus, especially with large datasets or complex vector embeddings. To mitigate this, consider processing data in smaller batches, adjusting Milvus's index parameters for efficiency, and configuring the Regarding the For utilizing multiple GPUs, ensure your setup supports distributed processing, which can help manage memory usage more effectively across GPUs.
|
model config里embedding模型用的cuda吗?或者是auto? |
This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
return self.fget.get(instance, owner)()
Batches: 0%| | 0/396 [00:00<?, ?it/s]
2024-05-08 13:41:01,091 - embeddings_api.py[line:39] - ERROR: CUDA out of memory. Tried to allocate 17.93 GiB. GPU 0 has a total capacty of 23.65 GiB of which 1.62 GiB is free. Including non-PyTorch memory, this process has 22.02 GiB memory in use. Of the allocated memory 21.55 GiB is allocated by PyTorch, and 15.99 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
AttributeError: 'NoneType' object has no attribute 'conjugate'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/init_database.py", line 107, in
folder2db(kb_names=args.kb_name, mode="recreate_vs", embed_model=args.embed_model)
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/server/knowledge_base/migrate.py", line 128, in folder2db
files2vs(kb_name, kb_files)
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/server/knowledge_base/migrate.py", line 113, in files2vs
kb.add_doc(kb_file=kb_file, not_refresh_vs_cache=True)
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/server/knowledge_base/kb_service/base.py", line 131, in add_doc
doc_infos = self.do_add_doc(docs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/server/knowledge_base/kb_service/milvus_kb_service.py", line 83, in do_add_doc
ids = self.milvus.add_documents(docs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zwm/miniconda3/envs/Langchain-Chatchat2/lib/python3.11/site-packages/langchain_core/vectorstores.py", line 119, in add_documents
return self.add_texts(texts, metadatas, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zwm/miniconda3/envs/Langchain-Chatchat2/lib/python3.11/site-packages/langchain_community/vectorstores/milvus.py", line 531, in add_texts
embeddings = self.embedding_func.embed_documents(texts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/server/knowledge_base/kb_service/base.py", line 439, in embed_documents
return normalize(embeddings).tolist()
^^^^^^^^^^^^^^^^^^^^^
File "/home/zwm/Code_Program/Chatchat/milvus-Langchain-Chatchat/server/knowledge_base/kb_service/base.py", line 37, in normalize
norm = np.linalg.norm(embeddings, axis=1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zwm/miniconda3/envs/Langchain-Chatchat2/lib/python3.11/site-packages/numpy/linalg/linalg.py", line 2582, in norm
s = (x.conj() * x).real
^^^^^^^^
TypeError: loop of ufunc does not support argument 0 of type NoneType which has no callable conjugate method
有多张显卡,不知道可以一起共用不,初始化数据库的时候,为什么需要那么大显存,是程序的问题吗?
The text was updated successfully, but these errors were encountered: