Vector Dimension Issue While Querying #8142
-
|
I have created an index in Qdrant using Instructor embeddings with 768 dimensions. However, I am running into issues while trying to query it. Here is my code: ` client = qdrant_client.QdrantClient(url='http://localhost:6333', prefer_grpc=True) llm = ChatOpenAI(model_name=gpt3_5_turbo_model_name, index = VectorStoreIndex.from_vector_store(vector_store=vector_store, retriever = VectorIndexRetriever( service_context = ServiceContext.from_defaults( response_synthesizer = get_response_synthesizer( query_engine = ( Write query answerst.markdown("### Answer:") result = query_engine.query(query) Display the resultsst.write(f"Answer: {str(result)}") Here is my error: Traceback (most recent call last): For some reason, it is assuming embedding size of 1536 instead of 768 and was wondering if it is set to OpenAI default somewhere. Any ideas? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
|
🤖 Hello, The error message you're seeing is due to a mismatch between the expected and actual dimensions of the embeddings. In your case, the Qdrant index is expecting embeddings of size 768, but it's receiving embeddings of size 1536. This mismatch is likely because the size of the vector in Qdrant is determined by the first node that is added to the collection. If the first node that was added to the collection had an embedding size of 1536, that would set the vector size for the entire collection to 1536. If you want the vector size to be 768, you need to ensure that the first node added to the collection has an embedding size of 768. This behavior is evident from the Here is the relevant code from the if len(nodes) > 0 and not self._collection_initialized:
self._create_collection(
collection_name=self.collection_name,
vector_size=len(nodes[0].get_embedding()),
)And from the if len(nodes) > 0 and not self._collection_initialized:
await self._async_create_collection(
collection_name=self.collection_name,
vector_size=len(nodes[0].get_embedding()),
)You can find this code in the QdrantVectorStore class in the LlamaIndex repository. To resolve this issue, you might need to recreate your Qdrant index ensuring that the first node added has an embedding size of 768. I hope this helps! Let me know if you have any other questions. Sources
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. |
Beta Was this translation helpful? Give feedback.
-
|
I deleted and recreated the index in Qdrant and ensured all 333 entries are of 768 dimensions. When I use the following code when the index is initialized from the documents versus from the vector store, the query code from above works as expected with no errors about vector dimensions. However, when I move back to the following code to initialize the index from Qdrant, I get the same 1536 dimension error I initially reported. Any other ideas? |
Beta Was this translation helpful? Give feedback.
-
|
@hodgesz you need to pass in the service context when loading the index (with the service context setup with the same embedding model you used to create the index) |
Beta Was this translation helpful? Give feedback.
@hodgesz you need to pass in the service context when loading the index (with the service context setup with the same embedding model you used to create the index)