[Question]: Can I skip using indexing if the documents are very less?

[saidineshpola]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I just want to know whether we can call the query engine in RAG with out vectorstoreindex as I have very limited data for retrieval and it will be changed at every inference call
all_sentences=['Document1 data','Document2 data'] documents = [Document(text=sent) for sent in all_sentences] self.index = VectorStoreIndex.from_documents( documents, service_context=self.service_context) # Query the index with the prompt query_engine = self.index.as_query_engine()

[dosubot]

For your scenario with very limited and frequently changing data, you can indeed bypass using a VectorStoreIndex in RAG by leveraging alternative approaches such as a KnowledgeGraphRAGQueryEngine. This method is particularly useful as it allows for dynamic querying of a Knowledge Graph without the overhead of maintaining a vector index, making it ideal for small or frequently updated datasets.

To implement this, you would need to build or use an existing Knowledge Graph and then utilize the KnowledgeGraphRAGQueryEngine for querying. This approach dynamically generates responses based on the current data in the Knowledge Graph, thus eliminating the need for a pre-built vector index.

For more detailed guidance and examples on how to set this up, you can refer to the LlamaIndex documentation and example code provided in their repository. This will give you a clear understanding of how to leverage Knowledge Graphs for efficient information retrieval in scenarios like yours.

Details

Sources

• llama-index-networks/examples/demo/contributor-2/contributor_2/app_query_engine.py
• docs/docs/examples/query_engine/knowledge_graph_rag_query_engine.ipynb
• llama-index-core/llama_index/core/query_engine/knowledge_graph_query_engine.py
• llama-index-core/llama_index/core/indices/knowledge_graph/retrievers.py
• llama-index-packs/llama-index-packs-neo4j-query-engine/llama_index/packs/neo4j_query_engine/base.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.