[Question]: Which Open Source LLM is best for Knowledge Graphs ?

[nikhitaKanoj]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I want to understand Which Open Source LLM is best for Knowledge Graphs ?
Because I am using KnowledgeGraphIndex functionality in llama-index and creating Knowledge graphs ,
I used Nous-hermes-Llama2 , the accuracy of the results is not that appreciating ,
Can you please suggest which open source current LLM is good for this ?

[dosubot]

For enhancing Knowledge Graphs with high accuracy and efficiency, transitioning to the Llama Index framework, especially its Knowledge Graph RAG Query Engine, is recommended. This framework is well-suited for creating and querying Knowledge Graphs, leveraging Large Language Models like OpenAI's GPT-3.5-turbo and Azure's GPT-3.5-turbo-16k. It utilizes NebulaGraphStore for graph storage, which, in combination with the Knowledge Graph RAG Query Engine, facilitates efficient data storage and querying.

The Llama Index framework significantly improves upon Nous-hermes-Llama2 by supporting embedding models and enabling a hybrid querying approach that combines text and embeddings. This approach offers a substantial upgrade in terms of accuracy and user experience for Knowledge Graph applications.

For practical implementation and further exploration, consider reviewing the following resources:

• Knowledge Graph Query Engine
• Neo4jKGIndexDemo
• NebulaGraphKGIndexDemo
• Knowledge Graph RAG Query Engine

These resources should guide you in leveraging the Llama Index framework for your Knowledge Graph projects effectively.

Details

Sources

• [Question]: Knowledge Graph Vs RAG
• docs/docs/examples/query_engine/knowledge_graph_query_engine.ipynb
• docs/docs/examples/index_structs/knowledge_graph/Neo4jKGIndexDemo.ipynb
• docs/docs/examples/index_structs/knowledge_graph/NebulaGraphKGIndexDemo.ipynb
• docs/docs/examples/query_engine/knowledge_graph_rag_query_engine.ipynb
• llama-index-legacy/llama_index/legacy/indices/knowledge_graph/base.py
• llama-index-core/llama_index/core/indices/knowledge_graph/base.py
• llama-index-integrations/graph_stores/llama-index-graph-stores-nebula/llama_index/graph_stores/nebula/base.py
• llama-index-core/llama_index/core/indices/knowledge_graph/retrievers.py
• llama-index-legacy/llama_index/legacy/indices/knowledge_graph/retrievers.py
• docs/docs/examples/index_structs/knowledge_graph/KnowledgeGraphDemo.ipynb
• docs/docs/examples/index_structs/knowledge_graph/knowledge_graph2.ipynb
• CHANGELOG.md
• docs/docs/CHANGELOG.md

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[nikhitaKanoj]

@dosu , I am using llama-index only for my application with RAG , I want to know which OPEN SOURCE llm to us with this so that the results are better ,
Current Components I am using : KnowledgeGraphIndex - for indexing , SimpleGraphStore()- for graph store , Nous Hermes llama2 13B - llm , all mini - text embedding .
With these , the results are not so accurate,
I want to know how to increase its performance and which LLM is better ?

[dosubot]

To enhance the performance and accuracy of your Knowledge Graph application using the llama-index framework, consider the following steps:

1. Switch to IpexLLMEmbedding for Text Embedding: The IpexLLMEmbedding is optimized for Intel hardware, potentially offering better efficiency and performance. The model BAAI/bge-large-en-v1.5 used here could provide improvements over the Nous Hermes llama2 13B model. Explore its integration as detailed in the basic.py example.

2. Benchmark the BAAI/bge-large-en-v1.5 Model: Test this model's performance in your application to see how it compares with your current setup. Focus on accuracy and efficiency, especially if you're using Intel hardware where you might see significant gains.

3. Utilize Device-Specific Optimizations: The efficiency improvements from IpexLLMEmbedding could be substantial, especially on Intel platforms. Experimenting with different hardware setups might reveal the most cost-effective configuration for your application.

4. Leverage Advanced Features: The IpexLLMEmbedding offers features like batch processing for text embeddings, which could improve the processing efficiency of your Knowledge Graph application. Make sure to integrate these features to fully benefit from the model's capabilities.

5. Continuous Benchmarking and Iteration: Keep testing different models and configurations. Continuous benchmarking will help you identify the best setup for your specific needs, balancing accuracy, efficiency, and resource consumption.

6. Share Your Findings: If you achieve significant improvements, consider sharing your experience and insights with the community. This can help others facing similar challenges and contribute to the development of the ecosystem.

For optimizing the KnowledgeGraphIndex, explore strategies discussed in community threads such as optimizing triplet extraction and parallelizing processes. Remember, these optimizations might impact result quality, so thorough testing is crucial to ensure they meet your application's requirements.

Details

Sources

• [Question]: How to increase the performance of extracting triples from words by using Knowledge Graph Index
• [Tracking]: Better support for Local Models
• [Question]: How can I improve responses?
• [Question]:
• KnowledgeGraph building using KnowledgeGraphIndex is very slow
• [Feature Request]: Tools for tuning retrieval hyperparameters
• llama-index-core/llama_index/core/indices/knowledge_graph/base.py
• llama-index-legacy/llama_index/legacy/indices/knowledge_graph/base.py
• llama-index-integrations/embeddings/llama-index-embeddings-ipex-llm/examples/basic.py
• llama-index-integrations/embeddings/llama-index-embeddings-ipex-llm/llama_index/embeddings/ipex_llm/base.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.