LlamaIndex Integration

[steventkrawczyk]

🚀 The feature

We need an experiment or example for LlamaIndex. This will be similar to the work we need to do for LangChain and MindsDB.

Motivation, pitch

LlamaIndex is a popular framework for connecting LLMs to data. It would be good to support LlamaIndex testing using our existing vectorDB and LLM evaluations.

Alternatives

N/A

Additional context

N/A

[rachittshah]

hi @steventkrawczyk I'd love to work on this.

My guess is we'd need abstractions over LlamaIndex over core prompttools classes, but we'll need to update it with each release of LlamaIndex.

[steventkrawczyk]

Hey @rachittshah we'd love your help! I think you're right that we'll need to build prompttools abstractions on top of the llamaindex abstractions. I'm hoping there are a few abstractions which are stable enough for us to build once, but if we need to update with each release we can explore that too.

Feel free to open a draft PR with what you're thinking now, and we can work together to iterate. If you want to chat in depth, I can add you to the dev channel of our discord, and we can set up a call to brainstorm if you'd find it useful.

Discord invite: https://discord.gg/72a9xh5Z7P

[rachittshah]

Hi @steventkrawczyk ! Joining discord, I think I'd open a draft PR with a proposed implementation of changing(since this would be a breaking change).

Joining discord, and keen to work on this!

[baggiponte]

+1 on this feature 😊 A client I work uses llama-index and there aren't notable libraries for LLM eval that are compatible. Yesterday deeplearning.ai released a course on RAG and LLM Eval using llama-index and truelens but I feel that promptools design would feel more natural and provide a simpler integration.