Use LibreChat to connect with LangChain?

[bweinlich]

First of all: Thank you for your amazing Work Here!

I was playing around a bit with LangChain apps and I wonder If I can Connect my favorite Chat Client with a LangChain App. I think it would be necessary to expose an OpenAI compatible REST endpoint in order to connect with LibreChat.

Ist it possible to do so?

[danny-avila]

Thanks for checking out the project! Yes, if it’s OpenAI compatible, it can be used without changing any code. See the documentation on the custom config file and custom endpoints. I’ll send links in a few minutes.

[danny-avila]

https://docs.librechat.ai/install/configuration/ai_endpoints.html

https://docs.librechat.ai/install/configuration/custom_config.html

[bweinlich]

Thankyou very much!

Does anybody know how to expose an OpenAI compatible API with LangChain? LangServe does not seem to Support it.

Another Idea would be to embedd my LangChain App into a Tool that can bei registered as a LibreChat Plugin. Is this possible?

[ndrsfel]

I had this idea several times but didn’t manage to find an example, until now. Besides LibreChat plugins using Langchain JavaScript library, a Python alternativ requires a Flask or FastAPI backend that wraps the OpenAI standard and handles the Langchain implementation.

For the wrapper, I’ve found
https://github.com/Azure-Samples/openai-chat-backend-fastapi
that wraps the OpenAI standard in FastAPI ✅.

Muissibg is the Langchain integration, preferably with streaming support, but I think that’s doable with the template.

Let me know what you think 🐰🪺

[ndrsfel]

Also helpful: HTTP protocol for AI chat apps aka. OpenAI standard

[bweinlich]

Hey, thank you for your response! However, I don't understand how this could work. My problem is:

LibreChat (needs OpenAI-API) --> SOME_ADAPTER --> My LangChain App with LangServe REST-API exposed (https://python.langchain.com/docs/langserve#endpoints)

How can I make them compatible?

[Waidhoferj]

Hey @bweinlich I had the same question! Here's a working example with some setup directions in the README. Let me know if you have any questions.

[bweinlich]

Thankyou! I will try it out. Meanwhile I used LibreChat just for the vanilla OpenAI GPT-4 endpoint as a cheap alternative to an OpenAI Pro account and used the 'nlux' library for creating my own frontends for my models!

But maybe I'll switch back then :)