LibreChat API for external access to LLMs available in the LibreChat instance

[moritzfl]

What features would you like to see added?

An API to talk with the models available on LibreChat on a per user-account basis

More details

It seems like the request-schema that is used for the web-frontend to communicate with the backend is already fairly easy to understand. However, we still need to manage a user session and through OAuth before LibreChat this gets complicated rather quickly.

It would therefore be nice to have an official LibreChat-API that allows the linking of LibreChat with other software that wants access to a LLM (possibly with access through a LibreChat-API-Token, in the very best case compatible to OpenAIs API).

In a company setting, LibreChat could be the web-frontend that is offered to all employees.
Sometimes, the integration in another UI is desirable however. For coding, we want access within our IDE, for our technical writers it would be nice to have integration in their tooling too etc.

This would also have the following advantages:

• we could provide API access to all desired models without disclosing the LLM API keys (e.g. Azure or OpenAI API Keys) to employees.
• we can manage, switch out and easily try out multiple AI models
• we can easily integrate our AI-Plugins in a way so that they are accessible from the other UIs as well
• we have centralized user management that now also works for AI and can restrict token usage etc.

As an example, you could look at CodeGPT which is what we currently use for our developers:
https://github.com/carlrobertoh/CodeGPT
carlrobertoh/CodeGPT#426

Which components are impacted by your request?

General

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi thanks for checking out the project. This has been brought up a few times to me but as a full stack developer, this is concerning as it introduces a new potential attack vector. Also concerning: I really started seeing this request following counter measures taken in my work against users bypassing the frontend for backend access. While I'm sure your use case and others' are legitimate, it can often not be the case.

I would almost rather provide an "alternate" api that only exposes a few routes, mainly the chat stuff, and exclusively deals with API "keys" or w/e authentication would work best for this. Also, personally, this is not very high on my list of requests.

I'll keep this issue open so other users can "vote" on it.

[stevenbedrick]

I would like to "second" this idea; we are evaluating LibreChat as a way to allow groups of users to access various LLM services and it would be great if LibreChat could also act as an API endpoint.