Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenAICompatibleChatExperiment class for enhanced LLM compatibility #125

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

agyaatcoder
Copy link

I have introduced a new class called OpenAICompatibleChatExperiment. This class aims to enhance the compatibility of prompttools with a wider range of large language models (LLMs) that support OpenAI-compatible inference endpoints. This enables usage of popular open source models hosted on Model as API service providers like together.ai, and self hosted models via Huggingface Text Generation Inference/ vLLM/ ollama etc, as they provide openai compatible rest api.

Key features and benefits of the OpenAICompatibleChatExperiment class:

Flexibility: The class allows users to specify the base URL and API key for any OpenAI-compatible inference endpoint, enabling seamless integration with various LLMs.
Ease of use: The class maintains a similar interface to the existing OpenAIChatExperiment class, ensuring a smooth transition for users familiar with prompttools.
Expanded LLM support: By leveraging OpenAI-compatible endpoints, prompttools can now interface with a broader ecosystem of LLMs, empowering users to explore and utilize different models effortlessly.

@CLAassistant
Copy link

CLAassistant commented May 1, 2024

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants