custom api

[SubGlitch1]

is this how u want it? @psyray

[yogeshojha]

What is this related to? Please provide more context @SubGlitch1

[psyray]

@yogeshojha related to #1114
@SubGlitch1 👍
But, please, document explicitely your changes

[SubGlitch1]

@yogeshojha
I had seen a lot of people wanting to be able to use their own custom/local LLms. This can be done quite easily by changing the code to include an env variable which the gpt.py treats as the base url if it exists.

There are a growing number of openai compatible api server on github one of the most known ones being text-generation-ui. By using a project like this a user can host their own LLM api server which has the exact same scheme as the official openai api.

This obviously greatly increases privacy as the information of targets doesn't get sent to openai anymore. It also makes the gpt feature free as you don't have to pay for the api tokens anymore obviously. It also enables users who live in geo blocked areas to use the gpt feature.

[yogeshojha]

Thanks @SubGlitch1 , do all these api have same response? Can you provide me the response for text-generation-ui

[SubGlitch1]

@yogeshojha text-generation-webui has the exact same scheme of openai. The request/response scheme will be exactly the same. I have tested on my own PC
Test details
OS: amd64 arch Linux
llm: openchat3.5_Q3

[SubGlitch1]

@yogeshojha

https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API#sse-streaming

You can see the examples here. A lot of projects which use the openai api e.g. gpt-engineer also have options to change the base api url for the exact same reason.

[psyray]

@SubGlitch1 I didn't know the subject, but copilot is usable with your modifications ?

[SubGlitch1]

@psyray the issue you are referencing has been opened because the op wants to get gpt4 access for free. His suggested integration is not possible as copilot isn't open source but by integrating my solution one could host any model that they want for free.

[psyray]

@psyray the issue you are referencing has been opened because the op wants to get gpt4 access for free. His suggested integration is not possible as copilot isn't open source but by integrating my solution one could host any model that they want for free.

Thanks 👍