Other backend (koboldcpp) #669
neph1
started this conversation in
Show and tell
Replies: 1 comment
-
Have been working some more on this and finally I get something sensible out of the ai. Now I will get to work on it using tools. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I'm mostly interested in using local llm's, and since I'm using koboldcpp (ggml) the "built in" gui doesn't support it, I set out to add it.
I don't really understand the reason to bundle twgui when all you could do is launch it separately? Seems like unnecessary lock-in and maintenance.
It's a small change, but not ready for a PR, yet, because I haven't been able to get v0.0.7 working. Something about superagi is also eating up all my HDD space, and I haven't been able to find out what.
I also don't know if this is of interest for the project.
The branch is here, if anyone wants to have a look: https://github.com/neph1/SuperAGI/tree/koboldcpp
In any case, my changes should make it easier to connect to other backends, should the user want to do so, and it's relatively easy to add more classes.
The section is called 'show and tell', so here's a 'show':
I had limited it to 30 tokens for test purposes. There may be some further work needed with the prompt formatting, as all of it is submitted as one string currently.
Ideas are welcome!
Beta Was this translation helpful? Give feedback.
All reactions