is it possible to use it with llama.cpp

[kroryan]

so i would like to know if is possible to use it with llama.cpp using ./server and listening port 5001, i tried and didnt work

[neph1]

It should be, it's just not configured.
Try sticking to the koboldcpp config, but change "ENDPOINT" to "/completion/"
https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md

[kroryan]

so i have to open the llm_config.yaml and change "/api/v1/generate" to "/v1/chat/completions" ? or where do you mean to change it?

[neph1]

Yes, but just to "completion" according to that doc.

[kroryan]

I wrote /completion as endpoint and didnt work i will try only with completion

[neph1]

I'm not familiar with llama.cpp, I'm just guessing from the documentation. But I see no reason why it shouldn't work.

[neph1]

Not sure if you solved this, but I just added a guide to the wiki: https://github.com/neph1/LlamaTale/wiki/Configuring-llama.cpp-server