[Request]: koboldcpp-cu12 (new version using CUDA 12)

[Spacellary]

Prerequisites

• I have searched all issues/PRs to ensure it has not already been reported or fixed.

Criteria

• Reasonably well-known and widely used (e.g. if it's a GitHub project, it should have at least 100 stars and/or 50 forks)
• English interface (or at least English documentation)
• Latest stable version
• Full version (i.e. not a trial version)
• Fairly standard install (e.g. uses a version-specific download URL, no elaborate pre/post install scripts)

Name

koboldcpp-cu12

Description

"A simple one-file way to run various GGML models like LLAMA, ALPACA, VICUNA. This version uses the newer CUDA 12 binaries. If you have a newer NVIDIA GPU and don't mind larger files, you may get increased speeds by using this new version."

Homepage

https://github.com/lostruins/koboldcpp

Download Link(s)

https://github.com/LostRuins/koboldcpp/releases/download/{latest-tag}/koboldcpp_cu12.exe

Eg.: https://github.com/LostRuins/koboldcpp/releases/download/v1.65/koboldcpp_cu12.exe

Some Indication of Popularity/Repute

3.9k stars
282 forks

Same regular install as the standard koboldcpp, just a new executable with newer cuda binaries.

[aliesbelik]

Guess this belongs rather to Versions bucket.

[Spacellary]

@aliesbelik You're absolutely correct. It totally escaped me. I'll move it over there.