implementation of Exllama #219
LitenBuzzTh
started this conversation in
General
Replies: 1 comment
-
Thats a great idea. Will look into that. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi
recently oobabooga text generation webui have implemented Exllama that replaces AutoGPTQ for quantized models. the benefit of Exllama is that it can increase the generation speed up to 4x (with same system resources!).
it would be nice to see that in localGPT if possible.
Beta Was this translation helpful? Give feedback.
All reactions