Quantizing LLM to GGML or GUFF Format: A Comprehensive Guide #533
SiraHaruethaipree
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
{{ message }}
-
I would like to know about the detail, How to quantize LLM to GGML or GUFF format. Currently, I found few referent that describe about this e.g. https://github.com/rustformers/llm/blob/main/crates/ggml/README.md. In constant GPTQ have the referent paper that explain about the detail.
So have any website,blog suggest that describe about technique quantize GGML format.
Thank you for your help. :)
Beta Was this translation helpful? Give feedback.
All reactions