Add support for more sizes of LLaMA

[cnbeining]

Hey folks,

Trying to get 13/30B model with 4 bit fine tuning - any chance you folk could release the script used to convert the 7B version of model to 4 bit?

Thanks,

[shreyansh26]

That would be really helpful!

[shreyansh26]

I think this is the standard repo which people use - https://github.com/qwopqwop200/GPTQ-for-LLaMa

[StochasticRomanAgeev]

Hi @cnbeining @shreyansh26,
We have contributing guide for model addition
We will appreciate any help on this field!

[StochasticRomanAgeev]

Hi again, @cnbeining @shreyansh26
In latest release we added Generic model, you can use it for llama-13b models in our library!
Also maybe a good option for you will be falcon-7b model, that is now supported by xturing.

[StochasticRomanAgeev]

We are also working on addition kbit quantisation to generic model, so it should be released soon.
Then you will be able to use llama-13b or any model with 4bit quantization.