-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Supporting device_map = 'auto' similar to the one in .from_pretrained method from Huggingface #36
Comments
Hi. It would be challenging to do this since the pruned model does not follow a uniform configuration, like different dimensions for different modules and different head numbers for different layers. |
Thank you. Do you suggest any similar methods that can enable distributed training while working with the Train method from Huggingface? |
I am interested in this as well. Is it possible to implement it on Llama2-70b? Or use some distributed method to prune the model? |
The pruned model is saved using torch.save and torch.load for loading the model. I was wondering if there is a way to use a similar method such as device_map='auto' similar to the one in .from_pretrained method from Huggingface
The text was updated successfully, but these errors were encountered: