-
Notifications
You must be signed in to change notification settings - Fork 813
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About using multiple GPUs to do lisa fine-tuning #774
Comments
Thanks for your interest in LMFlow! Currently we are working on the full multi-GPU support for LISA. Model parallelism is not integrated yet. If you run the script directly, it is data parallelism and may require more memory than its single GPU version. Please stay tuned for our latest update, thanks for your understanding 🙏 |
Can other training methods be configured with multiple GPUs? Do I need to set parameters manually? |
When will Lisa training’s support for multiple GPUs be updated? |
Yes. You may use |
If there are multiple GPUs, using the lisa method is also a direct script./scripts/run_finetune_with_lisa.sh? Do I need to set multi-GPU parameters?
The text was updated successfully, but these errors were encountered: