-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can peft support ColumnParallelLinear? #1711
Comments
So I assume you're using megatron. Did you try this: https://huggingface.co/docs/peft/v0.10.0/en/package_reference/lora#peft.LoraConfig.megatron_config Here is an example: https://github.com/huggingface/peft/blob/main/tests/test_lora_megatron.py |
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
System Info
I have a model, and the architecture has xxxParallel attributes, which are used for parallel inference:
I want to directly load this model with peft (lora), but it throws an error:
So, how can I implement this process without any model architecture update?
Who can help?
@pacman100 @younesbelkada @BenjaminBossan @sayakpaul
Information
Tasks
examples
folderReproduction
Expected behavior
solve this issue.
The text was updated successfully, but these errors were encountered: