-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ignore keys for modules to save #1709
Comments
I'm not quite sure what you mean. On the one hand, you mention targeting LoRA layers, which implies you want to exclude certain layers from On the other hand, you mention |
Feature request
It would be nice if you could specify a regex pattern for layer names to ignore from target modules.
Motivation
I have a base model that I apply lora to, but then I also add a few extra attention layers at the end of the model. I don't want lora to target these new decoder layers.
Your contribution
I could work on a PR if there is interest.
The text was updated successfully, but these errors were encountered: