-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updating only one adapter using multi adapter #1670
Comments
Mixed batches are not related, find a description of mixed adapter batches in the docs. For what you want, I think what you would need is to manually set Note that we normally only train with a single adapter at a time, so your specific use case is a bit of an outlier and may run into some problems I haven't thought of yet. |
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. |
I am curious if I can update only one adapter in multi adapter siutations.
For example, I have two adapter A and B.
In forward pass, LLM + adapter A + adapter B to get Loss,
and update only adapter A using peft.
I found _mixed_batch_forward in your code, is it related to my question?
The text was updated successfully, but these errors were encountered: