-
Notifications
You must be signed in to change notification settings - Fork 149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Lora adapter layers for LLM inference #181
Labels
Comments
MNN-2.9.0 will support apply lora on device. But now there are some accuracy problems caused by quantization. |
Sure. Thanks @wangzhaode . |
Marking as stale. No activity in 30 days. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
the llm-export utility https://github.com/wangzhaode/llm-export seems to have support to directly export a lora.mnn file during conversion in the llm_export.py .
However , it seems to me the framework does not yet support inference with lora.mnn exported file.
Any pointers regarding this would be useful :).
@wangzhaode
The text was updated successfully, but these errors were encountered: