-
Notifications
You must be signed in to change notification settings - Fork 594
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can not reproduce the result of Roberta-Base #151
Comments
I also met this question, Specifcally, I downloaded the LoRa weights and run the following commands. python -m torch.distributed.launch- It turns out that the accuracy is 0.6655 while the number in paper is 0.875 |
Hello, I guess you used the wrong hyper parameters since the gap is really large. And for Roberta-large, I also can not reproduce the results. |
Does anyone have reproduce the LoRA result of roberta-base? I found the reproduction result of LoRA cannot achieve the result that paper has claimed.
e.g.:
Paper claimed that the RTE has achieved 86.6 accuracy, but my result has only achieved ~60 accuracy.
The text was updated successfully, but these errors were encountered: