-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question about the results of the attack #37
Comments
I am also not sure why there is such a huge change. Did yo use the same BERT model parameters that I provided? |
Yes, the model parameters and data are all provided by you.All the other attacks are close to the results given in your paper, only this one deviates a lot, so I wonder whether it is due to the data. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
hi,
when I experimented with the 1000 samples you provided from the AG dataset and the bert model, the results are:
For dataset data/ag: For target model bert: original accuracy: 94.200%, adv accuracy: 29.500%, avg changed rate: 25.076%, num of queries: 481.7
But the adv accuracy in your paper is 11.5%.
I'm not quite sure what led to this result. So did you use the same 1000 examples in your paper?
Thanks.
The text was updated successfully, but these errors were encountered: