-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to generate in batches? #60
Comments
Thanks for raising this. Very valid requirement. Will add to the list. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi Prithiviraj, thank you for the great work!
Is it possible to run this model with batches of input sentences so that we can leverage using the GPU much better? At the moment, setting use_gpu to True doesn't achieve much performance gains because we're not parallelizing across input phrases. Unless I missed something in the source code, in which case please let me know (and this would be good instruction to better emphasize in the documentation, at least in my case and I'm sure for many others if they try using this model for paraphrasing phrases in the 1mil+ data sizes)
The text was updated successfully, but these errors were encountered: