-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplifying evaluation process #52
Comments
Hi @xhluca, Xueguang |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Right now, it's possible to train DPR in a single command, via the
tevatron.driver.train
module. However, to evaluate, a more complex series of command (involving lower-level for loops) is needed, e.g. for DPR on NQ:I think it would be nicer if all this could be reduce to 1 or 2 commands:
Note the usage of
tevatron.driver.evaluate
in order to keepdriver.encode
at a lower level and backward compatible, whileevaluate
would be for higher-level usage like reproducing results. Moreover,tevatron.driver.evaluate
could throw an error if pyserini is not available, e.g.:The text was updated successfully, but these errors were encountered: