-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change default scoring to mean squared error #322
Labels
enhancement
Idea or request for a new feature
Milestone
Comments
leouieda
added
enhancement
Idea or request for a new feature
question
Further information is requested
labels
Apr 8, 2021
If we decide to do this, we should include a |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
At the moment, the default scoring metric is R², which has the advantage of being unit-less so it's easy to compare. But it also doesn't really tell us much about the actual prediction error to expect. It also doesn't work when we do leave-one-out cross-validation since it results in NaNs (thanks to @dangilbert1337 for finding this).
With Verde 1.6.0, we can specify a different metric for
cross_val_score
and there is a private functionscore_estimator
that could be used instead of thescore
method. But it would be much more convenient to get the MSE fromscore
instead of always having to use these other options.What do people think about making this change? A 👍🏽 👎🏽 here would be appreciated.
This would break backward compatibility so it should be reserved for Verde 2.0. We might want to start a branch for that so we can begin to work on these features instead of letting them sit in the issues.
The text was updated successfully, but these errors were encountered: