-
Notifications
You must be signed in to change notification settings - Fork 548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[v2 BUG]: MCC loss can give Nan #846
Comments
Is it possible that the prediction could be negative? Should the prediction be the output after the sigmoid function? In v 1, PR #309 made the multiclass MCC loss return 0 in cases where the loss was infinite. However, the binary MCC was not fixed at that time. For datasets that are very imbalanced, it is possible that there is only a positive or negative result in a batch, leading to the MCC being not meaningful. Using class balance might be a work around in this case. |
I believe the error is here: Lines 195 to 197 in cc5b3c1
This assumes the I also agree with #380 in that we should add a small |
Describe the bug
BinaryMCCLoss
can give a loss ofnan
.Example(s)
The confusion matrix elements (TP, FP, TN, FN) are calculated as
targets * predictions
. At the start of training model predictions can happen to be negative, in which case some of the matrix elements can be negative. This leads((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN)).sqrt()
to possibly take the square root of a negative number and givenan
.Expected behavior
I'm not sure what the expected behavior should be. Perhaps raw predictions should be forced to be positive (via a sigmoid?). Or the loss set to zero when the loss is
nan
?Additional context
This was also a problem in v1 (#380). Our current implementation seems to be taken directly from v1.
The text was updated successfully, but these errors were encountered: