Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jacaard_similarity_accuracy to the objectives #816

Open
saeedizadi opened this issue Mar 18, 2017 · 1 comment
Open

Jacaard_similarity_accuracy to the objectives #816

saeedizadi opened this issue Mar 18, 2017 · 1 comment

Comments

@saeedizadi
Copy link

One of widely-used metrics for evaluationg the results of a segmentation task is the Jaccard_similarity_coefficients. It is as simple as diving the intersection region over the union region between two binary maps.

Adding this accuracy metric to the objectives (just like binary_accuracy) can be so helpful.
For more information, please see this Wikipedia page:
https://en.wikipedia.org/wiki/Jaccard_index

@f0k
Copy link
Member

f0k commented Mar 19, 2017

One of widely-used metrics for evaluationg the results of a segmentation task

Do you have some references at hand?

Adding this accuracy metric to the objectives (just like binary_accuracy) can be so helpful.

I'd be fine with this. Feel free to submit a PR. There are some things to consider:

  • For binary classification, when binarizing the predictions, the Jaccard index is the same as the accuracy.
  • One could go for a non-binarized version that computes minimum(A, B).sum() / maximum(A, B).sum(). This would even be differentiable. Probably there's literature for that as well.
  • For multi-class problems, this should support both one-hot encoded and integer targets, like categorical_accuracy.
  • We should think about whether we want jaccard_index() (jaccard_similarity_coefficient() is a bit long for my taste) or jaccard_distance(). Depends on whether we advertise this as a loss function or evaluation measure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants