-
Notifications
You must be signed in to change notification settings - Fork 608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NDCG computation #624
Comments
The Are you seeing instances where models aren't returning ids sorted by their scores? |
I guess this issue can be closed, since the bug of sorting the output of KNN models was fixed in November 2022 with #629. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello @benfred. I think there might be an issue with the computation of NDCG in ranking_metrics_at_k. I believe NDCG should be calculated by first sorting the recommended ids by the recommendation scores, so that the logarithmic discount is applied in the correct ranking of the recommendations. Right now, Implicit's recommend methods do not return the ids sorted by their descending scores, so the way NDCG is being calculated does not take into account the order of the ranking.
I would appreciate your opinion, maybe I'm missing something and I'm wrong. Thanks in advance.
If changes turn out to be necessary, I would suggest adding some lines on this part of ranking_metrics_at_k
The text was updated successfully, but these errors were encountered: