Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add r-precision metric #2067

Closed
daviddavo opened this issue Feb 23, 2024 · 1 comment
Closed

[FEATURE] Add r-precision metric #2067

daviddavo opened this issue Feb 23, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@daviddavo
Copy link
Collaborator

daviddavo commented Feb 23, 2024

Description

R-precision can be defined as the precision@R, with R being the number of relevant documents for the query [1]. This means that for a certain user $u$ which has $R$ relevant items, we would ask the recommendation engine for the top $R$ recommendations, and see which of them are true positives.

[1]: C. D. Manning, P. Raghavan, and H. Schütze, ‘Chapter 8. Evaluation in information retrieval’, in Introduction to information retrieval, New York: Cambridge University Press, 2008, pp. 151–175.

Expected behavior with the suggested feature

Adding R-Precision will allow evaluating systems with some characteristics.

Other comments

Implementation idea:

  1. Get the number of relevant documents (R) for all users
  2. Get the top-k recommendations of the model where $k = \max R$
  3. Calculate the R-precision for each user, and return the mean as in other metrics
@daviddavo daviddavo added the enhancement New feature or request label Feb 23, 2024
@miguelgfierro
Copy link
Collaborator

@daviddavo would you be able to contribute to this?

@daviddavo daviddavo self-assigned this Mar 20, 2024
@daviddavo daviddavo mentioned this issue Apr 11, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants