Filter by score

[curquiza]

Related product team resources: PRD (internal only)

Motivation

Required by users

Usage

Filter by score

https://meilisearch.notion.site/Filter-by-score-usage-224a183ce7b24ca99b6a9a8da755668a?pvs=74

TODO

Reminders when modifying the Setting API

• Ensure the new setting route is at least tested by the test_setting_routes macro
• Ensure Analytics are fully implemented
  • /settings/my-new-setting configurated in the make_setting_routes macro
  • global /settings route configurated in the update_all function
• Ensure the dump serializing is consistent with the /settings route serializing, e.g., enums case can be different (camelCase in route and PascalCase in the dump)

Special cases when adding a setting for an experimental feature

• ⚠️ API stability: The setting does not appear on the main settings route when the feature has never been enabled (e.g. mark it Unset when returned from the index in this situation. See an example)
• The setting cannot be set when the feature is disabled, either by the main settings route or the subroute (see validate_settings function)
• If possible, the setting is reset when the feature is disabled (hard if it requires reindexing)

Impacted teams

@meilisearch/integration-team @meilisearch/docs-team

[dureuill]

Hello 👋

A prototype is available for filtering by ranking score!

Getting the prototype

You need to start from a fresh new database (remove the previously used data.ms) and use the following Docker image:

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-score-threshold-0

Using the prototype

The prototype adds a new optional parameter to search requests: rankingScoreThreshold.

The rankingScoreThreshold if present must be a number in the [0.0, 1.0] range.

During the search, any set of results whose _rankingScore is below the rankingScoreThreshold is discarded instead of returned. The corresponding documents are also removed from the facet distribution and don’t count towards the totalHits and estimatedTotalHits.

Warning

For performance reasons, the current implementation does not always return correct values for totalHits and estimatedTotalHits

Explanation

When the `limit` number of results is reached, then the algorithm stops without computing the `_rankingScore` for the remaining, unreturned documents. As a result, Meilisearch cannot determine if the remaining documents meet the threshold or not. The current behavior is to keep the remaining documents in the set of possible candidates. This means that increasing `limit` or requesting the next page can result in a decrease in `totalHits` and `estimatedTotalHits` (as well as the number of pages). Fixing this limitation involves computing the results exhaustively, which is going to be very performance-intensive and difficult to achieve at all for vector sort.

We're looking for feedback on this prototype, especially on the limitations stated above.

Caution

We do not recommend using this prototype in production. This is only for test purposes.