Unstable results when running the same query twice

[Kerollmops]

Describe the bug
When running the following query, the engine is returning unstable results. Note that everything is on the private Discord conversation with our customer.

{
    "displayedAttributes": [
        "*"
    ],
    "searchableAttributes": [
        "title"
    ],
    "filterableAttributes": [
        "category",
        "category_ids",
        "channel",
        "handle",
        "language",
        "made_for_kids",
        "num_ab_tests",
        "shorts",
        "subscriber_count",
        "upload_date",
        "view_count",
        "xf90"
    ],
    "sortableAttributes": [
        "upload_date",
        "view_count",
        "xf90"
    ],
    "rankingRules": [
        "words",
        "typo",
        "proximity",
        "sort",
        "exactness"
    ],
    "stopWords": [],
    "nonSeparatorTokens": [],
    "separatorTokens": [],
    "dictionary": [],
    "synonyms": {},
    "distinctAttribute": null,
    "proximityPrecision": "byWord",
    "typoTolerance": {
        "enabled": true,
        "minWordSizeForTypos": {
            "oneTypo": 5,
            "twoTypos": 9
        },
        "disableOnWords": [],
        "disableOnAttributes": []
    },
    "faceting": {
        "maxValuesPerFacet": 100,
        "sortFacetValuesBy": {
            "*": "alpha"
        }
    },
    "pagination": {
        "maxTotalHits": 300000000
    },
    "searchCutoffMs": null
}

To Reproduce
Unfortunately, we don't have a clear reproducer but running the above query multiple times.

Expected behavior
The results must be stable and sorted by xf90.

Meilisearch version:
v1.8.0

Additional context
Running on a dedicated Digital Ocean instance on the Cloud.

[dureuill]

A reason for result unstability introduced in v1.8 is the search cutoff.

I'm pretty sure this is what we're seeing.
To confirm, one can take a look at the score details looking for Skipped ranking rules.

If the behavior is undesirable, one should increase the search-cutoff-ms setting.

[Kerollmops]

Thank you, @dureuill 👏

You found the issue! This is only because the searchCutoffMs was set to 1500s, which stopped the computation and returned very non-relevant results. So, it is expected, and there is a workaround to increase the cutoff timing simply.

With the Default Cutoff

"_rankingScoreDetails": {
    "words": {
        "order": 0,
        "matchingWords": 1,
        "maxMatchingWords": 1,
        "score": 1.0
    },
    "typo": {
        "order": 1,
        "typoCount": 0,
        "maxTypoCount": 1,
        "score": 1.0
    },
    "proximity": {
        "order": 2,
        "score": 1.0
    },
    "xf90:desc": {
        "order": 3,
        "value": 258587.8854961832
    },
    "skipped": {
        "order": 4
    }
}

With a High Cutoff Everything is Fine

"_rankingScoreDetails": {
    "words": {
        "order": 0,
        "matchingWords": 1,
        "maxMatchingWords": 1,
        "score": 1.0
    },
    "typo": {
        "order": 1,
        "typoCount": 0,
        "maxTypoCount": 1,
        "score": 1.0
    },
    "proximity": {
        "order": 2,
        "score": 1.0
    },
    "xf90:desc": {
        "order": 3,
        "value": 258587.8854961832
    },
    "exactness": {
        "order": 4,
        "matchType": "noExactMatch",
        "matchingWords": 0,
        "maxMatchingWords": 1,
        "score": 0.16666666666666666
    }
}