Collection Helper should not make redundant COUNT queries. #7489
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have been looking into why
pagination_total: false
is not preventing the execution of multiplesubquery_for_count
queries. I found the issue appeared rooted in thecollection_size
method not validating whether a collection is already loaded. General context on my changes:count
, this triggers several redundant count subqueries (subquery_for_count
) that cause latency and even timeouts when loading index pagesArray#length
should be more performant thanArray#count
within this contextResolves #6492, #2638
Testing/QA
Because this is a refactor and not new behavior I did not add new specs. When looking into this, I did come across some bugs in the paginated collection specs that should probably be fixed at some point though; for example, the examples pertaining to pagination_total will always result in a false pass because they're testing for the absence of queries that will never occur, and it's going uncaught because the author did not test the opposite assertion when the option is false.
I've QA'd this as an override in a staging environment that uses a sanitized production DB snapshot by loading index pages for tables that I know to have millions of records. The staging environment has fewer computing resources, but after pushing this change it is rendering large index pages faster than the production environment — it's even rendering
pagination_total: false
pages that are timing out in production (due to the COUNT subqueries).In a development environment, I can also see that this prevents the redundant COUNT subqueries that drive many of us to attempt using the
pagination_total
option; here is my sanitized server log:Before
After