tighten rule pre-selection

[williballenthin]

closes #2074
ref #2063, particularly "tighten rule pre-selection" and "lots of time spent in instancecheck"

Stacked on #1950, so I've marked this as a PR onto that branch so the diff is sensible. I think we can probably rebase onto master, though, if necessary.

---

This PR implements the "tighten rule pre-selection" algorithm described here: #2063 (comment) . In summary:

Rather than indexing all features from all rules, we should pick and index the minimal set (ideally, one) of features from each rule that must be present for the rule to match. When we have multiple candidates, pick the feature that is probably most uncommon and therefore "selective".

This seems to work pretty well. Total evaluations when running against mimikatz drop from 19M to 815K (wow!) and capa seems to match around 3x more functions per second (wow wow). I did not expect such a good result - in fact, although the capa matches seem the be the same, I still wonder if something is broken 🤔. More tests needed.

label	count(evaluations)	min(time)	avg(time)	max(time)
8858537 pep8	19,939,632	25.74s	25.80s	25.84s
9c0c662 rules: optimize rule pre-filtering, first revision	815,892	9.45s	9.46s	9.48s

TODO:

• namespace matching
• prove that it matches exactly the same as before, just faster
• add some tests for the feature indexer, if only to show a human how it works
• add matcher tests for namespace matching
• xfail the tests and document the unsupported constructs
• inline documentation explaining the algorithm better
• wall clock performance numbers

[github-actions]

Please add bug fixes, new features, breaking changes and anything else you think is worthwhile mentioning to the master (unreleased) section of CHANGELOG.md. If no CHANGELOG update is needed add the following to the PR description: [x] No CHANGELOG update needed

[williballenthin]

for backwards compatibility. during a major version, we can probably remove these with preference to rules_by_scope.

[williballenthin]

Opened the PR here so the code is no longer sitting on my laptop and at risk of getting lost due to hardware failure.

[mr-tz]

nice work, we should do extensive tests comparing the results before and after to ensure everything works as expected. the speedup looks promising!

[williballenthin]

we should do extensive tests comparing the results before and after to ensure everything works as expected.

I plan to run this implementation side by side with the ceng.match implementation and assert the results are precisely the same across a wide range of samples. There should be no leaks of abstraction or details in the new one, it should just be faster.

CHANGELOG updated or no update needed, thanks! 😄