Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of query filters (especially on large bodies) #1839

Conversation

aukevanleeuwen
Copy link
Contributor

Fixes: #1838

Rewrite two QueryFilters with an implementation based on StringTokenizer rather than a possible O(n^2) regex (for more information see the linked issue).

Description

I've looked at the description for a application/x-www-form-urlencoded encoded body here: https://url.spec.whatwg.org/#urlencoded-parsing and tried to follow those lines. I did not put in the percentage encoding, since that wasn't in the current implementation and I didn't want to break backwards compatibility.

I've added a few test cases that weren't covered (and also not all handled correctly in the previous implementation), but I think they are rather obscure. Where normally you have key=value, it's also possible to just have key, if there is no = there, key is treated as the key and there is no value. Also key= and =value are allowed even, resulting in an empty key or empty value respectively.

Motivation and Context

Fixes #1838

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

@aukevanleeuwen
Copy link
Contributor Author

@whiskeysierra perhaps? :-)

@2k16daniel
Copy link

Hoping that this commit will be merged

@kasmarian
Copy link
Member

Thank you for the PR @aukevanleeuwen, it looks great, with StringTokenizer the performance will be way faster!

@kasmarian
Copy link
Member

In order for the PR to be compliant with our merging rules, could you please make sure your commits are signed?

@aukevanleeuwen aukevanleeuwen force-pushed the 1838-fix-queryfilter-performance branch from fde0074 to 678f2c8 Compare May 27, 2024 12:12
@aukevanleeuwen
Copy link
Contributor Author

@kasmarian

@aukevanleeuwen aukevanleeuwen force-pushed the 1838-fix-queryfilter-performance branch from 678f2c8 to c2a2a29 Compare May 27, 2024 12:14
@kasmarian
Copy link
Member

👍

1 similar comment
@lukasniemeier-zalando
Copy link
Member

👍

@lukasniemeier-zalando lukasniemeier-zalando merged commit 605db1b into zalando:main May 27, 2024
1 of 2 checks passed
@aukevanleeuwen aukevanleeuwen deleted the 1838-fix-queryfilter-performance branch May 30, 2024 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Default QueryFilter has a very slow RegEx on certain inputs
4 participants