Khepri: Use read-only transactions to query for user/topic permissions #11398
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Read-write transactions behave like commands (for example 'put' or 'delete') but read-only transactions behave like queries (for example 'get' or 'exists'). When a transaction is passed as read-only we can re-use the same default options we use for most Khepri queries and favor low-latency queries. This reads the values from the local node rather than the leader (or a consistent query), making the query faster and preventing timeouts when attempting the query when the cluster is in minority. The child commit will update some read-only transaction callsites that take advantage of this low-latency favor.
…sions
The parent commit favors low-latency queries for read-only transactions,
so marking these read-only transactions as explicitly read-only will
cause them to be run against the local node. This ensures that functions
like `rabbit_auth_backend_internal:list_permissions/0` do not return
`{error, timeout}` when the cluster is in minority, fixing 'bad
generator' errors during definitions export (via
`rabbit_definitions:all_definitions/0`).
This is a mix of a few changes: * Suppress the compiler warning from the export_all attribute. * Lower Khepri's command handling timeout value. By default this is set to 30s in rabbit which makes each of the cases in `client_operations` take an excessively long time. Before this change the suite took around 10 minutes to complete. Now it takes between two and three minutes. * Swap the order of client and broker teardown steps in end_per_group hook. The client teardown steps will always fail if run after the broker teardown steps because they rely on a value in `Config` that is deleted by broker teardown.
michaelklishin
approved these changes
Jun 6, 2024
the-mikedavis
deleted the
md/khepri/read-only-permissions-transaction-queries
branch
the-mikedavis
added a commit
that referenced
this pull request
Jun 7, 2024
Khepri: Use read-only transactions to query for user/topic permissions (backport #11398)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Read-only transactions in Khepri are done with queries rather than commands. We can set these queries to favor low-latency - querying the local node rather than trying to contact the leader - to make them faster and available when Khepri is in a minority. This fixes some 'bad generator' errors when exporting definitions from a cluster member in a minority.
This is related to #10915 but #10915 mostly covers handling timeouts from commands rather than queries, and these commits port out cleanly.