Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Range partition assigner #1631

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

pimpelsang
Copy link
Contributor

Hi,

Here is range partition assigner using the same algorithm as in java client. Please review :)

closes #1564

const assignee = sortedMembers[i]

for (let partition = start; partition < start + length; partition++) {
if (!assignment[assignee]) {
Copy link
Contributor

@Saeger Saeger Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have 3 levels of for loop which doesn't seem ideal, on the other hand, this is only used once assigning members to the consumers right? So it should be fine I guess. I wanted to ask if you have reasoned about this before as well?

Edit: Well, I noticed the same behaviour from the java version as well :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, same flow in java client. There isn't that much data to loop through and like you said it happens once. I see no reason to optimise.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yduman
Copy link

yduman commented Mar 11, 2024

What is preventing this PR from being merged? Our team is currently migrating from node-rdkafka to KafkaJS and since the partition.assignment.strategy is set by default to [range, roundrobin] in librdkafka, we are getting a KafkaJSProtocolError pointing to InconsistentGroupProtocolException, because KafkaJS uses by default only roundrobin. This PR would allow to fix this problem, where users can import both strategies for partitionAssigners. The alternative, without this PR, would be to copy the implementation of this solution and introduce it as a class manually by yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add range strategy assigner
3 participants