-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nsqd: discover topic/channel paused state on new topic discovery #1199
Labels
Comments
SGTM |
@jehiah @mreiferson any update here? I think i can spend a weekend to finish it. |
hi, anyone can help to take a look at this? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'd like to provide an experience report from introducing new nsqd nodes to a cluster while operational activities which involve pausing channels are underway.
Currently nsqd discovers the topic/channel topology from
nqlookupd
when it receives a new message. This ensures that if you have say 10 channels on a topic, all 10 channels get a copy of the first message.If a channel is in a paused state when the first message is published to a new
nsqd
node it's naturally expected that the new node would also end up with channels in an appropriately paused states. That doesn't happen unfortunately becausensqlookupd
has no context about the paused state of a channel, and so it can't provide it when queried by nsqd on a new topic creation. (There also isn't technically anything that requires all hosts to be paused or unpaused together, even though that's obviously most common)Feature Request:
nsqd
propagate topic/channel pause states tonsqlookupd
. On discovery, nsqlookupd can advise if the majority of clients for a topic/channel are paused and if so, nqsd will create a pre-paused topic or channel. If there are no clients for a topic/channel connected to nsqd at the time of topic discovery, nsqd should create a topic and channel un-paused. Whennsqd
queries multiplenqslookupd
if any response indicates paused, the result should be a paused state. i.e.paused
is preferred.Note: nsqlookupd response changes to support this should be backwards compatible.
The text was updated successfully, but these errors were encountered: