-
Notifications
You must be signed in to change notification settings - Fork 918
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A graceful-takeover can fail when RecoverNonWriteableMaster is true #1457
Comments
The graceful-takeover set old master read only first, and this will trigger try this: |
Thank you for the answer. That's what we've done in our playbooks.
My point is that it may be not very obvious. It's easy to miss it, especially when you use the webUI. |
This happens from commandline as well. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We noticed that a graceful-takeover command can fail sometimes, not promoting a new master and giving the following message:
"GracefulMasterTakeover: unexpected error: recovery not attempted. This should not happen"
If we turn off
RecoverNonWriteableMaster
flag, it works.If we disable global-recoveries before the graceful-takeover, it works too.
It does not fail all the time, it's unpredictable.
It looks like we have a race condition when the
NoWriteableMasterStructureWarning
is triggered, becauseGracefulMasterTakeover
start by setting the master as read_only.initial situation
final situation
Please see some logs:
The text was updated successfully, but these errors were encountered: