Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

miss-islington broken again #228

Closed
vstinner opened this issue May 21, 2019 · 9 comments · Fixed by #229
Closed

miss-islington broken again #228

vstinner opened this issue May 21, 2019 · 9 comments · Fixed by #229

Comments

@vstinner
Copy link
Member

I don't know what to do when this bot is broken :-(

I requested a backport from master to 3.7 on python/cpython#13421 The bot wrote "Thanks @erikjanss for the PR, and @vstinner for merging it .. I'm working now to backport this PR to: 3.7." and then nothing, as if it fails but doesn't say anything about its failure.

Who operates this bot? Who knows how to debug it?

@Mariatta
Copy link
Member

Sorry of the problem. This is known issue that was raised in https://mail.python.org/pipermail/python-committers/2019-May/006728.html.

Example error log from earlier incident: https://mail.python.org/pipermail/python-committers/2019-May/006729.html

I think one way to fix it is to catch the redis.exceptions.ConnectionError and retry the task. Sorry I haven't had time to fix it. I will try adding the fix later today. For now, it seems like there is merge conflict so you'll still need to backport manually 😰

@vstinner
Copy link
Member Author

I think one way to fix it is to catch the redis.exceptions.ConnectionError and retry the task.

Well, I cannot say which option is the best. But if the bot fails to produce the PR, would it be possible to add a comment "Sorry, I failed to backport the change: internal error." Something like that?

@vstinner
Copy link
Member Author

For now, it seems like there is merge conflict so you'll still need to backport manually cold_sweat

Oh, that's perfectly fine. I will ask the author to do the backport ;-)

@matrixise
Copy link
Member

just another point, when there is this issue, just add a label "waiting backport" ? it's not a problem to create the backport for the PRs I review.

@matrixise
Copy link
Member

@Mariatta not sure, but I think the issue is mainly with the celery worker and not with your task.

@matrixise
Copy link
Member

I add this issue to the current issue from the kombu project celery/kombu#1019

@Mariatta
Copy link
Member

just another point, when there is this issue, just add a label "waiting backport" ? it's not a problem to create the backport for the PRs I review.

I don't think we need another label. If miss-islington's backport failed, the "needs backport to ..." label is still there, so we know it is still needs to be backported. In addition, the PR becomes assigned to the core dev who reviews the PR, so core dev also knows they need to do something about it.

@terryjreedy
Copy link
Member

A workaround when no backport appears after, say, 20 minutes, is to remove the backport label and then add it again to trigger the bot again. I have done this twice and the second try worked both times.

@Mariatta
Copy link
Member

I've made it so it will post a comment that there is problem with backporting because of redis Connection Error. It will also retry the backport up to 5 times (with 60 minutes delay).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants