Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create silece for alert till next working hours #174

Closed
freeseacher opened this issue Jun 29, 2022 · 13 comments
Closed

create silece for alert till next working hours #174

freeseacher opened this issue Jun 29, 2022 · 13 comments
Labels
feature request New feature or request

Comments

@freeseacher
Copy link

Lots of small p5 alerts can wait till next working hours.
As schedule owner i would like to setup some kind of working hours or time to debug and close low priority alerts. something like
notify mon: 9-18, tue: 9-18... fri: 9-18, sat, sun do not notify
and two option notify if still firing, notify anyway.

of cause low priority alerts lead to alert fatigue and must me eliminated but still popular case

@raphael-batte raphael-batte added the feature request New feature or request label Jun 29, 2022
@raphael-batte
Copy link

raphael-batte commented Jun 29, 2022

@freeseacher We plan to add 'working hours' time for users in our upcoming scheduling tools. @Matvey-Kuk Probably, we should add additional notification options in user settings as well.

@Matvey-Kuk
Copy link
Contributor

Low-priority alert groups could be paused in escalation chain using this step:
Screenshot 2022-06-29 at 15 41 30

@freeseacher will it work for you or you are looking for something else?

@freeseacher
Copy link
Author

Sounds great!
but that only a time. how i can skip till Monday morning?

@raphael-batte
Copy link

raphael-batte commented Jun 29, 2022

Something like this in user settings?
image

@freeseacher
Copy link
Author

🤔. that settings sounds very opinionated for me.
what is low-priority? how do they defined ?
for me that is not personal setting but option in silence
image

or part of alert proccessing before escalation

as for me i see them throw prism of another product like this

image

@raphael-batte
Copy link

raphael-batte commented Jun 29, 2022

@freeseacher how do these settings work if there are people in different timezones with different working hours in one schedule?

@freeseacher
Copy link
Author

great question!
i believe that notification should be delivered as fast as possible. so it should be delivered to someone who is on call first

@Matvey-Kuk
Copy link
Contributor

@raphael-batte we have this concept on the Escalation Chain level (with time), why do you think it should go to the Notification Level?

@raphael-batte
Copy link

@Matvey-Kuk I am not suggesting removing the silence from the OnCall escalation chain level.
But if users need local time silence, it should be on profile level.

Example:

  1. We have an escalation chain with silence step, the server time is Tel Aviv TZ
  2. At 9:00 a.m. monday we restart the chain, but at this time we have an engineer from New York oncall, who is now outside of his working hours.

If he does not have personal silence option in user-settings, he will receive an alerts.

@Matvey-Kuk
Copy link
Contributor

Time-based silence in the user space will mean "we escalate to a user" -> "user reacts in a few hours".

It's a pattern we generally want to avoid project-wise. We don't want alerts to "stick" to users for multiple hours. The goal of OnCall is to be an effective alert distributor between team members. What if the user will go to a vacation after the timeout? How to indicate the reason of long response delay to other users?

I believe in this particular case the alert should stuck on the escalation chain, or be routed to the other person if there is a time-zone distributed team immediately.

So I think we either should think of improving our Calendar either about adding more adjustable escaltion step.

@raphael-batte
Copy link

Yes, this is the other side of this situation. Agree with the need to rethink/improve the transition mechanic here.

Copy link
Contributor

github-actions bot commented Nov 9, 2023

This issue has been automatically marked as stale because it has not had activity in the last 120 days.

@github-actions github-actions bot added the pr:stale Added to a PR that has been deemed "stale". Managed by the actions/stale GitHub Action label Nov 9, 2023
@github-actions github-actions bot removed the pr:stale Added to a PR that has been deemed "stale". Managed by the actions/stale GitHub Action label May 22, 2024
@joeyorlando
Copy link
Contributor

@freeseacher I think this should now be achievable with the custom silence durations:
Screenshot 2024-06-14 at 13 14 41
Screenshot 2024-06-14 at 13 14 44

I'll go ahead and close this out but feel free to open a new feature request if this does not suit what you need!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants