Notification/alerting system in Modernisation Platform

[ewastempel]

User Story

As a Modernisation Platform member user
I want to be notified about health of resources or upcoming events that need actioning (e.g. expiring certificates)
So that I can react and fix/prevent an issue.

Value / Purpose

Healthy application/system means no outages.

Additional Information

This want came as an ask channel request and it is to look if we can implement a new one or reuse our existing alerting system that can be easily consumed by the members.

Currently MP alerting workflow is made of: CloudWatch -> SNS -> PagerDuty -> Slack and using this solution is documented here.

This ticket is to remove the need of PagerDuty acting as a mid-man and to integrate with a variety of resources (CloudWatch, EventBridge, SNS) rather than being limited to one only (although it could start with one and then build on it).

The user that requested this, suggested EventBridge → SNS → e-mail → Slack as an approach described in here, which could be considered.

Definition of Done

• New alerting system is implemented (ideally as a module) and it does not use PagerDuty
• If applicable, tests are implemented
• User docs have been updated
• Another team member has reviewed
• Pipeline runs green

[dms1981]

Is this potentially too broad? Is this ticket meant to cover the creation of a new alerting/notification module that we can use, or a one-off to cover alerting when certificates are reaching their expiration date that could later be extended to replace PagerDuty as a middleman?

Is this something that customers are presently empowered to do without us being involved?

[dms1981]

As you noted in Slack, @ewastempel , maybe this is a better fit for enrolling with Observability Platform and getting the information through there?