RolloutManager should create a ServiceMonitor for users with ServiceMonitor support

[jgwest]

RolloutManager controller needs to create a ServiceMonitor so that metrics from Rollouts will automatically included in Prometheus gathered metrics, including in OpenShift monitoring (for OpenShift users)

• A service, argo-rollouts-metrics, is already created to provide access to metrics, it just needs to be connected to monitoring via a ServiceMonitor CR.

Why is this important?

• Customers need to monitor components of the system including the Argo Rollouts which is deployed by the GitOps Operator

Scenarios

• Deploy Rollouts into a namespace with RolloutManager and be able to monitor Argo Rollouts

(Originally a feature request from https://issues.redhat.com/browse/GITOPS-3271)

Work Criteria

• When a RolloutsManager CR is reconciled, it should create the corresponding ServiceMonitor CR
  • Should we always create it, or should we add a new parameter to the CR which a user can set to true to enable this?
  • Investigate how this works in argocd-operator and/or gitops-operator, and follow their lead
• This should only occur when we detect that the cluster is an OpenShift cluster
• This should only occur when the user is using the Prometheus operator, which includes the ServiceMonitor CR.
  • Probably the easiest way to do this is check for CustomResourceDefinition for ServiceMonitor
  • For example, by checking the API version or looking for an OpenShift-only CR (e.g. Routes)
• Manually verify the functionality works as expected
• Unit/E2E tests

[jgwest]

Gerald notes that the following ServiceMonitor worked for him:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: argo-rollouts-metrics
spec:
  endpoints:
    - port: metrics
  selector:
    matchLabels:
      app.kubernetes.io/name: argo-rollouts-metrics 

Siddhesh notes that this can be used for reference:
Use https://github.com/redhat-developer/gitops-operator/blob/master/controllers/argocd_metrics_controller.go#L252 as reference.