You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the 0.73.0 release, if I run the operator in a Kind environment (1.26.6, and 1.29.2 both tested) I get a constant loop where the operator keeps killing and resyncing the AlertManagers and ThanosRulers:
level=info ts=2024-04-06T00:01:17.533270048Z caller=operator.go:471 component=thanos-controller key=observability/default msg="sync thanos-ruler"
level=info ts=2024-04-06T00:01:17.614336673Z caller=operator.go:766 component=prometheus-controller key=observability/scraper msg="sync prometheus"
level=warn ts=2024-04-06T00:01:17.629025757Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:17.629379048Z caller=operator.go:572 component=alertmanager-controller key=observability/alertmanager msg="sync alertmanager"
level=info ts=2024-04-06T00:01:17.71029034Z caller=operator.go:471 component=thanos-controller key=observability/default msg="sync thanos-ruler"
level=warn ts=2024-04-06T00:01:17.738319007Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:17.741981132Z caller=operator.go:572 component=alertmanager-controller key=observability/alertmanager msg="sync alertmanager"
level=warn ts=2024-04-06T00:01:17.768118048Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/alertmanager-kube-promethe-coredns
level=warn ts=2024-04-06T00:01:17.768276923Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-apiserver
level=warn ts=2024-04-06T00:01:17.768290298Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-coredns
level=warn ts=2024-04-06T00:01:17.768301673Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-controller-manager
level=warn ts=2024-04-06T00:01:17.76831184Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-etcd
level=warn ts=2024-04-06T00:01:17.768322382Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-proxy
level=warn ts=2024-04-06T00:01:17.768333298Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-scheduler
level=warn ts=2024-04-06T00:01:17.826784257Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:17.831138965Z caller=operator.go:766 component=prometheus-controller key=observability/scraper msg="sync prometheus"
level=warn ts=2024-04-06T00:01:17.863252798Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:17.863601923Z caller=operator.go:572 component=alertmanager-controller key=observability/alertmanager msg="sync alertmanager"
level=info ts=2024-04-06T00:01:17.910792007Z caller=operator.go:471 component=thanos-controller key=observability/default msg="sync thanos-ruler"
level=warn ts=2024-04-06T00:01:18.008474923Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:18.009143465Z caller=operator.go:572 component=alertmanager-controller key=observability/alertmanager msg="sync alertmanager"
level=warn ts=2024-04-06T00:01:18.043482424Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/alertmanager-kube-promethe-coredns
level=warn ts=2024-04-06T00:01:18.043598215Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-apiserver
level=warn ts=2024-04-06T00:01:18.04361659Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-coredns
level=warn ts=2024-04-06T00:01:18.043627507Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-controller-manager
level=warn ts=2024-04-06T00:01:18.043645965Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-etcd
level=warn ts=2024-04-06T00:01:18.043658674Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-proxy
level=warn ts=2024-04-06T00:01:18.043670382Z caller=promcfg.go:1374 component=prometheus-controller msg="'bearerTokenFile' is deprecated, use 'authorization' instead." version=v2.51.1 service_monitor=observability/prometheus-kube-prometheus-kube-scheduler
level=warn ts=2024-04-06T00:01:18.158803007Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:18.15981009Z caller=operator.go:572 component=alertmanager-controller key=observability/alertmanager msg="sync alertmanager"
level=warn ts=2024-04-06T00:01:18.187363549Z caller=klog.go:106 component=k8s_client_runtime func=Warning msg="spec.template.spec.containers[1].ports[0]: duplicate port definition with spec.template.spec.initContainers[0].ports[0]"
level=info ts=2024-04-06T00:01:18.187736132Z caller=operator.go:766 component=prometheus-controller key=observability/scraper msg="sync prometheus"
If I back out to the 0.72.0 release, it works fine. I've also tried building the images locally on my laptop and I get the same behavior ... building 0.72.0 works, but anything newer fails.
Steps to Reproduce
Install the prometheus-operator on a local KIND environment, create an AlertManager and ThanosRuler object, and watch.
Expected Result
It should work as it did in the 0.72.0 release - reconciling only when appropriate.
Is there an existing issue for this?
What happened?
Description
(moved from #6284 (comment))
In the 0.73.0 release, if I run the operator in a Kind environment (1.26.6, and 1.29.2 both tested) I get a constant loop where the operator keeps killing and resyncing the
AlertManagers
andThanosRulers
:If I back out to the 0.72.0 release, it works fine. I've also tried building the images locally on my laptop and I get the same behavior ... building 0.72.0 works, but anything newer fails.
Steps to Reproduce
Install the
prometheus-operator
on a local KIND environment, create anAlertManager
andThanosRuler
object, and watch.Expected Result
It should work as it did in the 0.72.0 release - reconciling only when appropriate.
Actual Result
Reconciliation loop.
Prometheus Operator Version
Kubernetes Version
1.26.6, 1.29.2
Kubernetes Cluster Type
kind
How did you deploy Prometheus-Operator?
helm chart:prometheus-community/kube-prometheus-stack
Manifests
No response
prometheus-operator log output
Anything else?
No response
The text was updated successfully, but these errors were encountered: