Grafana does not reload its TLS server certificate data #38095
Replies: 3 comments 3 replies
-
The current design is we load config.ini just one time at the server start time, It is a good idea to make it updatable on the fly without restart the server, The ticket should be a feature request. |
Beta Was this translation helpful? Give feedback.
-
@frebib, @ying-jeanne - a simple approach IMO would be a task that regularly checks the PKI data (ca cert, tls cert, tls key) mounted in the Grafana pod namespace and - upon change - updates the internal TLS config of the Grafana server endpoint. If running on Kubernetes, the Grafana pod - if supporting the liveness probe - could also simply send a negative response to the liveness probe upon the expiry of the server certificate. With this, Kubernetes re-starts the Grafana pod that picks up the recent PKI data. |
Beta Was this translation helpful? Give feedback.
-
Hello, as you may have heard, we are transitioning away from using discussions to discuss feature requests. Due to the age and number of responses to this discussion, we are deciding to close it. If this is something you would like to see in Grafana, feel free to open an issue so the discussion can continue. Thank you! |
Beta Was this translation helpful? Give feedback.
-
What happened:
After the start, the Grafana pod reads the PKI information from the file(s) configured in grafana.ini. The running Grafana pod does not seem to monitor the update of these files and so the browser accessing the Grafana UI reports an expired certificate even when the information in the files (e.g. /etc/grafana/pki/tls.crt) contains a refreshed and valid certificate data.
What you expected to happen:
Grafana should regularly check the files storing the TLS cert and key for updates and utilize the updated data on the fly, i.e. without the need to restart the Grafana pod.
How to reproduce it (as minimally and precisely as possible):
Configure Grafana with a certificate that has a short validity of e.g. 2h. Start the Grafana pod and update the file storing the TLS cert (e.g. /etc/grafana/pki/tls.crt) with a refreshed certificate data (validity e.g. 10h) while keeping the Grafana pod running. Observe the PKI files in the Grafana pod's namespace being updated with the new certificate data (and the new validity) while the browser connecting to the Grafana UI will report an expired certificate 2h after the pod start, i.e. when the cert loaded at the start of the Grafana pod has expired.
Anything else we need to know?:
Jetstack cert-manager and K8S Secret are useful in automating the test case
Environment:
Beta Was this translation helpful? Give feedback.
All reactions