[DRAFT] Prevent pod restarts on startup #920
Draft
+275
−42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pintohutch
changed the title
It is tempting to only rely on defaulting webhooks to ensure any changes to the OperatorConfig updates the .collection.externalLabels and rules.externalLabels fields with the default project, location, and cluster labels. However, for many cases, where an OperatorConfig already exists and the operator is restarted with this change, a webhook will not be invoked and the change to the generated Prometheus config's external_labels will never happen. So we add the defaulting webhook as well as implicit logic to fill in the generated config external_labels at runtime. Another approach could be to omit the mutating webhook entirely, however if we can reflect the state of the config in our custom resources, we should. This gives users a more clear and explicit sense of what is being configured.
pintohutch
force-pushed
the
pintohutch/fix-restarts
branch
from
3283a7e
to
bb576af
Compare
|
I like this approach, but we have to manually (or/and automatically) tested for two cases - new and existing OperatorConfig. (Not on it, don't have time now, help wanted) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It is tempting to only rely on defaulting webhooks to ensure any
changes to the OperatorConfig updates the .collection.externalLabels
and rules.externalLabels fields with the default project, location, and
cluster labels.
However, for many cases, where an OperatorConfig already exists and the
operator is restarted with this change, a webhook will not be invoked
and the change to the generated Prometheus config's external_labels will
never happen.
So we add the defaulting webhook as well as implicit logic to fill in
the generated config external_labels at runtime.
Another approach could be to omit the mutating webhook entirely, however
if we can reflect the state of the config in our custom resources, we
should. This gives users a more clear and explicit sense of what is
being configured.