Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ObservedGeneration and update block #1298

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

burmanm
Copy link
Contributor

@burmanm burmanm commented Apr 24, 2024

What this PR does:
Adds new status field: ObservedGeneration and ensure it is updated at the end of the reconciliation

Which issue(s) this PR fixes:
Fixes #1274

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CHANGELOG.md updated (not required for documentation PRs)
  • CLA Signed: DataStax CLA

@burmanm
Copy link
Contributor Author

burmanm commented Apr 30, 2024

@olim7t

@burmanm burmanm force-pushed the upgrade_process branch 2 times, most recently from 5801d6a to 2881e5f Compare May 13, 2024 10:44
@burmanm burmanm marked this pull request as ready for review May 13, 2024 10:44
@burmanm burmanm requested a review from a team as a code owner May 13, 2024 10:44
@adejanovski
Copy link
Contributor

@burmanm , is this ready for another round of review?

@burmanm
Copy link
Contributor Author

burmanm commented May 22, 2024

Yes

Copy link
Contributor

@olim7t olim7t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still have to test this manually, but publishing this review early because I think the task naming issue needs attention.

controllers/control/k8ssandratask_controller.go Outdated Show resolved Hide resolved
controllers/k8ssandra/datacenters.go Outdated Show resolved Hide resolved
controllers/k8ssandra/datacenters.go Show resolved Hide resolved
…tricted values for AutomatedUpdate, "always" and "once" and verify them in the webhook.

Remove tee outputting to stdout in the helm prepare script.

UpdateStatus should only delete and reset the ClusterRequiresUpdate if it was allowed to update in the first place. Also, we should listen to changes and reconcile if annotations change.

Add additional checks to the GenerationCheck test to ensure we do not touch the CassandraDatacenter unless given permission.
Since ObservedGeneration is being added by this PR, it will be 0 when
the operator gets upgraded to this version. We want to interpret that as
"the generation didn't change".

This was agreed upon previously but got lost in a force-push.
Copy link
Contributor

@olim7t olim7t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed commits to fix a few remaining issues. This is good to go as far as k8ssandra-operator is concerned.

However I'm noticing an issue with the refresh CassandraTask: it doesn't seem to ever be marked as finished (and as a consequence neither does the K8ssandraTask). It doesn't impact the new behavior we're introducing here, the conditions and annotations are correctly reset.

I'll test on cass-operator alone and file an issue if necessary.

Copy link

sonarcloud bot commented Jun 1, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
3.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

@olim7t
Copy link
Contributor

olim7t commented Jun 4, 2024

However I'm noticing an issue with the refresh CassandraTask: it doesn't seem to ever be marked as finished (and as a consequence neither does the K8ssandraTask).

OK, this might be a side-effect of my testing method: I was artificially setting the RequiresUpdate=true condition on the CassandraDatacenter to simulate a cass-operator upgrade, but refreshing the DC did not produce any actual changes to the stateful set. It looks like something in the way the refresh is implemented causes the task to not be marked as completed in that case. This is not something that will happen in real life, so I think we can safely ignore it.

@adejanovski
Copy link
Contributor

Given we found some bugs (or side effects) in cass-operator's implementation of this feature, I think we should first solve those and re-evaluate how it's done in this PR.
Also, and that's probably something we should have done from the beginning, I think we should make this an opt in feature rather than the default behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

k8ssandra-operator shouldn't modify the cassdc objects on operator upgrade
3 participants