✨ Add OpenStackServerGroup CRD and Controller #1912

dalees · 2024-02-28T01:41:53Z

What this PR does / why we need it:

Implements new CRD for OpenstackServerGroup in v1alpha8 to allow managed Server Groups with standard policies, and adds ServerGroupRef to OpenstackMachine that references the new CRD and uses it for VM creation.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1256

Special notes for your reviewer:

This implements comment #1256 (comment)

There are a few TODO's remaining in code comments, and documentation of the feature to do. This first version is to ensure we have general agreement on the approach before continuing work on this.

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

squashed commits
includes documentation
adds unit tests
Rebased onto v1beta1 commit (removes v1alpha8)

/hold

Implements new CRD for OpenstackServerGroup in v1alpha8 to allow managed Server Groups with standard policies, and adds ServerGroupRef to OpenstackMachine that references the new CRD and uses it for VM creation. Closes: kubernetes-sigs#1256

k8s-ci-robot · 2024-02-28T01:41:59Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dalees
Once this PR has been reviewed and has the lgtm label, please assign vincepri for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-02-28T01:42:03Z

Hi @dalees. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

netlify · 2024-02-28T01:42:12Z

✅ Deploy Preview for kubernetes-sigs-cluster-api-openstack ready!

Name	Link
🔨 Latest commit	`65a96b7`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-openstack/deploys/65de8f64b61e5700089670de
😎 Deploy Preview	https://deploy-preview-1912--kubernetes-sigs-cluster-api-openstack.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

k8s-ci-robot · 2024-02-28T06:00:13Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

dulek

This looks pretty good, some remarks inline.

dulek · 2024-02-28T16:42:39Z

api/v1alpha8/openstackservergroup_types.go

+ // The name of the cloud to use from the clouds secret
+ // +optional
+ CloudName string `json:"cloudName"`


This seems a bit weird, we should probably have a reference to an OpenStackCluster instead?

Thank you for the feedback! Yeah, this allows the resource to be reconciled alone, as it's self contained.

However that isn't in any of the use cases, it doesn't seem a limitation to be tied to an existing OpenStackCluster even if the OpenStackServerGroup was only used for workers. It would remove duplication of these creds.

I'll make this change, once the CRD approach is agreed.

Hm, okay, that's a fair point. The use case to keep all the workers from different clusters in a single ServerGroup makes sense, I see your point.

dulek · 2024-02-28T16:48:14Z

api/v1alpha8/types.go

+type ServerGroupRef struct {
+ // Name of the OpenStackServerGroup resource to be used.
+ // Must be in the same namespace as the resource(s) being provisioned.
+ Name string `json:"name"`
+}


I think LocalObjectReference should be used as a type.

Would it be an issue that it specifies omitempty?

Ugh, it probably would. Okay, the design of this field is good, we can change internals if we want to later.

dulek · 2024-02-28T16:50:37Z

controllers/openstackmachine_controller.go

 err = compute.ResolveReferencedMachineResources(scope, &openStackMachine.Spec, &openStackMachine.Status.ReferencedResources)
 if err != nil {
 return reconcile.Result{}, err
 }

+ // Resolve referenced resources CAPO resources, using the K8s client
+ err = resolveReferencedClientResources(ctx, r.Client, openStackMachine)


It feels like it's still a Machine resource. Couldn't we put that into ResolveReferencedMachineResources directly? Even if we need to change the arguments of the function.

Hmm, I did start by doing this; I changed to this separation as what they're fetching from is distinct (OpenStack resource vs Kubernetes resource) and the client objects used are different. The OpenStack compute package just doesn't feel like the right place to be looking up K8s resources. It also makes test cases clearer to mock each function.

However, I agree the naming isn't clear. I wonder if renaming ResolveReferencedMachineResources to ResolveReferencedOpenStackResources may help to this end.

I'm open to changing this, but wanted to provide my reasoning first.

I see that, sure. Let's see what other reviewers will say here, especially @mdbooth as ResolveReferencedMachineResources() is an idea of his.

dulek · 2024-02-28T16:51:17Z

controllers/openstackservergroup_controller.go

+func (r *OpenStackServerGroupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (result ctrl.Result, reterr error) {
+ log := ctrl.LoggerFrom(ctx)
+
+ // Fetch the OpenStackMachine instance.


This log seems wrong.

Agreed, I'll fix this in the next iteration.

dulek · 2024-02-28T16:52:48Z

controllers/openstackservergroup_controller.go

+ // Get the servergroup by name, even if our K8s resource already has the ID field set.
+ // TODO(dalees): If this returns a 404 do we try to delete with existing UUID? Do we just assume success?


I think we should look up by ID and only then fallback to looking up by name. IDs are safe in case of duplicate names.

Ok, happy to change this - I can see it will lead to less problems if a duplicate named resource was created after this managed one.

dulek · 2024-02-28T16:54:09Z

controllers/openstackservergroup_controller.go

+
+ serverGroupName := openStackServerGroup.Name
+
+ serverGroup, err := computeService.GetServerGroupByName(serverGroupName, false)


Again, we should probably lookup by ID first in case we have duplicate names.

dalees · 2024-03-01T02:58:13Z

controllers/openstackmachine_controller.go

 err = compute.ResolveReferencedMachineResources(scope, &openStackMachine.Spec, &openStackMachine.Status.ReferencedResources)
 if err != nil {
 return reconcile.Result{}, err
 }

+ // Resolve referenced resources CAPO resources, using the K8s client
+ err = resolveReferencedClientResources(ctx, r.Client, openStackMachine)


Hmm, I did start by doing this; I changed to this separation as what they're fetching from is distinct (OpenStack resource vs Kubernetes resource) and the client objects used are different. The OpenStack compute package just doesn't feel like the right place to be looking up K8s resources. It also makes test cases clearer to mock each function.

However, I agree the naming isn't clear. I wonder if renaming ResolveReferencedMachineResources to ResolveReferencedOpenStackResources may help to this end.

I'm open to changing this, but wanted to provide my reasoning first.

dalees · 2024-03-01T02:59:11Z

pkg/cloud/services/compute/referenced_resources.go

@@ -22,8 +22,8 @@ import (
 )

 // ResolveReferencedMachineResources is responsible for populating ReferencedMachineResources with IDs of
-// the resources referenced in the OpenStackMachineSpec by querying the OpenStack APIs. It'll return error
-// if resources cannot be found or their filters are ambiguous.
+// the resources referenced in the OpenStackMachineSpec by querying the OpenStack APIs and K8s resources.


This comment change will be removed, this package should probably not look up K8s resources.

dalees · 2024-03-01T03:03:46Z

controllers/openstackservergroup_controller.go

+func (r *OpenStackServerGroupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (result ctrl.Result, reterr error) {
+ log := ctrl.LoggerFrom(ctx)
+
+ // Fetch the OpenStackMachine instance.


Agreed, I'll fix this in the next iteration.

dalees · 2024-03-01T03:46:13Z

api/v1alpha8/types.go

+type ServerGroupRef struct {
+ // Name of the OpenStackServerGroup resource to be used.
+ // Must be in the same namespace as the resource(s) being provisioned.
+ Name string `json:"name"`
+}


Would it be an issue that it specifies omitempty?

jichenjc · 2024-03-04T08:05:22Z

/ok-to-test

mdbooth · 2024-03-08T18:17:28Z

@pierreprinetti We agreed this in principal this week. Pinging you because it's similar to something ORC would do.

chess-knight · 2024-05-22T11:15:13Z

Hi, at @SovereignCloudStack we are very interested in this feature. What is the progress here @dalees?

dalees · 2024-05-23T00:19:28Z

Hi, at @SovereignCloudStack we are very interested in this feature. What is the progress here @dalees?

Hello - pleased to hear of the interest! I'm keen to get this in, and I'm scheduled to revisit this in the next few weeks to get it back into a reviewable state.

Add OpenStackServerGroup CRD and Controller

65a96b7

Implements new CRD for OpenstackServerGroup in v1alpha8 to allow managed Server Groups with standard policies, and adds ServerGroupRef to OpenstackMachine that references the new CRD and uses it for VM creation. Closes: kubernetes-sigs#1256

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 28, 2024

k8s-ci-robot requested review from EmilienM and jichenjc February 28, 2024 01:41

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Feb 28, 2024

k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Feb 28, 2024

dalees mentioned this pull request Feb 28, 2024

Use a server group to ensure anti-affinity for control plane nodes #1256

Open

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 28, 2024

dulek suggested changes Feb 28, 2024

View reviewed changes

dalees commented Mar 1, 2024

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 4, 2024

chess-knight mentioned this pull request May 22, 2024

Create v2 of node distribution standard (issues/#494) SovereignCloudStack/standards#524

Open

robincron mentioned this pull request May 23, 2024

Make use of servergroups for cluster nodes vexxhost/magnum-cluster-api#375

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Add OpenStackServerGroup CRD and Controller #1912

✨ Add OpenStackServerGroup CRD and Controller #1912

dalees commented Feb 28, 2024

k8s-ci-robot commented Feb 28, 2024

k8s-ci-robot commented Feb 28, 2024

netlify bot commented Feb 28, 2024 •

edited

k8s-ci-robot commented Feb 28, 2024

dulek left a comment

dulek Feb 28, 2024

dalees Feb 28, 2024 •

edited

dulek Feb 29, 2024

dulek Feb 28, 2024 •

edited

dalees Mar 1, 2024

dulek Mar 1, 2024

dulek Feb 28, 2024

dalees Mar 1, 2024

dulek Mar 1, 2024

dulek Feb 28, 2024

dalees Mar 1, 2024

dulek Feb 28, 2024

dalees Feb 28, 2024

dulek Feb 28, 2024

dalees Mar 1, 2024

dalees Mar 1, 2024

dalees Mar 1, 2024

dalees Mar 1, 2024

jichenjc commented Mar 4, 2024

mdbooth commented Mar 8, 2024

chess-knight commented May 22, 2024

dalees commented May 23, 2024

		// Get the servergroup by name, even if our K8s resource already has the ID field set.
		// TODO(dalees): If this returns a 404 do we try to delete with existing UUID? Do we just assume success?


		serverGroupName := openStackServerGroup.Name

		serverGroup, err := computeService.GetServerGroupByName(serverGroupName, false)

✨ Add OpenStackServerGroup CRD and Controller #1912

Are you sure you want to change the base?

✨ Add OpenStackServerGroup CRD and Controller #1912

Conversation

dalees commented Feb 28, 2024

k8s-ci-robot commented Feb 28, 2024

k8s-ci-robot commented Feb 28, 2024

netlify bot commented Feb 28, 2024 • edited

✅ Deploy Preview for kubernetes-sigs-cluster-api-openstack ready!

k8s-ci-robot commented Feb 28, 2024

dulek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dalees Feb 28, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dulek Feb 28, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jichenjc commented Mar 4, 2024

mdbooth commented Mar 8, 2024

chess-knight commented May 22, 2024

dalees commented May 23, 2024

netlify bot commented Feb 28, 2024 •

edited

dalees Feb 28, 2024 •

edited

dulek Feb 28, 2024 •

edited