CRDs are not deleted when provider is removed

[Fedosin]

What steps did you take and what happened:

1. Install a provider:

apiVersion: operator.cluster.x-k8s.io/v1alpha1
kind: CoreProvider
metadata:
  name: cluster-api
  namespace: capi-system
spec:
  version: v1.4.2

2. Delete the provider
3. Ensure that all previously created CRDs from cluster.x-k8s.io are not deleted together with the provider.

What did you expect to happen:
All created resources should be removed.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-operator version: main
• Cluster-api version: 1.4.2
• Minikube/KIND version: 1.27
• Kubernetes version: (use kubectl version): 1.27
• OS (e.g. from /etc/os-release):

/kind bug
[One or more /area label. See https://github.com/kubernetes-sigs/cluster-api-operator/labels?q=area for the list of labels]

[Fedosin]

After some consideration it seems like a bad idea, because we will also need to delete all user created objects like Cluster or Machine.

[Fedosin]

After some consideration it seems like a good idea, but we need to differentiate scenarios when we delete resources during provider upgrade, and when we delete the provider itself completely.

[alexander-demicev]

Another option is to introduce a deletion policy API field, which doesn't remove CRDs by default

[alexander-demicev]

/triage accepted