Unable to skipPhases: addons/coredns

[Lirt]

What happened:

When trying to install kind cluster with coredns disabled using skipPhases, the skipPhases are not honored. From the verbose log I attached you can see that skipPhases were added to InitConfiguration using kubeadmConfigPatches I used in my config (see how to reproduce).

Attached Log: kind-test.log

The result is this:

$ kubectl get pods -A
NAMESPACE            NAME                                                   READY   STATUS    RESTARTS   AGE
kube-system          coredns-76f75df574-g4xnp                               0/1     Pending   0          92s
kube-system          coredns-76f75df574-kd54l                               0/1     Pending   0          92s
kube-system          etcd-my-dev-cluster-control-plane                      1/1     Running   0          108s
kube-system          kube-apiserver-my-dev-cluster-control-plane            1/1     Running   0          106s
kube-system          kube-controller-manager-my-dev-cluster-control-plane   1/1     Running   0          108s
kube-system          kube-proxy-2gwdt                                       1/1     Running   0          92s
kube-system          kube-scheduler-my-dev-cluster-control-plane            1/1     Running   0          106s
local-path-storage   local-path-provisioner-7577fdbbfb-ct4ff                0/1     Pending   0          92s

What you expected to happen:

CoreDNS will not be installed.

How to reproduce it (as minimally and precisely as possible):

Create cluster with following configuration:

---
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  disableDefaultCNI: true
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    skipPhases:
    - addon/coredns

For example kind create cluster --name my-dev-cluster --config=kind-test.yaml -v9.

Anything else we need to know?:

Probably it is overriden here - https://github.com/kubernetes-sigs/kind/blob/v0.22.0/pkg/cluster/internal/create/actions/kubeadminit/init.go#L63-L80

Environment:

• kind version: (use kind version): kind v0.22.0 go1.20.13 linux/amd64
• Runtime info: (use docker info, podman info or nerdctl info): Server Version: 20.10.22
• OS (e.g. from /etc/os-release): Fedora 35
• Kubernetes version: (use kubectl version):

- $ kubectl version
Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2

• Any proxies or other special environment settings?: No

[aojea]

cc: @neolit123

[neolit123]

kind should not use --skip-phases and instead allow the user to pass their custom skipPhases in init configuration. the user will have the override control that way.

also v1beta3 has noderegistration. imagepullpolicy: never. skipping image pull was the main reason to skip the preflight phase in kind.

prs welcome.

[BenTheElder]

I think we're still using skip-phases as a legacy thing because the config field didn't exist when we added the flag, +1 to revisiting that

[nojnhuh]

I could take a stab at this. @neolit123 Should we keep compatibility with Kubernetes 1.22 and earlier by only using the config file fields for 1.23 and newer?

[BenTheElder]

Yes, we have many places where we introduce behavior based on the version.

Normally we do that for future versions users have not already been depending on to avoid breakage, but the behavior here is broken enough that it's probably better to just backport the change to all possible versions.

[BenTheElder]

We can consider dropping support for <= 1.22 but that should probably be a separate discussion unless it's a significant obstacle.

We haven't published a 1.22 image since https://github.com/kubernetes-sigs/kind/releases/tag/v0.20.0 (we soft-support N-3 additional releases versus upstream Kubernetes currently, buy anything not currently supported by Kubernetes upstream is best-effort), but we haven't actively broken anything for a long time and old releases mostly still work.

[nojnhuh]

Thanks for the context! So just to make sure I understand, I'll plan to add skipPhases to the v1beta3 config template, then keep the existing --skip-phases flag only when the v1beta2 config template is in effect (k8s < 1.23). Does that sound reasonable?

/assign

[BenTheElder]

Sounds good, thanks!

[Lirt]

Thank you very much for prompt discussion and PR 🙏.