Release Proposal v2.17

[floryut]

Below is a proposal for release notes:

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Drop support for Fedora 32 (Add Fedora 34 image and CI, remove Fedora 31 (EOL) + update docker packages #7657)

Major changes

• Add support for Fedora 34 (Add Fedora 34 image and CI, remove Fedora 31 (EOL) + update docker packages #7657)
• Add Debian 11 (bullseye) support (kubernetes/preinstall: Add Debian 11 (bullseye) support #7853)
• Enable Graceful Node Shutdown for Kubernetes >= 1.21.0 (Enable Graceful Node Shutdown for Kubernetes >= 1.21.0 #7746)
• Move to Ansible 3.x by default (Move to Ansible 3.4.0 #7672) (see Notes 1)
• Set selinux type t_etc if selinux state is enforcing (set selinux type t_etc if selinux state is enforcing #7791)
• Add Infomaniak to compatible public clouds list (Add Infomaniak to compatible public clouds list #7910)
• During pre-upgrade add a flag to always cordon (During pre-upgrade add a flag to always cordon #7892) (see Notes 2)
• Update Terraform 0.15 to tf validated and tested versions (Add Terraform 0.15, remove 0.13, update 0.14 and add retry to packet PR #7927)
• Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA (Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA #7938) (see Notes 5)
• Inventory builder can now add IP to inventory (Fix: adding new ips with inventory builder (#7577) #7583) (see Notes 6)
• Add a new option kubeadm_upgrade_auto_cert_renewal to control certificates renewal during control plane upgrade (Add option to kubeadm upgrade command to control certificates renewal during control plane upgrade #7976)

Applications

• [Openstack] Openstack cloud config: store cloud.conf and API CA cert in k8s secret and avoid writing them to disk (store openstack external cloud controller ca.cert in a k8s secret instead of the host filesystem #7603)
• [vSphere] vSphere credentials can now be passed as environment variables (Added possibility to specify vSphere credentials via env variables #7646)
• [vSphere] Update vSphere CPI ClusterRole according to the latest official CPI manifests (Update vSphere CPI ClusterRole and docs #7838)
• [vSphere] Add suport of Vsphere CSI driver 2.2.X versions (Add suport of Vsphere CSI driver 2.2.X versions #7848)
• [Cinder] Add cinder_csi_ignore_volume_az (Add cinder_csi_ignore_volume_az #7624)
• [Cinder] Added support for application credentials for cinder-csi (Add application credentials support for cinder #7799)
• [Cinder] Added support for sourcing application credentials from environment variables (Add application credentials support for cinder #7799)
• [MetalLB] Update to v0.10.2 (MetalLB: update to v0.10.2 #7925)
• [MetalLB] Update default variable: keep nodeSelector in one place (MetalLB: keep nodeSelector in one place #7931)
• [CSI] Update CSI snapshotter and allow enabling it stand-alone (Update CSI snaphotter and make it independent #7943)
• [nginx-ingress] Bump to 1.0.0 to support kube 1.22 (nginx-ingress: bump to 1.0.0 to support kube 1.22 #7942) (see Notes 3) (see Notes 4)
• [UpCloud] Updated terraform script to use private network and dynamic additional disks (Updated UpCloud terraform script to use private network and dynamic additional disks #7779)

Container managers

• [Kata-container] Replace deprecated 1.x version of Kata containers with the new 2.x (Replace Kata 1.x with Kata 2.x #7670)
• [gVisor] Add initial support for gVisor container runtime (gVisor: initial support for gVisor container runtime #7661)
• [CRI-O] Allow cri-o offline install (Allow cri-o offline install #7777)
• [CRI-O] Add cri-o to support secure/insecure registry authentication (cri-o registry auth support #7837)
• [Containerd] Enable containerd on Fedora CoreOS (add containerd on fedora CoreOS #7794)
• [Containerd] Add containerd on Flatcar Container Linux (Add containerd on Flatcar Container Linux #7681)
• [Containerd] Add containerd secure/insecure registry authentication support (feat(containerd): auth support #7868)

Network

• [Calico] Add support for Calico 3.19.1 (Calico: add support for v3.19.1 #7630)
• [Calico] Add retries to 'Set label for route reflector' task (Add retries to 'Set label for route reflector' task #7645)
• [Calico] Support enabling the eBPF dataplane for Calico (Calico enable support for eBPF #7618)
• [Calico] Add Wireguard support (Calico wireguard #7638)
• [Calico] Use --allow-version-mismatch in calicoctl.sh to allow upgrades (Calico: use --allow-version-mismatch in calicoctl.sh to allow upgrades #7873)
• [Calico] kube_service_addresses_ipv6 is now added to serviceClusterIPs if enable_dual_stack_networks is true (Calico: Add kube_service_addresses_ipv6 to serviceClusterIPs #7944)
• [Cilium] Add cilium_operator_api_serve_addr to cilium operator config (add cilium_operator_api_serve_addr to cilium operator config #7901)

Other note worthy changes

• Add nodeSelector for other services and node labels before CNI setup (Add nodeSelector for other services and node labels before CNI setup #7613)
• Allow deployers to limit the interface on which nodelocaldns exposes its prometheus listening port (nodelocaldns: allow binding metrics address to host IP #7748)
• Ubuntu changed package name python-apt to python3-apt (Ubuntu changed package name python-apt to python3-apt  #7769)
• Retry to fetch binary if it fails first time (retry to fetch binary if it fails first time #7839)
• Remove environment variable in remove-node play (Remove environment variable in remove-node play #7729)
• addons/cert_manager: Retries until webhook pods has been created (addons/cert_manager: retries until webhook pods has been created #7850)
• Add tags: always to all included service playbook (add tags: always to all included sevice playbook #7906)
• Use --no-cache-dir flag to pip in dockerfiles to save space (chore : use --no-cache-dir flag to pip in dockerfiles to save space #7898)

Component versions:

• Kubernetes v1.21.5
• Etcd 3.4.13
• Docker 20.10
• Containerd 1.4.9
• CRI-O 1.21
• CNI-plugins v0.9.1
• Calico v3.19.2
• Cilium 1.9.10
• Flannel 0.14.0
• Kube-ovn 1.7.2
• Kube-Router 1.3.0
• Multus 3.7.2
• ovn4nfv v1.1.0
• Weave 2.8.1
• CoreDNS 1.8.0
• Nodelocaldns 1.17.1
• Helm 3.6.3
• ambassador: v1.5
• Nginx-ingress 1.0.0
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.3.1

Known issues

• Ubuntu-16 won't work with default containerd version (1.4.9) as packages are not available, please use 1.4.6

Notes

1. Users need to uninstall ansible 2.9 to be able to install on top ansible 3.x which was split between ansible-base and ansible-collections.
2. Setting roles/upgrade/pre-upgrade/defaults/main.yml:upgrade_node_always_cordon to true causes a node to be drained before an upgrade and uncordoned after an upgrade even if the node is not cordoned when the upgrade begins.
3. Ingress-nginx: upgrade to 1.0.0 with stable ingress API, this version requires explicitly setting kubernetes.io/ingress.class: nginx on managed ingresses
4. ⚠️ nginx-ingress 1.0 does not support networking.k8s.io/v1beta
5. Flag --dynamic-config-dir has been deprecated, Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA. It is planned to be removed from Kubernetes in the version 1.23. Please use alternative ways to update kubelet configuration.
6. The dynamic inventory builder will by default overwrite the inventory config. This was previously unintended behavior. In order to add new hosts into the already existing inventory config use the add command e.g. $ inventory.py add 10.0.1.8

[oomichi]

Thanks for this work @floryut !

[floryut]

Thanks for this work @floryut !

Thanks to you too for your continued work 🙇
I'll try to release tonight, otherwise tomorrow.

[RaulButuc]

Thank you very much for your work!

Regarding the recent vulnerability announced here https://groups.google.com/g/kubernetes-announce/c/-e9OlTcED5E can we expect K8s v1.21.5 to be released with kubespray tag v2.17.0?

[floryut]

Thank you very much for your work!

Regarding the recent vulnerability announced here https://groups.google.com/g/kubernetes-announce/c/-e9OlTcED5E can we expect K8s v1.21.5 to be released with kubespray tag v2.17.0?

yup the PR is ready, I'm waiting for a lgtm to release this version 👍

[floryut]

🎉

[floryut]

And crap.. Found an issue with default k8s version, will retag/rerelease in a few hours 😫
(#7977)

[RaulButuc]

No worries, and thank you very much once again! 😄