v2.15.1

This release includes the following changes (among other things):

• Set Kubernetes default version to v1.19.9
• Remove local lb privileged (#7454)
• Check kube-apiserver up on all masters before upgrade (#7217)
• Check for dummy kernel module (#7348)
• containerd,docker: stop installing extras repo on CentOS/RHEL
• Calico: fixup check when ipipMode / vxlanMode is not present
• Update azure cloud config (#7221)
• roles/docker: Make repokey fingerprint overrideable (#7263)
• Adding other masters sequentially, not in parallel (#7166)
• calico: fix NetworkManager check (#7169)
• Remove ignore_errors from drain tasks and enable retires (#7151)
• Correct Jinja Syntax for etcd-unsupported-arch (#6919)
• Fix unintended SIGPIPEs. (#7214)
• Fix: Bastion undefined variable (#7227)
• Ensure when use_oracle_public_repo is set to false the public Oracle
• Fix ansible calico route reflector tasks in calico role (#7224)
• Run containerd related tasks on OracleLinux. (#7250)
• Remove deletion of coredns deployment. (#7211)
• Fix Restart network doesn't work on Fedora CoreOS (#7271)
• Only use stat get_checksum: yes when needed (#7270)
• Fixup cri-o metacopy mount options (#7287)
• Ensure kubeadm doesn't use proxy (#7275)
• Ensure we gather IPv6 facts
• Add privileged_without_host_devices support (#7343)
• Auto renew control plane certificates (#7358)
• Fix k8s-certs-renew for k8s < 1.20 (#7410)
• Fixup kubelet.conf to point to kubelet-client-current.pem (#7347)
• Fix "api is up" check (#7295)
• Fix remove-node by removing jq usage (#7405)
• Fix reset when using containerd (#7308)
• Fix proxy usage when *_PROXY are present in environment (#7309)
• Fix the filename </etc/vault> is Duplicate in the reset role. (#7313)
• Fix recover-control-plane undefined 'proxy_disable_env' variable (#7326)
• Fix: added string to bool conversion for use_localhost_as_kube api load balancer (#7324)

v2.15.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Remove support for Fedora 31 (EOL)
• Remove support for Contiv CNI (#6964)
• Remove hyperkube support, no longer available in Kubernetes (#6965)
• Helm 2 can no longer be installed (#6846)

Major changes

• Add support for Fedora 33 (#7072) (see Notes)
• Add Kata Containers support to CRI-O runtime (#6830)
• Add RHEL support subscription registration (#6572)
• Add crun support (#6864)
• Add etcd tls cipher suites support (#7001)
• Add GCP terraform support (#6974)
• Allow airgapped CRI-O installation (#6927)
• Harden reset to work in more cases (#6781)
• Disable Kubernetes Dashboard by default (#6804) (see Notes)
• Add an option to force apiserver and respective client certificate to be regenerated without upgrading (#6403)
• Add a script to collect necessary container images and register the images to local registry (#7024)
• Major proxy rework on different playbooks (#7095)

Applications

• Allow configuration of nodelabels in local_volume_provisioner (#6620)
• [Openstack] Add external_openstack_lbaas_provider setting for occm (#6566)
• [Openstack] Add security groups not managed by terraform (#6865)
• [Openstack] Do not apply floating IP's before router port is created (#6887)
• [Openstack] Add cluster-name to external-openstack-cloud-controller-manager (#7055)
• [Azure / AWS] Added support for dynamic tags in AWS and Azure (#6752)

Container managers

• [All] Remove libseccomp install tasks (#7074)
• [Containerd] Add registry mirror support (#6962)
• [Containerd] Ensure libseccomp is installed before starting containerd on CentOS 8 (#6922)
• [Containerd] Add download run once feature (#6997)
• [Containerd] Allow root path and state path to be configured (#7098)
• [CRI-O] Use system default for storage driver by default (#6637)
• [CRI-O] Ensure service is started and enabled (#6753)
• [CRI-O] Reset is now working when CRI is set to CRI-O (#6812)
• [CRI-O] Avoid extra restart after install and upgrade (#6882)
• [CRI-O] Disable CRI-O restart by Multus (#6930)
• [CRI-O] Add registry mirror support (#6977)
• [CRI-O] Allow to enable download_run_once (#6998)
• [Docker] Add CentOS 8 and Fedora 32 docker repository (#6747)

Network

• [Weave] Add iptables_backend to weave options (#6639)
• [Calico] Add support for Calico CNI host-local IPAM plugin (#6580)
• [Calico] Added ability to set VXLAN vni and port. defaults to calico's documented default (#6678)
• [Calico] default to using kdd datastore (#6693)
• [Calico] Add retries to update calico-rr data in etcd through calicoctl (#6505)
• [Calico] Handle calico-rr nodes as workers so they get upgraded too (#6447)
• [Calico] Avoid POD restart during initial deploy (#6886)
• [Calico] Add serviceExternalIPs option for calico installation (#6928)
• [Calico] Update files to handle multi-asn bgp peering conditions (#6971)
• [Calico] Blacklist Calico's VXLAN interface from NetworkManager (#7037)
• [Calico] Check if inventory settings match cluster settings (#6969)
• [Flannel] Add multi architeture support to flannel (#6166)

Other note worthy changes

• Allow pre-existing floating IPs to be specified with k8s_master_fips (#6755)
• Set ansible_python_interpreter to python3 on debian (#6633)
• Allow resource management of metrics-server container (#6652)
• Use "kubeadm join" to join masters to control plane (#6661)
• Add new variable allowing additionnal audit webhook server configuration (#6726)
• Add leader election timeouts and durations to available parameters (#6691)
• Make sure node_ip is set if node is in etcd group (#6719)
• Install etcdctl to host when etcd deployment type is kubeadm (#6857)
• Chmod kubeconfig to avoid group-readable (#6800)
• Hold the docker-ce-cli upgrade in Debian (#6995)
• Removes apps tags from apps meta dependencies (#7041)
• Change owner to root for bin_dir directory (#6814)
• Add an option to disable globally applying a proxy to etc/yum.conf (#6828)
• Set feature gates in kube-proxy ConfigMap (#6851)
• Allow configuring container log limits for Kubelet (#6933) (see Notes)
• Remove executable bit from yaml and j2 files (#6894)
• Fails if kubeadm_version do not matches kubernetes version (#6302)
• Disable docker-ce yum repo by default (#7080)
• Improve reset with many tweak (#7094)
• Small Proxy fixes (add svc,svc.{{ dns_domain }} to no_proxy) (#7102)
• Restore ability to set pod eviction timer (#7114) (see Notes)
• Add ping_access_ip variable to enable/disable ping test during preinstall. Enabled by default (#7020)
• Remove unnecessary condition check when updating server field in kube-proxy kubeconfig (#7145)

Component versions:

• Kubernetes v1.19.7
• Etcd 3.4.13
• Docker 19.03
• Containerd 1.3.9
• CRI-O 1.19
• CNI-plugins v0.9.0
• Calico v3.16.5
• Cilium 1.8.6
• Flannel 0.13.0
• Kube-Router 1.1.1
• Multus 3.6
• Kube-ovn 1.5.2
• Weave 2.7.0
• CoreDNS 1.7.0
• Nodelocaldns 1.16.0
• Helm 3.3.4
• Nginx-ingress 0.41.2
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.1.0

Known issues

• Ansible 2.10 is not supported and using it will results in errors (cf #7130)

Notes

• Kubernetes Dashboard deployment needs to be explicitly configured with dashboard_enabled: true
• Docker version for Fedora 33 needs to be set to 20.10 as they are the only packages available and validated
• Two new variables are used for this use case kube_apiserver_pod_eviction_not_ready_timeout_seconds and kube_apiserver_pod_eviction_unreachable_timeout_seconds
• Action required: users that relies on the default value of calico_datastore needs to explicitly configure their datastore choice.

v2.14.2

This release includes the following changes:

• Set ansible_python_interpreter to python3 on debian (#6744)
• Kubernetes v1.18.10 as default (#6842)
• Fix cinder & external_openstack cacert deployment (#6832)
• Fix unintended SIGPIPE (#6817)

v2.14.1

This release includes the following changes:

• NetworkManager lists must be separated by , (#6649)
• Move from widehat.opensuse to download.opensuse for crio centos (#6682)
• fix kubelet_flexvolumes_plugins_dir undefined (#6670)
• Add Kubernetes hashes 1.19.2/1.18.9/1.17.12 and set default (#6699)
• Make sure node_ip is set if node is in etcd group (#6720)
• properly generate extravolumes in kubeadmconfig for centos (#6707)

v2.13.4

This release includes the following changes:

• Add Kubernetes hashes 1.17.12 and set default (#6701)
• Add missing 'Set up proxy environment' tasks (#6591)
• Add Kubernetes 1.16.15 hashes (#6626)

v2.12.10

This release includes the following changes:

• Kubernetes 1.16.15 (#6583)

v2.14.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Removed support for Fedora 29 and 30 (EOL)
• Remove support for CoreOS Container Linux (EOL)

Major changes:

• Add Oracle Linux 8 support and fixes (#6198)
• Add Ubuntu 20.04 support (#6157)
• Add support for Fedora 32 (#6426)
• Add support for Kata Containers (#6256)
• Switch to Python3 on Debian & Ubuntu (#6157)
• Add Ambassador OSS ingress controller (#6135)
• Add ovn4nfv-k8s-plugin as network plugin (#6381)
• Improve air-gap installation instructions (#6234)
• Add TLS cipher suites support for kubeadm and kubelet (#6024 #6490)
• Update most ETCDCTL_API call to v3 (#5998)
• Upgrade molecule to v3 (#6468)
• Remove-node play will now fail if node can not be drained (#6442)

Applications

• [Azure] Update documentation with az command (#6042)
• [Azure] Add azure_cloud parameter to cloud_config file (#6321)
• [CSI] Update CSI containers to latest versions (#6221)
• [MetalLB] Option to talk BGP (#6383)
• [MetalLB] The deployment becomes one of addons. You can deploy it with a new option metallb_enabled (#6238)
• [Openstack] Support volume type (#6524) (See Notes)
• [Openstack] Make it possible to open additional ports on masters (#6547)
• [Openstack] Add support for application credentials (#6534)
• [Openstack] Add snapshot-controller for CSI drivers (#6537)
• [Openstack] Added a default volumesnapshotclass for Cinder CSI (#6537)

Container managers

• Match docker-cli version with docker-engine version (#6163)
• [Docker] Set cgroup driver by default to systemd (#6563)
• [Containerd] Install package is now managed alongside docker (#6218)
• [Containerd] Add support for Fedora (#6094)
• [CRI-O] Use OS packaging default value for apparmor_profile in crio.conf (#6125)
• [CRI-O] Fix kubelet cgroup driver detection (#6331)
• [CRI-O] Align template crio.conf with upstream and set cgroup driver by default to systemd (#6432)
• [CRI-O] Harden downloads with retry (#6374)
• [CRI-O] Add variable to configure unsecure pull (#6568)

Network

• [Weave] Allow Weave DS to support any taint effect (#6159)
• [Calico] Disable bird-check flag for probes of calico-node pods when calico_network_backend is not bird (#6217)
• [Calico] Add FELIX_DEVICEROUTESOURCEADDRESS option (#6508)
• [Kube-Router] Enable portmap CNI plugin with kube-router to allow use of hostPort in container specs (#6204)
• [Kube-Router] Add selectable dns policy (#6586)
• [Cilium] Add a way to deploy cilium alongside another CNI (#6373)
• [Cilium] Add option to configure IPVS timeouts in kube-proxy configration manifest (#6396)
• [Cilium] Support the overwrite of MTU in Cilium agents (#6329)
• [Cilium] Add metrics in Cilium operator and add hubble metrics port in agents (#6513)
• [Cilium] Add hubble server support in cilium (#6575)

Other note worthy changes

• Create custom dashboard namespace if specified (#6107)
• Add support to expose etcd metrics on a custom port (#6092)
• Add additional network configuration options to external Openstack (#6085)
• Fix resolv.conf configuration for Fedora CoreOS (#6138)
• Replace seccomp profile docker/default with runtime/default (#6170)
• Multiples fixes for proxy and no_proxy variables (#6112 #6431 #6558)
• Use connection: local when delegate_to: localhost (#6322)
• Add DNS configuration in NetworkManager for Fedora CoreOS (#6291)
• Allow kubeadm to upgrade etcd (#6345) (See notes)
• Add docs for setting up your first cluster (#6544)
• Webhook authorization can now be enabled using inventory variable (#6502)
• Uncordon node that fail to drain (thus failing its upgrade) during upgrade procedure (#6546)
• Added variable kubelet_rotate_server_certificates which enables kubelet server certificate rotation (#6453)
• Add protectKernelDefaults option (default true) to kubelet config file (#6611)

Component versions:

• Kubernetes v1.18.8
• Etcd 3.4.3
• Docker 19.03
• containerd 1.2.13
• Cri-O 1.18
• CNI-plugins v0.8.7
• Calico v3.15.2
• Cilium 1.8.3 (See Notes)
• Contiv 1.2.1
• Flannel 0.12.0
• Kube-Router 1.0.1 (see Notes)
• Multus 3.6
• kube-ovn 1.3.0 (see Notes)
• Weave 2.7.0
• CoreDNS 1.6.7
• nodelocaldns 1.15.13
• Helm 3.2.4
• nginx-ingress 0.35.0
• cert-manager 0.16.1 (see Notes)
• Kubernetes Dashboard v2.0.4
• Oracle OCI: v0.7.0

Known issues

None

Notes

• etcd will now be upgraded and its certs renewed when using a kubeadm managed etcd (etcd_kubeadm_enabled: true)
• Cilium: Check upgrade guide regarding update to 1.8.0
• Kube-Router: Upgrade to 1.0.0 require an iptable flush
• Kube-ovn is now installed in kube-system namespace, version priori to 1.0.0 should be removed manually
• Cert-Manager: Refer to README.md prior to upgrading in your exisitng Kubernetes cluster
• Openstack: If the nova API is before Stein, Terraform will work but the new volume type feature will not be available. If the entire cloud is upgraded to Stein or later, the new feature can be used. However if the nova versions in the cloud are mixed, with nova server API >= Stein and any nova-compute node < Stein, you will get a HTTP 409 error and VolumeTypeSupportNotYetAvailable exception.

v2.13.3

This release includes the following changes:

• Kubernetes v1.17.9 (#6435)
• bugfix if openstack_cacert is a base64 string (#6475)
• bugfix no_proxy support (#6346)

v2.12.9

This release includes the following changes:

• bugfix if openstack_cacert is a base64 string (#6371)

v2.12.8

This release includes the following changes:

• Kubernetes 1.16.13 (#6477)