k3s-deploy

Deploy a K3S multi-node cluster with Ansible.

Motivation

Installing a single node K3S cluster is trivial (curl -sfL https://get.k3s.io | sh -). This repository aims at providing :

• A realistic multi-node cluster to illustrate monitoring and storage with Kubernetes (ReadWriteOnce, ReadWriteMany,...)
• A discovery cluster for mborne/cours-devbox where vagrant and Ansible are presented before Kubernetes.

Requirements

Copy and adapt inventory/vagrantbox or :

• Start some VM with vagrantbox
• Ensure that SSH connexion is working : ansible -i inventory/vagrantbox all -m ping

Parameters

See roles/k3s/defaults/main.yml.

Usage

Download external roles

ansible-galaxy install -r roles/requirements.yml

Deploy K3S

# Deploy K3S with default params :
ansible-playbook -i inventory/vagrantbox playbooks/k3s.yml
# Deploy K3S with a docker mirror :
# ansible-playbook -i inventory/vagrantbox playbooks/k3s.yml -e k3s_docker_mirror=https://docker-mirror.quadtreeworld.net

Configure kubectl

# In k3s-deploy directory :
export KUBECONFIG=$PWD/output/kubeconfig.yml
# List nodes
kubectl get nodes

See roles/k3s/tasks/fetch-config.yml

Check kubectl config

• kubectl cluster-info :

• kubectl get nodes -o wide :

Have fun with kubernetes!

• kubernetes.io
• container.training
• mborne/docker-devbox

Uninstall K3S

# uninstall k3s on agent nodes
ansible -i inventory/vagrantbox k3s_agent -m shell -a "k3s-agent-uninstall.sh" --become
# uninstall k3s on master node
ansible -i inventory/vagrantbox k3s_master -m shell -a "k3s-uninstall.sh" --become

Advanced usage

Installing NFS server on master node

# Install NFS on vagrantbox-1
ansible-playbook -i inventory/vagrantbox playbooks/nfs-server.yml
# Check from vagrantbox-2
ssh vagrant@vagrantbox-2 showmount -e vagrantbox-1

Enabling OIDC on K3S

See docs/oidc.md - K3S - OIDC experimentation with Keycloak

Alternatives

Using Kubernetes in docker (Kind) also allows to create a realistic multi-node cluster (see mborne/docker-devbox - kind - quickstart.sh)

License

MIT

Notes

• Playbook is based on curl -sfL https://get.k3s.io | sh -s -.

• Playbook is not intented to be used for production

• Playbook is configured throw the following files :

  • inventory/vagrantbox/group_vars/all/k3s.yml
  • inventory/vagrantbox/group_vars/k3s_master/k3s.yml
  • inventory/vagrantbox/group_vars/k3s_agent/k3s.yml

• Traefik is disabled (--disable traefik) as it is a good exercise to understand its deployment under kubernetes (see mborne/docker-devbox - traefik)

• --flannel-iface=eth1 is important as eth0 is NAT network created by vagrant (see stackoverflow.com - Is there any way to bind K3s / flannel to another interface?)