Kubeedge and edgemesh installation related to K8s cluster CNI

[ravaga]

Hello, I have a question regarding Kubeedge and edgemesh installation, more related to the interaction with the K8s cluster CNI.

Right now, I use as starting point a K8s cluster (v1.24.17) with a single node (the master node), in which I'm installing Flannel or Cilium as the cluster CNI.
Then, I install the cloud side of Kubeedge in the K8s cluster using keadm:

keadm init --advertise-address="192.168.250.208" --profile version=v1.15.0 --kube-config=/root/.kube/config

Once the cloud side is installed, I go to a VM running Ubuntu 22.04.3 LTS, in which I've previously installed containerd as container runtime, and I join this edge node to the Kubeedge cluster using keadm:

keadm join --cloudcore-ipport=192.168.250.208:10000 --token=<token> --kubeedge-version=v1.15.0 --cgroupdriver=systemd --runtimetype=remote

Now, the node has been registered in the K8s cluster and Kubeedge is working. Then, I follow the Edgemesh instructions to install it in the Kubeedge cluster and everything seems to be working properly.

However, the pod of the Flannel daemonset intended to be deployed in the edge node has always an Error status.
kube-flannel kube-flannel-ds-lfqwl 0/1 Error 937 (5m9s ago) 3d6h 192.168.250.241 kubeedge-faredge <none> <none>

So I have the following questions:

• Is this the expected working mode?
• Am I installing properly Kubeedge and Edgemesh?
• It is needed to not deploy the CNI related pods in the edge nodes (e.g. in Cilium it can be done)?
• Should I try to install another CNI in the cluster (I tried Flannel, Calico and Cilium)

[sajiduni]

Please send me the steps of installing it on Ubuntu 22.04. I am unable to do it. I have created two vms and followed all the official documentation but couldn't. Please, guide me the steps.

[ravaga]

Hello, do you mean the steps of installing both Kubeedge and Edgemesh? Only one of them?

[ravaga]

Finally, I followed this procedure and everything is working OK:

1. In the K8s cluster creation process, only deploy the CNI pods in the original K8s cluster nodes (not KubeEdge nodes running Edgecore).

   • I did it just by modifying the Flannel's Helm chart (specifically its daemonset by adding an affinity field to avoid nodes with both node-role.kubernetes.io/edge and node-role.kubernetes.io/agent labels).

     affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                - key: node-role.kubernetes.io/edge
                  operator: DoesNotExist
                - key: node-role.kubernetes.io/agent
                  operator: DoesNotExist

2. Install Kubeedge Cloudcore in the K8s cluster

3. In a future Kubeedge's edge node, before running the keadm join command, create a CNI configuration file in /etc/cni/net.d/ . For example:

{
  "cniVersion": "1.0.0",
  "name": "containerd-net",
  "plugins": [
    {
      "type": "bridge",
      "bridge": "cni0",
      "isGateway": true,
      "ipMasq": true,
      "promiscMode": true,
      "ipam": {
        "type": "host-local",
        "ranges": [
          [{
            "subnet": "10.216.3.0/24"
          }]
        ],
        "routes": [
          { "dst": "0.0.0.0/0" }
        ]
      }
    },
    {
      "type": "portmap",
      "capabilities": {"portMappings": true}
    }
  ]
}

4. Run the keadm join command with the required configuration.
5. Install Kubeedge Edgemesh to enable network connectivity among the Edge nodes.