LB for VIP HA doesn't work if there's no kubelet on master nodes

[YanzhaoLi]

Is your feature request related to a problem? Please describe.

LB for VIP HA doesn't work if there's no kubelet on master nodes, because it watches master node labels to add backends, see code here

kube-vip/pkg/cluster/clusterLeaderElection.go

Line 297 in ac5895a

listOptions := metav1.ListOptions{

no kubelet on masters => no master node object

Describe the solution you'd like

Maybe the ipvs LB can watch something else. For example, the kubernetes endpoint, but we need more investigation about the lifecycle of endpoint.

[lubronzhan]

Could you elaborate more of your setup. is kube-apiserver is running as a process?

Maybe the ipvs LB can watch something else. For example, the kubernetes endpoint,

What do you refer to kubernetes endpoint in this case, if there is no node object, how do you get control plane IP from kube-apiserver?

[YanzhaoLi]

• Yes. kube-apiserver is running as a process in my setup

endpoints/endpointslice named kubernetes. kube-apiserver process automatically maintains a endpoints, endpointslice and service named kuberntes in the default namespace.

$ kubectl get endpoints kubernetes -o yaml
apiVersion: v1
kind: Endpoints
metadata:
  creationTimestamp: "2024-03-07T05:33:57Z"
  labels:
    endpointslice.kubernetes.io/skip-mirror: "true"
  name: kubernetes
  namespace: default
  resourceVersion: "5728"
  uid: 8ce9c4e8-3df7-47fd-b80b-cc7d7e72ea39
subsets:
- addresses:
  - ip: 30.1.1.2
  - ip: 30.1.1.3
  - ip: 30.1.1.4
  ports:
  - name: https
    port: 8443
    protocol: TCP

[lubronzhan]

What will update the addresses section of that endpoint?

subsets:
- addresses:
  - ip: 30.1.1.2
  - ip: 30.1.1.3
  - ip: 30.1.1.4

[YanzhaoLi]

I think it's kube-apiserver. Here's my test steps

1. stop kube-controller-manager and kube-scheduler on 3 master nodes => 3 ip
2. stop all workers (make sure no kube-proxy and kubelet) => 3 ip
3. stop kube-apiserver on 1 master node => 2 ip
4. start kube-apiserver on 1 master node => 3 ip

[lubronzhan]

Ok I checked my env as well, looks like that ip is picked from node's internal ip. Then this endpoint object could be a source of truth.

Then indeed a new option could be added for ipvs, to use endpoint object to find the backend node.