Unable to use service.beta.kubernetes.io/azure-disable-load-balancer-floating-ip=true and service.beta.kubernetes.io/azure-shared-securityrule: true together with multiple services

[robotal]

What happened:

We have a number of services deployed on our AKS cluster with the annotation service.beta.kubernetes.io/azure-disable-load-balancer-floating-ip=true. According to the azure docs, this will forward traffic to the backend pools using the DIP of the nodes rather than the frontend IP that is being hit. Within the NSG rule in Azure, we can see rules that are setup looking as such

kubectl --context <context> -n <namespace> get services
NAME                                                   TYPE           CLUSTER-IP                  EXTERNAL-IP                  PORT(S)                                                    AGE
service1                                               LoadBalancer   192.168.42.133              <frontend IP 1>              80:30183/TCP,443:31941/TCP,8443:32407/TCP,8444:30200/TCP   141d
service2                                               LoadBalancer   192.168.183.95              <frontend IP 2>              443:30685/TCP                                              137d

These are two services we have set up with separate frontend IPs (e.g. the VIPs in azure terminology). Both of them use port 443 as the listener. However, since floating IP is disabled, the port in the rule is actually the backend node port that will be hit by the service (e.g. ports 31941 and 30685).

When we try adding the annotation service.beta.kubernetes.io/azure-shared-securityrule: true to the services, we lose connectivity to one of them after the NSG rule is updated.

The new state looks something like this

When we try to telnet to the frontend IP, it is unable to do so

 telnet <frontend IP 1> 443
Trying <frontend IP 1>..
^C

No rule for port 31941 is available, as the second rule will overwrite the first one. It seems that the naming of the rules will clash, as it is no longer the LB listener port that is used in the rule. I believe the logic here is what picks the name

cloud-provider-azure/pkg/provider/azure_standard.go

Line 315 in 64f0448

name = fmt.Sprintf("shared-%s-%d-%s", port.Protocol, port.Port, safePrefix)

.

From what I can tell the sharedsecuritygroup rule will only combine the destination IPs, is it possible to also combine the ports here to condense the rules a bit? (As well as source IPs)

What you expected to happen:

Connectivity to both services should still work.

How to reproduce it (as minimally and precisely as possible):

Details are listed above

Environment:

kubectl --context <context> version
Client Version: v1.28.8
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.5

[robotal]

Hmm was this fixed in #5164