-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BGP advertisement not working for LoadBalancer with externalTrafficPolicy: Local #8741
Comments
I have also tried:
But that also just works if the service is configured to use What works (even if that looks wrong to me) is:
With this configuration, connections to all 3 services are working as expected. |
The main difference between the traffic flow for cluster / local is that cluster traffic will be SNAT'd upon arrival in the cluster - I wonder if perhaps there is something dropping that traffic. Do you have any network policy on the destination pods? Or cloud firewall rules that might impact cross node traffic? |
Expected Behavior
Successful connection to services within the CIDR of the
serviceClusterIPs
while using theexternalTrafficPolicy: Cluster
service.Current Behavior
I have 3
LoadBalancer
services on my cluster, the external IP addresses are allocated by MetallLB. A calicoBGPConfiguration
is used to advertise the CIDR used for the external addresses via BGP. While I can connect to bothnging-ingress
services via the external IP (both useexternalTrafficPolicy: Local
), I cannot connect to the mqtt service, which usesexternalTrafficPolicy: Cluster
. After changingexternalTrafficPolicy: Cluster
toexternalTrafficPolicy: Local
in the servicehomeassistant/mosquitto
I can successfully connect to10.168.65.47:8883
.Services and connection tests:
❯ kubectl get service -A | grep Load nginx-public ingress-nginx-public-controller LoadBalancer 172.17.216.223 10.168.65.45 80:30199/TCP,443:30776/TCP 25d homeassistant mosquitto LoadBalancer 172.17.213.36 10.168.65.47 8883:30891/TCP 10d nginx-private ingress-nginx-private-controller LoadBalancer 172.17.33.119 10.168.65.46 80:31795/TCP,443:32707/TCP 25d ❯ nc -v 10.168.65.47 8883 -w 1 Ncat: Version 7.94 ( https://nmap.org/ncat ) Ncat: TIMEOUT. ❯ nc -v 10.168.65.45 443 -w 1 Ncat: Version 7.94 ( https://nmap.org/ncat ) Ncat: Connected to 10.168.65.45:443.
BGPConfiguration:
Test after
externalTrafficPolicy: Local
was set:Your Environment
The text was updated successfully, but these errors were encountered: