Support IP fragmentation in eBPF

[nick-oconnor]

Expected Behavior

UDP packet fragments destined for a pod's IP which are not denied by policy arrive on the pod's interface.

Current Behavior

The eBPF data plane appears to be dropping UDP packet fragments by policy. The initial fragment is correctly forwarded from the node interface to the pod interface, but subsequent fragments do not appear on the pod's interface. When a UDP packet fragment is dropped, calico's dropped by policy counter for the interface is incremented. The pod interface eventually responds with "fragment reassembly time exceeded".

The only policies I have defined are k8s network policies. This problem does not occur when using the IPTables data plane.

Possible Solution

No idea. There may be a bug in calico's eBPF policy code.

Steps to Reproduce (for bugs)

1. Enable the eBPF data plane (kube-proxy not running, with or without DSR)
   • BGP w/ no encapsulation + dual stack (I'm unsure if this is relevant, packet captures were all IPv4)
2. Deploy a pod
3. Start a packet capture on the node running the pod
4. Send a fragmented UDP packet to the pod IP (I'm unsure how to replicate this outside of SNMP)

Context

I experienced this behavior after migrating from the IPTables data plane to the eBPF data plane. All SNMP responses exceeding the network's MTU caused my SNMP collector to timeout. I used captures from various points to determine where the packets were being dropped.

Your Environment

• Calico version: v3.28.0
• Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes 1.29.4
• Operating System and version: Ubuntu 24.04 (6.8.0-31 kernel)
• Relevant Calico config: BGP w/ no encapsulation, dual stack (added in v3.28.0)

[tomastigera]

That is correct observation. Unfortunately, ebpf dataplane does not support ip fragmentation as only the first fragment contains udp ports. The subsequent fragments cannot be matched reliably with the ongoing flow. We cannot reassemble the fragments in eBPF easily (that is a limitation of the technology). This said, we might consider some improvements/workarounds in a future release.

[nick-oconnor]

@tomastigera Wow thanks for the quick reply! Very interesting. Looks like I have some homework regarding eBPF APIs. It'll probably save folks some time by adding this to the eBPF docs for Calico.

[nick-oconnor]

Related: cilium/cilium#25709 (comment)

[tomastigera]

Thanks for the pointer. Problem with kfunc is that they are in "newer" kernels only and are not necessarily a stable API. But we could perhaps add it for kernels that have that feature! 👍

[tomastigera]

Seems like the patch ⬆️ is not present in any released kernel :(