There are many ways/tools for network development. If you think about it carefully, it should be divided into the following categories.
- Hardware-
specific integrated circuits: Asic (Application Specific Integrated Circuit)
programmable hardware with fixed processing logic represented by switching chips : such as FPGA, NP; there is also a relatively new broadcom OF-DPA that supports openflow, and barefoot Tofino that supports P4 . - Software
kernel mode: driver, TCP/IP protocol stack, XDP
user mode: socket, tun, tan
user/kernel combination: DPDK, Netmap
Compared to other network development methods/tools, XDP appeared later. So first give a brief introduction to XDP.
Introduction to XDP
XDP (eXpress Data Path), first appeared in Linux kernel version 4.8, based on eBPF implementation. Speaking of eBPF, it is necessary to introduce BPF (Berkeley Packet Filter, Berkeley Packet Filter) at the beginning of all this. BPF was introduced to the Linux kernel very early. For me or probably most people, the first time I heard about BPF, it is estimated that it is using tcpdump, libpcap, etc. and network packet capture BPF syntax. As below
'host 192.168.0.1 and (port 80 or port 8080)'
The original use of BPF is indeed here. Usually, packet capture is to analyze and locate the communication problem of a certain network application, that is, not to care about all the traffic on the network card. If you want to filter out the network data of the relationship from a large number of network messages. One way is that the kernel sends all network messages to the user mode, and a user mode program is used for filtering. The shortcomings of this method are obvious, each message has to go through the kernel to the user mode memory copy. Therefore, the BPF method provides a filter type, which is dynamically encoded into a BPF target code (intermediate code), and finally the kernel BPF virtual machine completes the execution of the BPF target code, that is, completes the function of message filtering. BPF only sends the packets that meet the filtering conditions to the user mode.
The BPF introduced above is now called classical BPF. Quote wiki information.
"EBPF was designed by Alexei Starovoitov while working at PluMgrid. This company focused on researching new ways to design software-defined network solutions. When it was just a proposal, Daniel Borkmann, a kernel engineer at Red Hat, helped modify it Can enter the kernel code and completely replace the existing BPF implementation. This is the first major update of BPF in two decades, making BPF a general-purpose virtual machine." At
this point, the Linux kernel opened a window to developers. This window not only improves the observability of the system, such as kprobes, uprobes, tracepoints and perf_events support. (Note: Personal opinion, ebpf's contribution to system observability is far greater than network processing).
The following picture gives a good introduction to the composition and overall processing of XDP.
1) Write the BPF code in user mode and compile it into a BPF object
2) BPF loader downloads BPF to the BPF virtual machine (this virtual machine is not a Kvm virtual machine, and I personally understand it as similar to jvm)
3) Different action code return codes , Defines the subsequent processing method of the message:
XDP_PASS: The message continues to be sent to the protocol stack
XDP_DROP: Packet loss
XDP_ABORTED: Packet loss and tracepoint exception
XDP_TX: Forward the message from the received network card
XDP_REDIRECT: The message is redirected to another network card or sent to the user mode AF_XDP socke
Careful Have you found that XDP can only process packets in the receiving direction, but cannot process packets in the sending direction of the machine. This seems to be less flexible than netfilter. But as an intermediate network forwarding device, XDP only processes packets in the receiving direction, which seems to be fine. The following is a simple example of XDP application based on the processing of the mobile core network UPF processing GTPU tunnel. The code is at https://github.com/801room/upf-xdp. The complete toy code, but the basic technical points and solutions should be reflected.
Simple example
The code is divided into two parts. One part is written for go, loads the BPF object into the kernel and binds it to the specified network card, and can set the map to interact with the bpf object as a control plane.
err := bpf.LoadElf(elf)
…
xdp := bpf.GetProgramByName("upf_input")
…
err = xdp.Attach(iface)
The other part is written in c and compiled into a BPF object as the data plane.
SEC("xdp")
int upf_input(struct xdp_md *ctx)
{
void *data = (void *)(long)ctx->data;
struct ethhdr *eth = data;
return eth_handle(ctx, eth);
}
The compiler will compile SEC ("xdp") into a section in elf, bpf.GetProgramByName will look for this section and find the function entry.
The following code is processed layer by layer from the second layer, but there is no special explanation. It should be noted that there cannot be loops without boundaries in xdp, and there can be no global variables. When accessing the content of the message, first verify the effective boundary of the access. For example, the following code will report an error when loading bpf.
static u32 eth_handle(struct xdp_md *ctx, struct ethhdr *ethh)
{
u16 eth_type;
u64 offset;
struct vlan_hdr *vlan_hdr;
eth_type = htons(ethh->h_proto);
switch(eth_type) {
...
}
}
This code needs to be modified to determine whether it is out of range before access.
static u32 eth_handle(struct xdp_md *ctx, struct ethhdr *ethh)
{
void *data_end = (void *)(long)ctx->data_end;
u16 eth_type;
u64 offset;
struct vlan_hdr *vlan_hdr;
offset = sizeof(*ethh);
if((void *)ethh + offset > data_end) {
bpf_debug("Cannot parse L2\n");
return XDP_PASS;
}
eth_type = htons(ethh->h_proto);
switch(eth_type) {
...
}
}
Finally, I want to talk about bpf_xdp_adjust_head(), this function can modify the message header, push or pull. See the definition of specific bpf related functions.
https://man7.org/linux/man-pages/man7/bpf-helpers.7.html
summary
Compared with P4, Xdp has a much lower threshold and learning curve for Linux network developers. For simple forwarding and ACL functions. Using xdp is fast and easy. If it is a tunnel encapsulation/unencapsulation process, xdp should also be handy.
And just like P4, it may be a little difficult for in-depth message analysis and complex policy control. The author's ultimate goal is to write a UPF. After a brief investigation of P4 and XDP, let's embrace VPP obediently.