Writing eBPF tc programs in Ubuntu 21.10

1. Install basic dependencies

apt-get install -y make gcc libssl-dev bc libelf-dev libcap-dev \
  clang gcc-multilib llvm libncurses5-dev git pkg-config libmnl-dev bison flex \
  graphviz

2. Compile and install libbpf

wget https://github.com/libbpf/libbpf/archive/refs/tags/v0.6.0.tar.gz
tar xzf v0.6.0.tar.gz
cd libbpf-0.6.0/src/
make
make install

Explanation: libbpf is included in the kernel code library. Try to choose a version that matches the kernel version. Because the Ubuntu 21.10 kernel version is 5.13, the version I chose is 0.6.0.

After installation, there are some more header files in the /usr/include/bpf/ directory:

total 324K
-rw-r--r-- 1 root root  14K Oct 28 15:04 bpf.h
-rw-r--r-- 1 root root  18K Oct 28 15:04 bpf_core_read.h
-rw-r--r-- 1 root root 3.7K Oct 28 15:04 bpf_endian.h
-rw-r--r-- 1 root root 149K Oct 28 15:04 bpf_helper_defs.h
-rw-r--r-- 1 root root 8.7K Oct 28 15:04 bpf_helpers.h
-rw-r--r-- 1 root root  21K Oct 28 15:04 bpf_tracing.h
-rw-r--r-- 1 root root  21K Oct 28 15:04 btf.h
-rw-r--r-- 1 root root  41K Oct 28 15:04 libbpf.h
-rw-r--r-- 1 root root 2.9K Oct 28 15:04 libbpf_common.h
-rw-r--r-- 1 root root 2.3K Oct 28 15:04 libbpf_legacy.h
-rw-r--r-- 1 root root  242 Oct 28 15:04 libbpf_version.h
-rw-r--r-- 1 root root 3.1K Oct 28 15:04 skel_internal.h
-rw-r--r-- 1 root root 9.6K Oct 28 15:04 xsk.h

3. Write a tc program

The following is tc-example.c, the source code of an example program, modified fromthis article: [This program defines two section, one will be loaded into tc ingress, and the other will be loaded into tc egress, and both use a global map to count the number of bytes of packets entering and exiting the interface]

#include <linux/bpf.h>
#include <linux/pkt_cls.h>
#include <stdint.h>

#include <iproute2/bpf_elf.h>

#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>

struct bpf_elf_map SEC("maps") acc_map = {
    
    
        .type = BPF_MAP_TYPE_ARRAY,
        .size_key = sizeof(uint32_t),
        .size_value = sizeof(uint32_t),
        .max_elem = 2,
        .pinning = PIN_GLOBAL_NS,
};

static __always_inline int account_data(struct __sk_buff *skb, uint32_t dir)
{
    
    
    uint32_t *bytes;

    bytes = bpf_map_lookup_elem(&acc_map, &dir);
    if (bytes)
            __sync_fetch_and_add(bytes, skb->len);

    return TC_ACT_OK;
}

SEC("ingress")
int tc_ingress(struct __sk_buff *skb)
{
    
    
    return account_data(skb, 0);
}

SEC("egress")
int tc_egress(struct __sk_buff *skb)
{
    
    
    return account_data(skb, 1);
}

char __license[] SEC("license") = "GPL";

Important: <iproute2/bpf_elf.h> The header file defines the bpf_elf_map structure. In Ubuntu 21.10, this header file has been included by the system, and the full path is /usr/include/iproute2/bpf_elf.h. If it is a lower version of the system, it may not have this header file, then you need to add the following structure and macro definition [you may also need to recompile iproute2 and libbpf, so try to choose a higher version of the system]:

#define PIN_GLOBAL_NS 2

struct bpf_elf_map {
    
    
        __u32 type;
        __u32 size_key;
        __u32 size_value;
        __u32 max_elem;
        __u32 flags;
        __u32 id;
        __u32 pinning;
};

This structure is used becauseWe will use tc this command later to load the program into the kernel, if you do not use the tc command but write your own program to load it, then you do not need to use this structure, but use the bpf_map_def structure defined in libbpf [ /usr/include/bpf/bpf_helpers.h], the corresponding map definition is changed to the following:

struct bpf_map_def SEC("maps") acc_map = {
    
    
      .type        = BPF_MAP_TYPE_ARRAY,
      .key_size    = sizeof(uint32_t),
      .value_size  = sizeof(uint32_t),
      .max_entries = 2,
      .map_flags   = LIBBPF_PIN_BY_NAME
};

4. Compile and load the program

Use clang to compile the ebpf program:

clang -O2 -Wall -target bpf -c tc-example.c -o tc-example.o

Use tc to load the ingress section and egress section of the ebpf program to the tc ingress and tc egress of an interface respectively:

tc qdisc add dev ens33 clsact
tc filter add dev ens33 ingress bpf da obj tc-example.o sec ingress
tc filter add dev ens33 egress bpf da obj tc-example.o sec egress

5. Verify program effect

Sen'anso bpftool：

apt-get install linux-tools-$(uname -r)

Then use the bpftool map command to see the map created by the program:

root@ubuntu21:~/test# bpftool map
1: array  name acc_map  flags 0x0
        key 4B  value 4B  max_entries 2  memlock 4096B

You can also view the data in the map:

root@ubuntu21:~/test# bpftool map dump id 1
key: 00 00 00 00  value: 80 21 00 00
key: 01 00 00 00  value: 02 21 00 00
Found 2 elements
root@ubuntu21:~/test# bpftool map dump id 1
key: 00 00 00 00  value: a8 22 00 00
key: 01 00 00 00  value: 2a 22 00 00
Found 2 elements

The increasing statistical value proves that our tc program is working. Done!