1. Install basic dependencies
apt-get install -y make gcc libssl-dev bc libelf-dev libcap-dev \
clang gcc-multilib llvm libncurses5-dev git pkg-config libmnl-dev bison flex \
graphviz
2. Compile and install libbpf
wget https://github.com/libbpf/libbpf/archive/refs/tags/v0.6.0.tar.gz
tar xzf v0.6.0.tar.gz
cd libbpf-0.6.0/src/
make
make install
Explanation: libbpf is included in the kernel code library. Try to choose a version that matches the kernel version. Because the Ubuntu 21.10 kernel version is 5.13, the version I chose is 0.6.0.
After installation, there are some more header files in the /usr/include/bpf/ directory:
total 324K
-rw-r--r-- 1 root root 14K Oct 28 15:04 bpf.h
-rw-r--r-- 1 root root 18K Oct 28 15:04 bpf_core_read.h
-rw-r--r-- 1 root root 3.7K Oct 28 15:04 bpf_endian.h
-rw-r--r-- 1 root root 149K Oct 28 15:04 bpf_helper_defs.h
-rw-r--r-- 1 root root 8.7K Oct 28 15:04 bpf_helpers.h
-rw-r--r-- 1 root root 21K Oct 28 15:04 bpf_tracing.h
-rw-r--r-- 1 root root 21K Oct 28 15:04 btf.h
-rw-r--r-- 1 root root 41K Oct 28 15:04 libbpf.h
-rw-r--r-- 1 root root 2.9K Oct 28 15:04 libbpf_common.h
-rw-r--r-- 1 root root 2.3K Oct 28 15:04 libbpf_legacy.h
-rw-r--r-- 1 root root 242 Oct 28 15:04 libbpf_version.h
-rw-r--r-- 1 root root 3.1K Oct 28 15:04 skel_internal.h
-rw-r--r-- 1 root root 9.6K Oct 28 15:04 xsk.h
3. Write a tc program
The following is tc-example.c, the source code of an example program, modified fromthis article: [This program defines two section, one will be loaded into tc ingress, and the other will be loaded into tc egress, and both use a global map to count the number of bytes of packets entering and exiting the interface]
#include <linux/bpf.h>
#include <linux/pkt_cls.h>
#include <stdint.h>
#include <iproute2/bpf_elf.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>
struct bpf_elf_map SEC("maps") acc_map = {
.type = BPF_MAP_TYPE_ARRAY,
.size_key = sizeof(uint32_t),
.size_value = sizeof(uint32_t),
.max_elem = 2,
.pinning = PIN_GLOBAL_NS,
};
static __always_inline int account_data(struct __sk_buff *skb, uint32_t dir)
{
uint32_t *bytes;
bytes = bpf_map_lookup_elem(&acc_map, &dir);
if (bytes)
__sync_fetch_and_add(bytes, skb->len);
return TC_ACT_OK;
}
SEC("ingress")
int tc_ingress(struct __sk_buff *skb)
{
return account_data(skb, 0);
}
SEC("egress")
int tc_egress(struct __sk_buff *skb)
{
return account_data(skb, 1);
}
char __license[] SEC("license") = "GPL";
Important: <iproute2/bpf_elf.h>
The header file defines the bpf_elf_map
structure. In Ubuntu 21.10, this header file has been included by the system, and the full path is /usr/include/iproute2/bpf_elf.h
. If it is a lower version of the system, it may not have this header file, then you need to add the following structure and macro definition [you may also need to recompile iproute2 and libbpf, so try to choose a higher version of the system]:
#define PIN_GLOBAL_NS 2
struct bpf_elf_map {
__u32 type;
__u32 size_key;
__u32 size_value;
__u32 max_elem;
__u32 flags;
__u32 id;
__u32 pinning;
};
This structure is used becauseWe will use tc
this command later to load the program into the kernel, if you do not use the tc
command but write your own program to load it, then you do not need to use this structure, but use the bpf_map_def
structure defined in libbpf [ /usr/include/bpf/bpf_helpers.h], the corresponding map definition is changed to the following:
struct bpf_map_def SEC("maps") acc_map = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(uint32_t),
.value_size = sizeof(uint32_t),
.max_entries = 2,
.map_flags = LIBBPF_PIN_BY_NAME
};
4. Compile and load the program
Use clang
to compile the ebpf program:
clang -O2 -Wall -target bpf -c tc-example.c -o tc-example.o
Use tc
to load the ingress section and egress section of the ebpf program to the tc ingress and tc egress of an interface respectively:
tc qdisc add dev ens33 clsact
tc filter add dev ens33 ingress bpf da obj tc-example.o sec ingress
tc filter add dev ens33 egress bpf da obj tc-example.o sec egress
5. Verify program effect
Sen'anso bpftool
:
apt-get install linux-tools-$(uname -r)
Then use the bpftool map command to see the map created by the program:
root@ubuntu21:~/test# bpftool map
1: array name acc_map flags 0x0
key 4B value 4B max_entries 2 memlock 4096B
You can also view the data in the map:
root@ubuntu21:~/test# bpftool map dump id 1
key: 00 00 00 00 value: 80 21 00 00
key: 01 00 00 00 value: 02 21 00 00
Found 2 elements
root@ubuntu21:~/test# bpftool map dump id 1
key: 00 00 00 00 value: a8 22 00 00
key: 01 00 00 00 value: 2a 22 00 00
Found 2 elements
The increasing statistical value proves that our tc program is working. Done!