1. DPDK principle
Network equipment (routers, switches, media gateways, SBCs, PS gateways, etc.) need to send and receive a large number of messages in an instant. Therefore, on traditional network equipment, you can often see special NP (Network Process) processors, and some Use FPGA, some use ASIC. These dedicated devices efficiently forward messages through built-in hardware circuits (or hardware circuits formed by programming), and CPU intervention is only required when in-depth processing of messages is required.
However, in public cloud, NFV and other application scenarios, the infrastructure uses the CPU as the computing core and often does not have a dedicated NP processor. The operating system is also based on general Linux. The processing path of network data packets is shown in the following figure:
In a virtualized environment, the path will be longer:
Due to the switching between kernel mode and user mode in packet processing tasks and multiple memory copies, the system consumption becomes larger, and there is a big processing bottleneck in the system with CPU as the core. In order to improve the performance of data packet processing in the general-purpose server (COTS), Intel has introduced DPDK technology for IA (Intel Architecture) systems.
DPDK is the abbreviation of Data Plane Development Kit. Simply put, the DPDK application runs in the User Space of the operating system, and uses the data plane library provided by itself to send and receive packets, bypassing the Linux kernel mode protocol stack to improve message processing efficiency.
DPDK is a collection of lib libraries and toolkits. The simplest architecture description is shown in the figure below:
The blue part of the above figure is the main component of DPDK (for a more comprehensive and authoritative DPDK architecture, please refer to Intel's official website), briefly explain:
-
PMD: Pool Mode Driver, polling mode driver, through non-interruption, and zero copy mechanism of data frames in and out of the application buffer memory, improve the efficiency of sending/receiving data frames
-
Flow Classification: Flow Classification, which provides optimized search algorithms for N-tuple matching and LPM (longest prefix matching)
-
Ring Queue: Ring Queue, which provides a lock-free mechanism for the entry and exit queues of single or multiple packet producers and single packet consumers, effectively reducing system overhead
-
MBUF buffer management: allocate memory to create buffers, and encapsulate actual data frames by creating MBUF objects for use by application programs
-
EAL: Environment Abstract Layer, environment abstraction (adaptation) layer, PMD initialization, CPU core and DPDK thread configuration/binding, setting HugePage memory and other system initialization
In this way, there may be a little abstraction, and then summarize the core ideas of DPDK:
-
PMD driver in user mode, removes interrupts, avoids kernel mode and user mode memory copy, reduces system overhead, and improves I/O throughput
-
User mode has an advantage, once the program crashes, it will not lead to the end of the kernel, which brings higher robustness
-
HugePage, through larger memory pages (such as 1G memory pages), reduce TLB (Translation Lookaside Buffer, fast table) Miss, Miss has a great impact on message forwarding performance
-
Multi-threads are created on multi-core devices, and each thread is bound to an independent physical core, reducing the overhead of thread scheduling. At the same time, each thread corresponds to an independent lock-free queue, also in order to reduce system overhead
-
Vector instruction set, improve CPU pipeline efficiency, reduce memory waiting overhead
The following figure briefly describes the multi-queue and multi-thread mechanism of DPDK:
DPDK assigns the network card receiving queue to a certain CPU core, and the messages received by this queue are handed over to the DPDK thread on the core for processing. There are two ways to send data packets to the receiving queue:
-
RSS (Receive Side Scaling, receiver extension) mechanism: based on keywords, such as UDP quadruple <srcIP><dstIP><srcPort><dstPort> hash
-
Flow Director mechanism: It can be set to accurately match certain information of the data packet and assign it to the specified queue and CPU core
When the network data packet (frame) is received by the network card, the DPDK network card driver stores it in an efficient buffer, and creates an MBUF object in the MBUF cache to connect to the actual network packet. The analysis and processing of the network packet will be based on the MBUF , The actual network packets in the buffer will be accessed only when necessary
The above is the basic knowledge of DPDK, about how to use DPDK in applications, and how the system should be optimized for the mature and optimized way of message sending and receiving. Later, learn and practice while recording.
Two, DPDK learning route summary DPDK learning route and click on the video to explain the learning materials acquisition
1. dpdk PCI principle and testpmd/l3fwd/skeletion
2.kni data flow
3.dpdk implements dns
4. Implementation of dpdk high-performance gateway
5. Acceleration of paravirtualized virtio/vhost
