As we all know, Ethernet has penetrated into our lives everywhere. Enterprises, campuses, big data centers and families cannot do without the network, otherwise our life will be seriously affected.
The interface rate of Ethernet is also developing rapidly: 10M, 100M, GE, 10GE, 40GE, 100GE, and now gradually mature 2.5GE, 5GE, 25GE, 50GE and even 400GE, which is basically a development trend of 10-fold increase every 10 years.
Many data centers and operators are also preparing to expand their networks from 100GE to 400GE to support fifth-generation wireless technology (5G), artificial intelligence (AI), virtual reality (VR), Internet of Things (IoT), self-driving cars The bandwidth and response time required by emerging technologies such as
However, any new network interface rate, network equipment research and development to form the final product, and network new technology from research to implementation, every stage is inseparable from test verification, and has brought great challenges to the test. At the same time, the development of high-performance, stable and high-speed network testers cannot keep up with the research and development needs of network equipment.
Therefore, the development of high-speed and stable network testers is urgent, especially the domestic network tester products that have mastered the core technology.
The test scenarios of large-scale data center switches or core routers conducted by domestic telecom operators place extremely high requirements on the tester.
- · Stability
- It has the ability of long-term stable streaming, statistics and protocol simulation, such as 7x24 hours of long-term stable operation;
- · Repeatability
- The same physical environment and network conditions, multiple test results must be consistent;
- · Accuracy
- The test results must accurately reflect the real indicators of the device or system under test, such as throughput indicators, accuracy of delay and jitter, accuracy of traffic scheduling, and accuracy of traffic statistics;
- · High performance
- Support all packet lengths (such as 64-16000 bytes or IMIX mixed packet length) wire-speed streaming and statistical capabilities, ultra-high routing and switching protocol simulation capabilities (such as BGP/OSPF/ISIS/PPPOE/IPOE/ EVPN, etc.); multi-port (such as hundreds of 100GE/10GE) multi-service (such as IPV4/IPV6/MPLS/Multicast) traffic scenario simulation capabilities;
- · Standard
- With international test standards RFC-2544, RFC-2889, RFC3511, RFC3918, etc.;
- · Rich interface types
- Support 1GE/2.5GE/5GE/10GE/
- 25GE/40GE/50GE/100GE/400GE and other interface types support multi-chassis cascading to build large-scale test scenarios.
Currently, network testers currently on the market mainly have two architectures.
Tester based on x86+DPDK+ network card
The programming of X86 is relatively easy, the debugging methods are more abundant, and the cost has certain advantages. It is a good choice for functional testing with low requirements.
Tester based on FPGA+x86 hybrid
A test system with a combination of software and hardware such as FPGA+x86 is suitable for test scenarios with high performance, full coverage, large-scale test scenarios, and complex test services.
The hybrid architecture of FPGA+x86, on the one hand, utilizes the increasingly powerful data parallelism of the FPGA at the hardware level, and on the other hand combines the processing flexibility of the CPU at the control level. At the same time, since FPGA and CPU are essentially programmable According to the needs of business processing, the system can flexibly move the business division boundary between FPGA hardware and CPU software to realize the overall optimization of the entire business process.
According to the complex test scenario requirements of telecom operators mentioned above, we conduct a comprehensive comparative analysis of the testers of the two architectures as follows:
64-16000 byte packet length wire-speed streaming and statistical capabilities
x86+DPDK+ network card: Taking the 100G network shown in the figure as an example, in the case of a 64-byte packet length, about 150M data packets will be sent and received per second, which cannot be handled by the current CPU computing and storage access capabilities. Testers based on FPGA fabrics do not have this limitation. According to the latest data publicly released on the DPDK official website (2019·10·9) [data source: DPDK official website www·dpdk·org], using the system configuration in Figure 1, it can be clearly seen that it cannot be achieved in the case of small bytes such as 64, 128, 256 100G wire-speed sending and receiving packets;
FPGA+x86: All bytes can achieve wire-speed packet sending and statistics.
Figure 1. X86+DPDK test system configuration
The achievable line-speed performance of small-byte packet lengths is shown in the figure below (the figure below - the test performance of each packet length of the X86 system).
Delay and Jitter Accuracy
The accuracy of delay test is a very important index in network test.
x86+DPDK+ network card: The X86 system is a general-purpose computing system, and its own reference clock accuracy is not high, and the scheduling error of the OS is at least in the order of us; if the NIC of the x86 system does not support inserting time stamps at the physical layer, The network delay needs to be handled by the software system, which will bring further errors; therefore, the delay accurate to the order of 10 ns usually required by the network test is difficult to achieve on the computing-oriented general-purpose x86 platform;
FPGA+x86: On the FPGA platform, a clock of up to 400M is generated through a crystal oscillator with a precision of 0.1-0.001PPM, and the precision of the delay can be controlled at the level of 2.5ns.
storage system flexibility
x86+DPDK+ network card: The x86 system is oriented to general computing. The current mainstream memory system is DDR4 memory, which has a large bandwidth but also a large access delay. Depending on the read and write access mode, there may be delay jitter;
FPGA+x86: FPGA memory can combine various memory technologies such as on-chip RAM (which can realize the cache function) + DDR+QDR+RLDRAM according to needs, and optimize bandwidth requirements and delay access requirements.
Ability to Accelerate Protocols
x86+DPDK+network card: no additional resources to implement protocol acceleration functions such as TCP offloading;
FPGA+x86: FPGA is a hardware programmable system. According to the amount of hardware resources and the needs of business processing, it can flexibly divide the interface boundaries with the x86 system in terms of protocol processing, realize protocol acceleration functions such as TCP offloading, and integrate protocol processing The computing-intensive stateless tasks are parallelized at the hardware level, which can greatly enhance the processing capacity of the entire system.
Accuracy of layer 2-3 traffic scheduling
x86+DPDK+ network card: The x86 system cannot realize small-byte packet length line-speed transmission on high-speed ports, let alone accurate traffic scheduling;
FPGA+x86: In the face of switches and routers with increasingly complex test traffic and larger scale, the FPGA system architecture supports the generation of tens of thousands of streams (such as the typical 64K streams in high-end testers), and It can precisely control the bandwidth ratio between each flow and the sending scheduling mode, even to 5 decimal places.
Statistical real-time performance and accuracy
x86+DPDK+ network card: The CPU test function software implementation is essentially a serial instruction set. With the realization of new technologies such as multi-core hyperthreading, partial parallelism can be realized at the instruction level, but for some statistical data, such as real-time The number of sending and receiving frames per second, etc., is defined by at least two parameters (a certain time interval delta and the number of sending and receiving packets within this interval), if the reading of these two parameters is implemented on a CPU core, The serial nature of the instruction will inevitably bring about a large error; if the reading of these two parameters is implemented on two cores, it is difficult for the current CPU technology to achieve ns-level synchronization between the cores, which also brings statistics imprecision of value;
FPGA+x86: Inside the FPGA, through the hardware programming technology, the statistical value snapshot function can be easily realized, and the reading of the above two parameters is strictly guaranteed to be accurately corresponding.
System scalability
x86+DPDK+network card: For a large-scale system under test, whether it is a x86 software implementation or a hybrid system such as FPGA+X86, a single machine cannot complete the test task. System cascading and synchronization at the 10ns level are inevitable options. The x86 system is oriented to general computing, and can realize multi-machine synchronization by running the NTP protocol, but the synchronization accuracy of NTP cannot meet the requirements of the delay test business;
FPGA+x86: In the hybrid system of FPGA+x86, high-precision synchronization technologies such as local cable cascading/GPS/1588v2 can be realized through FPGA to ensure the accuracy of time testing.
In addition, in the implementation of the FPGA+x86 hybrid system, the 2-3 layers of traffic processing are implemented in the FPGA without going through the CPU's protocol stack or upper-layer applications. The CPU only needs to implement lightweight configuration delivery and interface presentation, etc. The calculation avoids the natural defects of the CPU in line-speed sending and receiving flow processing; on the X86 side, DPDK technology can also be flexibly deployed, and the accelerated pure protocol processing part is realized by the X86 system, combining the advantages of FPGA and x86 to realize Efficient business processing.
Obviously, adopting FPGA+x86 hybrid system is the best choice to build a high-performance network tester.
- In recent years, foreign Ethernet testing technology has developed rapidly, and new products emerge in an endless stream. With years of technology accumulation for high-speed and high-performance testing software and hardware platforms, Spirent and Keysight, two companies in the United States, have long occupied the global leading position in the field of Ethernet testing. The industry is at the forefront of the world.
- Domestic research on Ethernet test technology began in the early 21st century. After more than ten years of hard work, the independent design and development capabilities of related test products have also made great progress.
The network tester adopts the form of a chassis structure + pluggable boards. The overall hardware solution adopts a distributed computing architecture, which is logically divided into a data plane and a management plane, and the test module and the main control module are separated on the physical unit. Design, using the FPGA+x86 hybrid system architecture to achieve the goal of a high-performance network tester.
