In the rapid iterative development model of the Internet in recent years, automated interface testing has become an important means of server quality assurance. However, as the complexity of the system gradually increases, any change in a line of code or a configuration file may cause the entire system to collapse or cause huge losses. It is difficult to fully cover special business scenarios by relying solely on precipitated automated test scripts or manual testing. In this context, automated testing based on traffic playback technology emerged.
Traffic playback testing technology uses traffic recording and playback to distribute images of actual traffic to test servers to verify whether new code modifications have an impact on the original system and whether new functions are available. This article will mainly introduce the application of traffic playback technology in testing and currently commonly used traffic playback related tools. I hope it will be helpful to students who have not been exposed to traffic playback technology or are interested in traffic playback technology.
Application in testing
Traffic playback technology is widely used in testing. Three common application methods are introduced below:
1. Diff test
Comparing the functions of new and old codes based on traffic playback technology, referred to as Diff testing, is one of the more widespread applications of traffic playback technology. The basic idea is to obtain the traffic online, play it back in different code environments, and compare the results returned by the interface.
Application scenarios:
• System reconstruction to ensure that the functional logic of old and new versions is consistent
• New function testing to ensure that historical logic is not affected by new code
Application benefits:
• Significantly improve regression testing efficiency
• Quickly discover problems, especially unpredictable problems
2. Performance & exception testing
In performance testing, using real traffic instead of test requests constructed by testers is closer to the real online situation, and the test data obtained is more credible. At the same time, it is easier to detect serious problems such as system crashes by testing services that are not online based on traffic playback technology.
Application scenarios:
• Special performance verification to test whether the performance improvement is in line with expectations
• Performance stuck points before going online to ensure that new functions do not affect performance
• Abnormal traffic verification to discover serious problems such as system crashes
• Online full-link stress testing to ensure stress testing real traffic
Application benefits:
• Restore real online traffic, making performance results more authentic and credible
• Quickly discover serious problems that cause system crashes
3. Functional testing & troubleshooting
In daily functional testing or online problem troubleshooting, traffic playback technology can also play a big role. For example, obtaining traffic from online, filtering specific traffic, playing it back in the test environment, observing the return of the test environment or troubleshooting logs question
Application scenarios:
• Functional testing, finding real online requests and converting them into test requests
• Problem troubleshooting, supporting diverse business test scenarios
Comparison of commonly used tools
There are currently many open source traffic playback tools. How do you choose the one that suits your business? This article roughly divides traffic playback tools into the following three categories based on different recording methods, namely application process-based recording, middleware-based recording, and network protocol stack-based recording. The characteristics and scope of application of each tool are separately introduced for readers' reference.
1. Based on application process
①.Sharingan
Introduction: Sharingan is a traffic recording and playback tool based on golang. It can record real online request traffic for playback testing.
Principle: Modify the golang source code and add hooks to intercept and mirror traffic in the Socket Connect, Accept, Send, Receive, Close and other stages.
Advantages:
• Full-featured, supports traffic filtering/ratio control
• Supports http, mysql, redis, thrift and other protocols
• Traffic playback comes with diff/mock function
Disadvantages:
• Need to recompile the program according to the customized version of golang
• Only suitable for applications developed in go language
2. Based on middleware
①.ngx_http_mirror_module
Introduction: nginx 1.13.4 and subsequent versions have a built-in ngx_http_mirror_module module, which provides the function of traffic mirroring (replication).
Principle: When receiving traffic inside nginx, copy the same traffic to the mirror environment
Advantages:
• Supports traffic amplification, simple configuration, and can be forwarded to multiple mirrors
• No need to record traffic, real-time forwarding
• Origin site requests are returned directly to the original route; the return of mirror requests is not processed
Disadvantages:
•Severely affects online Nginx performance when traffic is large
•Only supports HTTP protocol
②. Introduction to JVM-SANDBOX-Repeater
: JVM-SANDBOX-Repeater is an important module under the JVM-Sandbox ecosystem. It has all the features of JVM-Sandbox. The plug-in design facilitates rapid adaptation to various middlewares and encapsulates request recording. /Playback basic protocol also provides a variety of rich APIs that are universal and extensible.
Principle: JVM-based non-intrusive runtime AOP solution
Advantages:
• Traffic playback is only a basic function and can be extended to other capabilities of the sandbox
• Universal recording/playback capabilities, multi-protocol support
• Internal call chain can be traced, convenient for debugging
Disadvantages:
• Only applicable to JAVA applications
• Difficulty to get started, familiar with JAVA/JVM
• Still has performance impact on online JVM
3. Based on network protocol stack
①.Goreplay
Introduction: GoReplay is an open source tool written in golang for capturing real-time HTTP traffic and playing it back into a test environment
Principle: Use the gopacket library to capture packets based on libpacp, and use the coroutine feature of go to control the playback rate.
Advantages:
• Simple installation and deployment, no intrusion into business applications, only occupying machine bandwidth
• Supports filtering and modifying traffic, and can forward traffic in real time
• Supports playback rate control, which can be used for performance testing
Disadvantages:
• Currently only supports HTTP1.1 and below protocols
• Unable to parse binary data interaction protocols, such as Protobuf
②.TcpCopy
Introduction: TCPCopy is an open source traffic playback tool that supports real-time and offline playback of multiple types of traffic.
Principle: Copy the online TCP type request packet, modify the TCP/IP header information, and send it to the test server to achieve the purpose of deceiving the TCP program of the test server.
Advantages:
• No intrusion into the business, because of the existence of intercept, real-time traffic diversion has less impact on the line • All
protocols of the basic tcp protocol stack can be supported, and it is almost the traffic playback tool that supports the most protocols at present
• Sending based on the IP layer, the traffic is more Real
•Supports distributed architecture and can be used as a stress testing tool
Disadvantages:
• Difficult to deploy, tcpcopy and intercept need to be deployed at the same time, and the parameters are complex
• Recording and playback are based on underlying data packets, making it difficult to filter and customize traffic
Possible problems
In the process of applying traffic playback technology to testing, you will encounter many problems and challenges. The specific problems will depend on the business background and technology. Here are some more general problems and some solutions for readers' reference:
1: How to ensure the consistency between test environment data and online?
Reference plan: You can consider timing synchronization of online data to the test database, etc.
2: The difference between read/write interfaces. For the read interface, traffic playback has no side effects and will not generate dirty data; for the write interface, how to deal with it?
Reference plan: For the read interface, use traffic playback for comparison. For the write interface, you can use automated test cases to ensure
3: The interface has external dependencies, and the consistency returned by the external interface/service cannot be guaranteed?
Reference plan: When you need to call external interfaces/services, use mock methods and other external dependencies to isolate them.
4: The program is random and has dynamically changing fields (such as timestamps, etc.). How to compare Diff results?
Reference plan: (1) Add program switches to avoid function randomness problems (2) Filter specified specific fields when comparing results, and do not compare them (3) Perform noise reduction processing to automatically identify fields that do not need to be compared.
5: Desensitize core data, record sensitive data, and be sure to encrypt and desensitize the data
Summarize
Traffic playback technology has many successful implementation cases in the industry. We hope that through the introduction of this article, more readers can apply traffic playback technology in actual testing work to achieve both improvement in testing efficiency and service quality!
Finally: The complete software testing video tutorial below has been compiled and uploaded. Friends who need it can get it by themselves [guaranteed 100% free]
Software Testing Interview Document
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
