This article mainly introduces when to carry out performance testing, how to carry out performance testing, what preparations need to be made for carrying out performance testing, receiving a new project, ideas for carrying out performance testing, etc.
1. Three questions about performance testing
1. When should performance testing be performed?
The work of performance testing is based on the fact that the system functions are complete or tend to be complete , and it does not make much sense if the functions are not complete enough. Because the improvement of later functions will affect the performance of the system, entering the performance test too early will result in inaccurate test results and waste of test resources. Therefore, performance testing is firstly based on functional testing, and performance testing must be carried out only after understanding its functional requirements.
2. How to conduct performance test?
For a system under test, we need 3to analyze it in parts:
- Entrance: how to send the request, how much pressure should be applied by the pressurer, and what method should be used to exert pressure;
- System under test: how the system responds to a single request, what is the system business process, system network element nodes, data flow direction, etc., whether there are overall performance requirements, which indicators need to be investigated, and how to monitor;
- Export: What are the received data, how to obtain and compare;
Does it feel like a functional test? Yes, it is to first analyze the functional process of a single user and the structure diagram of the data flow of the system (including the data flow in the background), and then consider a large number of user operations.
3. What are the steps to carry out performance testing?
The general system performance testing steps are as follows:
1. Confirm the test objectives;
2. Analyze the business requirements of the system under test;
3. Analyze the system structure of the system under test;
4. Analyze the performance test points of the system under test;
5. Design test plan, test plan and test case;
6. Select test tool;
7. Test development;
8. Test execution;
9. Analysis of test results;
10. Test tuning, test verification, test analysis;
11. Output test report;
2. Preparation before the performance test
The more adequate the test preparation, the smoother the later test execution. The general test preparation is as follows:
1. Confirm the test objectives;
2. Analyze the business of the system under test;
3. Analyze the structure of the system under test;
4. Analyze the nodes that may have performance bottlenecks in the system under test;
5. Design test plan, test plan and test plan;
The following five steps are analyzed in detail:
1. Confirm the test target
Any task must first confirm what the goal of the task is. If the goal is not known, the result of any effort may not be the final desired result. The same is true for performance testing, first of all need to establish a clear goal. Whether it is to randomly test the current performance of the system, or to optimize the system, or to check whether the performance of the system meets the requirements, etc., these are all goals before performance testing. Subsequent analysis, scheme and use case design, test execution monitoring, and final test analysis and reporting are all based on this goal. Therefore, the first task is to confirm the target requirements of the test, and what kind of test purpose and goals need to be achieved.
If there are some test tasks without clear goals or requirements, it does not mean that they have no goals and goals, which requires us to communicate and analyze. Timely reach an agreement with the project team on the purpose requirements, analyze the requirements, analyze the system, and finally clarify the purpose requirements of the project or system testing tasks.
2. Analyze the business of the system under test
A friend once had an interview, and an interviewer gave him such a question: "There is a website, I only know that its total visits are 3 million a day, how to test its performance?"
How do you think about the design?
Guess the interviewer wants the interviewer to answer, the application of basic testing principles such as normal distribution and 28th principle. My friend did not answer anything related to the normal distribution and the 28th principle. At that time, the interviewer seemed to smile at his answer with "contempt". Maybe he felt that he didn't even know the basic normal distribution and the 28th principle, and he still engaged in performance testing? . In fact, performance testing is not as simple as imagined. It is not just the application of a simple principle. If it is so easy, then anyone can do it.
The basis of performance testing is that the business functions of the system tend to be basically stable. The first task is to develop the performance when the system meets the business function requirements. Therefore, we must analyze the business of the system. Whether it is an ordinary website or a more professional system, it has business function requirements, and all performance tests must be based on these functions. Performance testing without business functions is meaningless. The primary task of performance testing is to analyze the business functions of the system and analyze the performance limitations of the system business, that is, business requirements.
So, how to analyze the business requirements of the system?
- If there is a user requirement specification, the first task is to read, understand and analyze the user requirement specification;
- If there is no user requirements specification, then it is necessary to analyze the system functions and extract the business requirements of the system. If possible, it is best for someone familiar with the project team to tell the story.
- In the end, no matter which one is the best way is to draw the business process of the system or the functional structure diagram of the system according to your own understanding, and get it to the project team for confirmation. Be sure to confirm and reach a consensus with the entire project team.
Some people may ask, what should we do when our project team has no identifiable business requirements?
First of all, we still need to start with analysis. If you don’t analyze it, you won’t know the functional data flow direction of the system, the requested data composition, the network element structure of the system, and which node the system may have a bottleneck in. How can you optimize it?
Of course, when faced with a new field of knowledge, we may need to accumulate more experience and conduct more analysis; we may need to combine practice, actually run the system or perform tests many times, and continuously optimize and improve our knowledge in the test. The analysis process, analysis results, test plan, test development and even test execution.
Analyzing the business of the system under test is sometimes not done overnight. We need to conduct repeated analysis, confirmation, reanalysis, and reconfirmation until the system is understood, and you may even need to analyze it again in the final stage of test execution. and confirm, then reschedule the test.
3. Analyze the structure of the system under test
The structure of the system is as important as the business of the system. If you do not know the network element structure of the system, you may not be able to monitor it, and you will not be able to know which node the bottleneck is, and you will not be able to optimize it.
The best way to analyze the system structure is for the project team to provide the system deployment and composition diagram; if the project team cannot provide or does not have a project team, then it is necessary to use packet capture tools such as TCPDUMP to analyze the data flow.
Use of TCPDUMP:
tcpdump tcp -i eth1 -t -s 0 -c 100 and dst port ! 22 and src net 192.168.1.0/24 -w ./target.cap
(1)tcp: ip icmp arp rarp 和 tcp、udp、icmp这些选项等都要放到第一个参数的位置，用来过滤数据报的类型；
(2)-i eth1 : 只抓经过接口eth1的包；
(3)-t : 不显示时间戳；
(4)-s 0 : 抓取数据包时默认抓取长度为68字节。加上-S 0 后可以抓到完整的数据包；
(5)-c 100 : 只抓取100个数据包；
(6)dst port ! 22 : 不抓取目标端口是22的数据包；
(7)src net 192.168.1.0/24 : 数据包的源网络地址为192.168.1.0/24；
(8)-w ./target.cap : 保存成cap文件，方便用ethereal(即wireshark)分析；
Analyze where the flow is going from the first node, and determine the nodes of the second layer; then analyze the nodes of the third layer from each node of the second layer, analyze layer by layer, and improve all structural levels and nodes of the data flow direction of the system; and then figure out The application program or process queue deployed by each node; the application program or process queue of each node is tested and monitored; finally, which applications or process queues need to be optimized can be obtained.
In addition to understanding the node composition of the system, it is also necessary to understand the communication protocol and data format between each node, and then test tool selection, test data preparation, and test script development need to be based on this. The basis of all this is to analyze and understand all the nodes of the system, that is, to analyze and understand the structure of the system.
4. Analyze the possible performance bottlenecks of the system
Analyzing the business requirements of the system and the structural composition of the system, while predicting the possible performance bottlenecks of the system, is one of the goals of the analysis; after obtaining the predicted performance bottlenecks, we need to pay more attention to these nodes when monitoring later.
Of course, there are some common nodes that may be system bottlenecks that we need to pay attention to:
1. Log in. Generally, system login requires multiple verifications, and data interaction may be frequent;
2. There will be a certain amount of concurrent demand at this time when placing an order, grabbing an order, and grabbing a red envelope;
3. Big data query, statistics and report analysis will put pressure on the system;
4. Videos, animations, etc. will put pressure on the network;
5. System function nodes with relatively concentrated messages will put pressure on the system;
6. Some special business requirements will put pressure on the system;
Common bottlenecks:
1. The bottleneck of the database is generally caused by the process blocking caused by the high disk IOPS;
2. Too many system processes generally consume system memory space;
3. For message queue and cache service, you need to check the disk IOPS after enabling persistence, and you need to check the memory usage if you don’t enable persistence;
4. Frequent pipeline opening and destruction will lead to high CPU usage;
5. Some programs cannot utilize multiple CPUs structurally;
In the process of analyzing the business and system structure, we need to consider whether this business point or structure point will have a large amount of data access, whether there will be pressure, and whether our design will cause a performance bottleneck.
5. Design of test plan and test cases
The design of the test plan and the formation of the final test summary document are actually the summary of all analysis work.
The process of writing a test plan is a summary of the process of clarifying the purpose of testing, analyzing business requirements, system structure, and evaluating test methods, test arrangements, and test risks. And these all come from the analysis before the test is executed, and sometimes you may need to make some analysis and adjustments during the test.
The test plan includes all aspects of analysis and organization. A good test plan includes: test purpose, test goal, test content (may include business performance, reliability, stability, etc.), business requirement goals, system business composition , system node composition, test method flow, indicator requirements and nodes to be monitored, etc.
Test cases are generally included in the test plan. Test cases are actually common business operation processes. Test tools or other test methods are used to simulate business operations with large data volumes, and monitor each node of the system to obtain monitoring data. The comparison between the expected monitoring data and the actual monitoring data, meeting the requirements is the expected requirement, and the actual comparison result is the test result.
6. Preparation before the test
Environment construction : The workload of the test environment construction step is not large. If necessary, please ask the development assistance to complete the configuration.
Scenario modeling : Consider which scenarios may have performance bottlenecks, and set up corresponding test scripts and test logic to simulate the production environment as much as possible (you must be familiar with the system business, because the generation of requirements is user + scenario).
Test data preparation : There are two commonly used methods for test data preparation: copy a copy of the production data, and pre-embed the data in the development script (but no matter which one, we must pay attention to data isolation to prevent data pollution).
Test script development : First, you need to obtain development interface documents and database table design documents from the development. Then, debug the interface through tools or write test scripts to ensure that the interface can be called successfully.
3. Formal launch of performance testing
1. Execute the test script
After ensuring that the interface can be called successfully, perform a single-interface benchmark test first, that is, perform a stress test on an interface, keep increasing the pressure until the response time reaches or exceeds the target, and observe the current 并发数sum TPS. With the same number of concurrency, execute it several times to get an average or stable value (that is, the relatively stable value of the TPS and TRT curves), and record it.
The purpose of recording is to obtain the maximum TPS of a single interface and the response time changes under different concurrency conditions through intuitive data changes.
"80% of performance bottlenecks can be obtained by analyzing the numerical changes of TPS and TRT." Although a bit one-sided, it is also a method. For example, according to the numerical changes recorded in the above figure, it is obvious that the performance of the interface for receiving coupons is extremely poor. At this time, you can inform the development, and check the reason by checking the log, checking the code, and SQL statements. Of course, if you are capable enough, you can do it yourself.
2. Monitoring and debugging
The so-called monitoring and debugging is a process of continuous adjustment and repetition, which needs to be judged according to the purpose of performance testing.
Jmeter itself uses the listener component to provide a certain monitoring value reporting component, but it is an open source tool after all, and its components are not powerful enough. You can monitor it by downloading an enhanced function plug-in that supports jmeter.
Jmeter插件下载地址：https://jmeter-plugins.org/
After downloading, you can decompress it, put the plugins-manager.jar into the jmeter installation directory lib/ext, and then restart Jmeter. You can click the button circled in the figure below to check whether the installation is successful:
Whether it is server resource usage, test data report generation, or even monitoring of TPS, TRT, etc., the plug-in provides component support, and the specific usage method can be explored by yourself.
3. Analysis and tuning
In addition to obtaining performance indicators, performance testing is more to discover performance bottlenecks and performance problems, and then analyze and optimize performance problems and bottlenecks. In today's era of rapid Internet development, the performance tuning model can be summarized as follows shown.
Performance tuning is to continuously collect the performance indicators in the system and the resource consumption of each layer in the system model, find performance bottlenecks and performance problems, and then analyze and diagnose the bottlenecks and problems to determine the performance tuning plan, and finally carry out performance pressure testing Verify whether the tuning scheme is valid. If it is invalid, continue to repeat this process for performance analysis until the tuning scheme is valid and the bottleneck and problem are resolved. This process is generally very long, because in many cases, performance tuning solutions are often not effective at one time or can solve all bottlenecks and problems at one time, or solve the current bottlenecks and problems, but new performance problems may appear when performance pressure testing is continued. bottlenecks and problems.
4. Output test report
According to the above steps, get the test results, analyze the bottlenecks in the system, then use various methods to propose solutions or optimization suggestions, and finally make a complete summary of this performance test, so that a performance test is completed. In the whole process, the time-consuming is generally in the test data preparation and test execution and monitoring and tuning stages.
Performance testing has a long way to go, and if you want to do a good job in performance testing, you still need to search up and down.
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey, and I hope it can help you! Partners can click the small card below to receive