First, what is a performance test? Here are two scenarios:
1. 1000W WeChat users go to Moments at the same time 2. 1000W
Taobao users go to buy goods on Taobao at the same time
Of course, the above two examples are very likely to have happened. And although we all participated in the above test, because we are the client, it is not really clear how much stress is put on the server or what is happening to the server.
ok, above we probably know the difference between functional testing and performance testing. The purpose of functional testing is to ensure that there is no problem when a single user requests; the purpose of performance testing is to ensure that there are no problems when many users (system supported) request at the same time. And our performance test is a targeted test by analyzing the problems that may occur when multiple users request at the same time in the real situation (this definition is purely personal YY, if there is any similarity, it is not an honor).
It seems to be very simple, so let's try to answer the following questions:
1. When there are 1000W WeChat users sending Moments at the same time, where is the pressure on the server? How much pressure is there in each place? Where is the bottleneck?
2. What is the real user scenario? How to ensure that the performance test results of our laboratory can meet the real user scenarios?
3. If the number of users later goes from 1000W to 2000W, how will our pressure point change?
4. After the pressure analysis is done, how do we cover these pressure points, such as: how to construct 1000W users to request data?
5. Can we obtain an optimal result of our server based on continuous tuning (eg, adjusting user behavior) during the performance testing process?
......
Seeing these questions, I believe that no classmates think it is very simple. We will share what knowledge we need to learn to answer the above questions later.
First, we divide performance testing into performance index testing and performance stress testing.
Performance index testing refers to simulating real concurrent user operations to determine whether the product can meet the needs of users; for example, WeChat needs to support the access of 400 million users at the same time as a real user demand, but it is impossible for us internally To access 400 million users, it can only be accessed by simulating 400 million real users. If we achieve this goal through simulation, it means that this requirement is met.
Performance stress testing refers to verifying whether the functions of the product can still operate normally and whether the system itself is normal under a certain load situation (resource occupation, number of connected users, etc.). For example, when the cpu and memory usage reaches more than 80%, the system crashes.
Other friends may say that there are other performance tests such as stability testing, load testing, etc., which will not be discussed here, because the testing methods are similar and the purposes are similar. We will focus on sharing how to perform performance index testing and performance testing. pressure test.
1. Performance index test:
First of all, it is necessary to know whether the corresponding requirements of the performance index are reasonable and clear enough. Brother Stone has always emphasized that before doing something, we must figure out what the meaning of doing it is, especially for work. For example: the requirement states that we want to support 10,000 users to pay online at the same time. Then we will have the following questions:
1. How did 10,000 users come from, and what kind of model was used to infer them, is it reasonable?
2. Is the payment method determined, and whether different payment methods have an impact?
3. Is the payment process (generally not a non-atomic operation) consistent?
4. Is it required that all users choose to pay successfully, or a certain number of users are required to choose to fail?
...
when we truly understand the requirements, we can better think about our performance index testing process from the user's point of view, and can also make our user index testing process more in line with real user usage scenarios .
After the analysis of the requirements is completed, we can start to design the corresponding performance indicator use cases according to the requirements. At the same time, we need to analyze whether there are test methods that can cover the specific requirements (it is recommended to understand the entire business logic first). For example: How do we simulate the concurrent access of 1000W users? At this time, we may need to develop our own testing tools, or we need to develop and provide us with corresponding interfaces.
After researching and determining the test method, we start to carry out the coverage test according to the specific test method. Of course, the testing process is generally not smooth sailing. This also requires us to have a certain analytical ability (the previous business learning and understanding are the foundation), for example: when we find that the number of concurrent users reaches 10W, it has been unable to get up. Can we investigate possible causes ourselves? Tool itself problems, network problems, interface problems, system load problems, etc. The ability to troubleshoot and locate problems should be one of the basic abilities of a performance test engineer, which is also difficult to master.
According to our established test plan, after completing the performance index test, we need to analyze and organize the entire performance test process and results, so that we can submit a high-quality performance test report. In addition, we may have to test this performance index table many times. At this time, we can analyze whether it can be tested in an automated way. After all, you should not want to do the same job twice, right?
2. Performance stress testing
Performance stress testing, like functional testing, is mainly a process of evaluating quality by finding bugs. So here are two questions:
1. What kind of problems are performance problems (that is, the difference between functional problems and performance problems)
2. How to ensure the quality of performance (that is, all performance problems can be found)
Generally speaking: the author believes that when A problem that occurs when the system is under a certain load is a performance problem. For example: Take QQ as an example, there may be no problem with the server when one QQ user logs in, but there may be a problem with the server when 10,000 QQ users log in at the same time and send data (because the occupied resources are very different), other if Logic problems that can be found by testing functions are not performance problems.
So, how to ensure the quality of performance? Here, refer to a method of ensuring the quality of functions, that is, to ensure the performance quality by covering the performance business logic. The details are as follows:
1. Draw the business logic in detail.
2. Extract performance points according to the business logic. One principle for extracting performance points is to determine whether a single connection or multiple connections will affect the logic. Extracted, such as resource application and release, multi-threaded work, etc.
3. Completing performance use cases according to performance points, be sure to ensure that these performance testing methods can achieve the purpose of testing
4. Analyze whether some test methods or checkpoints can be used
5. Learn the corresponding test tools, and familiarity with the tools can effectively improve your test efficiency
6. Like functional testing, more analysis must be done during the testing process. It is best to analyze each performance bug, and then see if there are any places that may not have been considered, and then perform supplementary testing and improve use cases (and Analyze why it was not considered at the time) to form an empirical document.
7. Strengthen the study of the system under test, because the performance stress test itself is a whole, and the analysis can be more accurate only if you have a sufficient understanding of the system under test. In-depth study of the system is more necessary.
8. Try to troubleshoot and locate the problem. Again, Brother Shishi really thinks that the ability to troubleshoot and locate the problem is a very important ability for a performance test engineer, because for a performance test engineer, It is necessary to know where the bottleneck of the entire performance is and how to cover it, rather than simply using loadrunner.