The performance test of traditional enterprise software products and Internet products is basically the same in principle and test method, the biggest difference is in the order of the number of concurrent, and
the performance test of Internet software products also needs to be carried out directly in the production environment. Link pressure test. The full-link test is actually based on the performance test of traditional enterprise software products
and has been expanded.
Performance benchmark test
It is the type of test that must be completed each time the product is released.
The performance benchmark test will be given a fixed hardware environment and deployment architecture (such as a dedicated server, a fixed dedicated network environment, a fixed size cluster size, the same system configuration, the same
database background data, etc.), by performing a fixed performance test The scenario gets the performance test report of the system, and then compares it with the indicator at the time of the previous version release. If the indicator is found to be deteriorating
, further investigation is needed.
The typical deterioration trend is mainly manifested in the following aspects:
The response time of the same transaction is slower.
For example, in the previous version, the response time for user login was 2s, but in the latest version tested, the response time has to be 4S; the
system resource occupancy rate has increased.
For example, in the previous version, the average CPU usage was 15%, but in the latest version tested, the average CPU usage became 30% and
network bandwidth usage became higher.
For example, in the previous version, the total number of bytes sent was 20MB, and the total number of bytes received was 200MB, but in the new version, the total number of bytes sent became 25MB, and the total number of bytes received became 250MB.
It should be noted here that the premise of these deterioration trends is: exactly the same environment and test load. There are different methods for checking different deterioration indicators. I will use the most common transaction response time
as an example to explain the troubleshooting method to you.
Suppose, through the comparison results of the performance benchmark test, we know that the response time of the user login has changed from 2s to 4s. So the first thing we have to do is to verify whether the response time
becomes longer in the case of a single user . The specific method is to take out the virtual user script of the user login separately, establish a single user performance test scenario and execute it, and observe whether the response time of the user login is slow
. If it becomes slow, it means that this is a single user login scenario. The performance problem can be reproduced, and the subsequent processing is relatively simple. The solution is to analyze the back-end log file of single user login, look at
the back-end log file to complete the login, and see which steps are taken to complete the login operation time, compared with the previous steps, the time is longer, or What are the extra steps?
If it does not slow down, it means that we must try to reproduce this performance problem under pressure. To this end, we need to build a concurrent test scenario based on the virtual user script of the user login, but we
are not clear how many concurrent users should be used in this scenario design and how long the thinking time should be added. At this time, the usual practice is to directly use the number of concurrent users and thinking
time in the performance benchmark to try to reproduce the problem. If it cannot be reproduced, we can gradually increase the test load appropriately and observe the changing trend of response time.
It should be noted here that you must not use excessive test loads. If the test load is too large, system resources will also become a performance bottleneck, and the response time will definitely become longer. But this time, the longer response time is
mainly caused by resource bottlenecks, not the reason you started looking for.
If the problem can be reproduced at this time, you can further analyze the time slice of the user login operation in the concurrent scenario to find the specific reason. If the problem cannot be reproduced at this time, the situation is more complicated
, That is, the performance of the login operation may depend on other business operations, or a certain resource competition relationship, this is a specific analysis of specific problems.
In general, when the cause of performance "deterioration" is located and fixed, we will perform another round of performance benchmark tests to ensure that the performance benchmark indicators before the system is released have not changed. It can be said that
through the performance benchmark test of each pre-release version, we can ensure that the overall performance of the new release system environment will not decline, which is the ultimate goal of performance testing.
Many large traditional software companies have dedicated performance testing teams. This team will establish standard performance benchmarking scenarios and use the results of the performance benchmarking as
one of the basis for whether the product can be released . For example, the HP software I used to work on was maintained by the Performance Testing Center of Excellence, performing performance benchmark tests, and analyzing test results.
From the perspective of the design of the performance benchmark test, you need to pay special attention to the following three points:
1. The selection and ratio of virtual user scripts in the performance benchmark test need to match the actual load situation as much as possible;
2. The overall load design is not suitable Too high, usually all kinds of occupancy index of the tested system need to be controlled within 30%, try to avoid the operation delay due to resource bottlenecks;
3. Before each performance benchmark test, generally need to do a system and network resources A round of rapid benchmark tests to ensure the consistency of the environment under test each time, but also to ensure that the data volume of the database is at the
same level. In short, you need to use all possible means to ensure environmental consistency between multiple performance benchmarks.
Stability test
Stability testing, also known as reliability testing, mainly simulates the test load of the system under test for a long time to observe whether the system has potential problems during long-term operation. Through the monitoring of system indicators
, stability tests can find problems such as memory leaks and illegal occupation of resources.
Many enterprise-level server-side products often undergo stability testing before they are released. The stability test usually uses the virtual user script in the performance benchmark test directly, but the design of the performance test scenario
and the performance benchmark test scenario will be very different: the "wave" test load is generally used, such as gradually increasing the test load first In the case of high load, it lasts for more than 10 hours, and then gradually reduces the load.
This constitutes a "wave", and the entire stability test will consist of multiple such waves in succession.
Stability tests successfully completed flag, mainly in the following three:
all monitoring indicators of system resources is not an irreversible upward trend exists;
the response time of the transaction trend gradually slowed down does not exist;
the error rate does not exceed 1% of the transaction
the actual project In the project, due to the high cost of the stability test execution time, it often takes 3-7 days, so we generally
start the stability test after all other tests have been completed and all problems have been fixed.
In addition, in order to shorten the stability test time, some companies often adopt the "time axis compression" method. The specific approach is: under the premise of increasing the test load, appropriately shorten each "wave"
time, thereby reducing the overall Test execution time.
Finally, it should be emphasized that although a lot of time, especially when the product version has gradually matured, the stability test will not find problems, but do n’t underestimate the value of
stability testing because once the stability test is found Problems, then these problems are very serious and very hidden problems.
Therefore, many large enterprise software companies will perform strict stability tests, and use the results of the stability test as a hard requirement for the product to be released.
Concurrency testing
Concurrency testing is a test method to verify the correctness and performance of a single business function under high concurrency. High concurrency testing generally uses virtual user scripts with zero thinking time to initiate
tests with "aggregation points" . Concurrency testing is often used as a supplement to functional testing and is mainly used to find errors such as multi-threading, resource competition, and resource deadlock. To perform concurrent testing, you need to add a "rendezvous point"
so often need to modify the virtual user script. There are generally two ways to join rendezvous points:
1. Add directly during the recording of the virtual user script;
2. Add in the virtual user script by adding the lr_rendezvous () function.
Capacity planning and testing
Capacity planning test is a test designed and executed to complete capacity planning. What is capacity planning? The so-called capacity planning is a process in which software products adjust their production capacity to meet the user's target load.
Therefore, the main purpose of capacity planning is to solve how we should
increase the overall load processing capacity of the system through vertical expansion (increasing the hardware resources of a single machine) and horizontal expansion (increasing the number of machines in the cluster) when the system load is about to reach the limit processing capacity The problem. At present, the main method of capacity planning is based on horizontal expansion. However, how many machines should be added, and whether the load processing capacity of the system will
increase linearly after the increase, these problems need to be verified through capacity planning.
So, what exactly do capacity planning tests do? We can use the virtual user scripts in the performance benchmark test and the percentage of each business operation script to test the system under test deployed by a single machine. We will use
manual methods to continuously increase the test load until the throughput index of the single-machine system reaches a critical value, which can reach the processing capacity of a single machine.
to sum up
Four types of performance testing methods, how to complete these tests in actual projects to ensure the performance of the software.
Performance benchmark tests can ensure that the overall performance of the newly released system will not decline;
stability tests mainly simulate the test load of the system under test for a long time to observe whether the system has problems during long-term operation;
concurrency tests are often regarded as Functional testing is added to discover problems such as multithreading, resource competition, and resource deadlock.
Capacity planning and testing is mainly used to determine the size of the system cluster under a given load, and its test results can be used as a basis for system capacity design.
