In the usual software production cycle, users first propose user requirements or system
requirements , and developers then propose software requirements specifications through requirements analysis, carry out outline design, propose outline design instructions, and
carry out detailed design. A detailed design specification is presented, and the final step is to code each module. In the testing stage, the testing
is verified and implemented step by step according to the development process, which reflects the idea of verifying the system layer by layer from the part to the whole, from the low level to the high level
. Corresponding to the software development process, software testing steps are divided into code review, unit testing, integration testing, and system testing.
The performance test is a software system-level test, and its ultimate purpose is to verify whether the user's performance requirements are met. Under
this goal, performance testing is often used to:
(1) Identify system bottlenecks and the causes of bottlenecks;
(2) ) optimize and adjust the configuration of the platform (including hardware and software) to achieve the highest performance;
(3) determine whether a new module has an impact on the performance of the entire system.
Typical distributed system architecture: as shown in the figure
If the software system is regarded as a transportation system, then the network is a road, and each system node such as client, firewall, load
balancer , web server, application server (middleware), database, etc. is the traffic fortress, and customer
requests and data Just like a vehicle on the road, if there is a traffic jam somewhere, the entire transportation system will be blocked.
At this time, we have to analyze where the problem is, whether the road is not wide enough, or whether the overpass is not properly designed
and causes blockage. Find the key point of the problem, then this key point is the bottleneck of the system. The same is true for software systems, and
most of our performance testing is to find where the bottleneck is.
1. Performance testing is not functional testing.
Performance testing does not require and cannot cover all the functions of the software. Usually, we only perform performance testing on some functions or
modules . In general, we need to follow the following principles when selecting performance test cases:
(1) Basic and commonly used For
example, an E-mail system, the basic and commonly used functions include registration, login, email reception, and email query. Users
use these The frequency of the function is high, and the performance test is to be done. However, functions such as advanced queries, filters, and mailing lists are used
less frequently, so performance testing can be omitted, or the priority of performance testing is lower.
(2) Such requirements for demanding response time
often appear in systems with high real-time requirements such as finance and telecommunications. For example, starting from a mobile phone call
, going through the base station, the core network, and then to the ringing of the called mobile phone, the processing time of the entire system should be within the
range . In addition, a base station responsible for communicating with mobile phones must be able to quickly resume working after a failure or power failure.
These functions all have strict requirements on time, and performance tests must be done. Of course, in actual operation,
the performance tests done when the telecommunication system goes online is not limited to these functions.
The subdivision of these functions is the transaction in the performance test.
Common performance testing methods are as follows:
1. Load test
Here, load test refers to the most common performance test to verify general performance requirements. We
mentioned the most common performance requirement of users is "both for horses to run and for horses to eat less grass" . Therefore, load testing mainly examines
the performance of the software system under a given load. We can understand the load test as follows:
(1) The load test is to observe the performance of the software system under certain conditions from the user's point of view.
(2) The expected result of the load test is that the user's performance requirements are met. This indicator is generally reflected in response time,
transaction capacity, concurrent capacity, resource utilization, etc.
2. Stress testing
Stress testing is to examine the performance of the system under extreme conditions, which can be overloaded transaction volume and
number of concurrent users. Note that this extreme condition is not necessarily the user's performance requirements, and may be much higher than the user's performance requirements.
It can be understood in this way that the difference between stress testing and load testing is that the expected result of stress testing is that there is a problem with the system, and
what we want to examine is the way the system handles the problem. For example, we expect a system to remain stable under stress, to
slow down, but not to crash. Therefore, stress testing is what allows us to identify the
weaknesses and how the program will behave under extreme loads.
Example: Load testing is concerned with user rules and requirements, while stress testing is concerned with the software system itself. For their
difference, we can use the example of Huashan Lunjian to describe it more vividly. If Guo Jing is regarded as the subject to be tested, then
the stress test is like Guo Jing fighting against Ouyang Feng, who has already gone into trouble.
Guo Jing had to be able to deal with it and not lose his life. And the routine performance test is like Guo Jing and xanthate
Master and Hong Qigong agreed that as long as Guo Jing could pick up one hundred moves from each of the two masters, Guo Jing would win. As for the three hundred moves, even if
Guo Jing loses, it doesn't matter. As long as he can take over a hundred moves, he will pass.
Thinking
that when we do software stress testing, we often need to add more concurrent users and transactions than load testing.
Why ?
3. Concurrency testing
verifies the concurrent processing capability of the system. Generally, a large number of concurrent connections are established with the server, and the response
time of the client and the performance monitoring of the server are used to determine whether the system has reached the established concurrency indicator. Load testing often
uses concurrency to create load. The reason why concurrency testing is proposed separately is that concurrency testing often involves
the concurrent capacity of the server and the possible problems caused by multi-process/multi-thread coordination and synchronization. This is something to pay special attention to and must be tested.
4. Benchmark test
When a new module is added to a software system, a benchmark test needs to be done to judge the performance impact of the new module on the entire software
system . According to the benchmark method, you need to open/close the new module at least once for each test. Each performance index of the system before the module is turned off
is marked as a benchmark, and then compared with the system
performance to judge the impact of the module on the system performance.
5. Stability test
"Road to know horsepower", here we want to talk about the stability test related to the performance test, that is, to test
whether the system will have problems after running for a long time under a certain load. Some problems of software systems cannot be exposed at once,
or it takes time to accumulate to the extent that they can be measured. Why would such a test be needed? Because some software
problems will only be exposed after running for a day or a week or even longer. This problem is generally caused by the program occupying resources
but not released in time. For example, the memory leak problem will gradually become obvious after a period of accumulation,
but it is difficult to detect it in the early stage of operation; and the client and the server have established a large number of
connection , but cannot Efficient reuse or timely release.
6. Recoverable testing
tests whether the system can quickly recover from an error state to a normal state. For example, in a system with load balancing
, can the backup machine quickly take over the load after the main machine is under pressure and cannot function normally. Recovery testing is usually done
in conjunction with stress testing.
Hint: Every test has its place and purpose. When we take over a software project, under the
condition , we choose which kind of test to do, which should be selected according to the current software process stage and the characteristics of the project itself.
For example