In the face of increasingly complex business scenarios and different system architectures, the preliminary requirements analysis and preparation work will take a lot of time. And different testing strategies are also crucial to whether our test results meet the expected goals.
In this blog, I will talk about my personal understanding of common performance testing strategies and their applicable scenarios. . .
1. Common testing strategies
During the implementation of performance testing, for different business scenarios, we will choose different testing strategies after analysis and scenario modeling. The following ten testing strategies cover most scenarios.
1. Concurrency test
Simulate client requests, initiate a certain amount of requests at the same time (S) per unit time, and verify whether the system has concurrency issues.
PS: Don't be brainless and high concurrency! ! !
2. Load test
Continuously increase the request pressure until a certain resource item of the server reaches saturation (such as CPU usage reaches 90%+) or a certain indicator reaches a safety critical value (such as the monitoring alarm threshold or inflection point of operation and maintenance);
Load testing (also called stepped stress testing) is generally used to find the inflection point of performance and verify whether the system can run normally under different request pressures of the existing test environment. Examples are as follows:
3. Capacity test
Adopt a load testing strategy to verify the maximum performance of the system under test under the existing test environment (acceptable maximum performance, not necessarily optimal performance).
4. Limit test
In the existing test environment, regardless of the extreme situation of resource usage (CPU usage rate above 95% or abnormally busy IO or high Load value), the maximum processing capacity without system downtime.
PS: Due to the different business scenarios of the systems under test, the adoption rate of this strategy is relatively small.
5. Configuration test
Constantly adjust the configuration of all aspects of the system (software, hardware, parameter configuration, etc.), and verify the optimal configuration when the performance is optimal (the optimal performance must be a balance point found by weighing various factors).
6. Surge test
Verify that the system can provide services normally and stably when the system has a sudden increase in concurrency or a large fluctuation in the amount of requests within a certain period of time.
PS: This test strategy is relatively seldom used, and it is mainly aimed at uncertain short-term peak traffic influx scenarios (such as divorce, love, and breakup topics on a Weibo).
7. Stability test
Use a constant number of concurrency (according to the results of the load test, the corresponding number of concurrency when the CPU usage is 70%) to verify the performance of the system in mixed scenarios.
8. Batch testing
Verify that under the existing environment of the system under test, whether the system's batch processing (usually a crontab or a triggered job) business capability can meet the production business demand indicators.
9. High availability test
In the case of multi-node or distributed clusters, destroy one or more of the cluster nodes to verify whether the system can restore service capabilities in time.
10. Fault tolerance recovery test
Verify that the system can maintain its ability to provide services normally in the event of a failure or self-recovery after a failure. For example, the following picture:
The larger the area of a1, the stronger the processing capability of the system; the larger the area of a2, the better the stability of the system; the larger the area of a3, the better the fault tolerance of the system (tsk tsk, the picture is a bit ugly...)
I have manually drawn several performance model diagrams before, but I can't find them, which is embarrassing. . .
2. Applicable scenarios
The above ten test strategies are divided according to the applicable business & test scenarios, the purpose of adopting the strategies and the frequency of occurrence of the scenarios, and are for reference only.
3. Talking from experience
1. Small and medium-sized teams: conventional test strategy selection: concurrency, load, capacity, configuration, batch processing, stability, and high availability strategies, which can cover most of the requirements.
2. E-commerce business: high concurrency, high availability, and stability are the top priorities.
3. Business scenarios: Many times a performance requirement includes several business scenarios, but concurrency, load, capacity, and stability are all recommended.
4. Requirement scenarios: If the requirements analysis and scenario modeling are not done well, the test results often deviate greatly.
5. Stress testing environment: research and selection of the environment, recommendations and configurations of the production environment to minimize deployment, which is a balance between cost and result accuracy.
6. Test data: The reference value of the test results is determined by the amount of data, the validity of the data, and the coverage of hot data.
7. Technical construction: Only when the infrastructure (including environment, service deployment, detailed monitoring system, and problem handling process) is complete can the performance test be moved to the left.
8. Document construction: We must pay attention to document construction and data retention, so as to avoid many unnecessary troubles and repetitive work.
9. Platformization: The role of the platform is to standardize the process and integrate the efficiency of multi-person collaborative work. Do not pursue platformization excessively (but you must have technical planning and program preparation).
10. Don't be brainless and high concurrency! ! !
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can help you too!