The most detailed performance test in the whole network, analysis of various test scenarios + performance test benchmark (ultra-detailed summary)

---

foreword

In the face of increasingly complex business scenarios and different system architectures, the preliminary requirements analysis and preparation work will take a lot of time. Different testing strategies are also crucial to whether our test results meet the expected goals.

1. Performance testing - testing strategy

During the implementation of performance testing, for different business scenarios, we will choose different testing strategies after analysis and scenario modeling. The following detailed explanations cover most of the scenarios.

Concurrency test
Simulate client requests, initiate a certain amount of requests at the same time (S) per unit time, and verify whether the system has concurrency issues.

PS: Don't be brainless and high concurrency! ! !

Load testing
Continuously increase the request pressure until a certain resource item of the server reaches saturation (for example, the CPU usage reaches 90%+) or a certain indicator reaches a safety critical value (such as the monitoring alarm threshold or inflection point of operation and maintenance);

Load testing (also called stepped stress testing) is generally used to find the inflection point of performance and verify whether the system can run normally under different request pressures of the existing test environment.
Examples are as follows:

Capacity testing
adopts the load testing strategy to verify the maximum performance of the system under test under the existing test environment (acceptable maximum performance, not necessarily optimal performance).

Limit test
In the existing test environment, regardless of the limit of resource usage (CPU usage rate above 95% or abnormally busy IO or high Load value), the maximum processing capacity without system downtime.

PS: Due to the different business scenarios of the systems under test, the adoption rate of this strategy is relatively small.

Configuration test
Constantly adjust the configuration of all aspects of the system (software, hardware, parameter configuration, etc.), and verify the optimal configuration when the performance is optimal (the optimal performance must be a balance point found by weighing various factors).

Surge test
To verify whether the system can provide services normally and stably when the system has a sudden increase in concurrency or a large fluctuation of requests within a certain period of time.

PS: This test strategy is relatively seldom used, and it is mainly aimed at uncertain short-term peak traffic influx scenarios (such as divorce, love, and breakup topics on a Weibo).

Stability test
Use a constant number of concurrency (according to the results of the load test, the corresponding number of concurrency when the CPU usage is 70%) to verify the performance of the system in mixed scenarios.

Batch processing test
Verify that the system under test is in the existing environment, and whether the batch processing (usually a crontab or triggered job) business capability of the system can meet the business demand indicators of production.

High availability test
In the case of multi-node or distributed clusters, destroy one or more of the cluster nodes to verify whether the system can restore service capabilities in time.

2. Performance Test - Applicable Scenarios

The above test strategies are divided according to the applicable business & test scenarios, the purpose of using the strategies, and the frequency of the scenarios, and are for reference only.

① Small and medium-sized teams: conventional test strategy selection: concurrency, load, capacity, configuration, batch processing, stability, and high availability strategies, which can cover most of the requirements.
②E-commerce business: High concurrency, high availability, and stability are the top priorities.
③Business scenarios: Many times a performance requirement includes several business scenarios, but concurrency, load, capacity, and stability are all recommended.
④Requirement scenarios: The requirements analysis and scenario modeling are not done well, and the test results often deviate greatly.
⑤Pressure testing environment: The research and selection of the environment, the configuration of the recommended and production environment to minimize the deployment, this is the balance between cost and result accuracy.
⑥Test data: Whether it is the amount of data, the validity of the data, and the coverage of hot data, all determine the reference value of the test results.
⑦Technical construction: Only when the infrastructure (including environment, service deployment, detailed monitoring system, and problem handling process) is complete can the performance test be moved to the left.
⑧ Documentation construction: We must pay attention to document construction and data retention, so as to avoid a lot of unnecessary trouble and repetitive work.
⑨Platformization: The role of the platform is to standardize the process and integrate the efficiency of multi-person collaborative work. Do not pursue platformization excessively (but you must have technical planning and program preparation).
⑩ Don't be brainless and high concurrency! ! !

3. Performance test - benchmark test

By designing a reasonable test method, selecting an appropriate test tool and the system under test, a quantitative and comparable test of a certain performance index of a specific target scene can be realized.

Benchmarking must be done under controlled conditions.

In the face of increasingly complex systems, increasing number of users, and multiple business systems that may be involved in performance testing, only when the business scenarios, system architecture, and test environments involved in benchmark testing are under control,

Only in this way can relatively accurate results be obtained, providing reference and basis for capacity planning, defect location, and system tuning.

Significance
To determine the limits of systems and applications for capacity planning;
to provide references for configuration test parameters and configuration options;
to determine whether the system has the capabilities it claims for acceptance tests;
to provide long-term data statistics sources and comparisons for the establishment of performance baselines benchmark;

Principles
Test strategy: Stable and continuous workload, run multiple times, look at the normal distribution trend of the test result data, try to take the average value; data
statistics: what is the average value and peak value of the test data in the real environment, and the value of the value Dimensions;
Differential risks: clarify what risks exist, the impact of risks on test results, whether to ignore them;
special circumstances: what special circumstances exist, and whether there are corresponding solutions (such as payment service calls in payment scenarios, whether to use baffles, etc.) ;

Factors to be considered Transaction ratio: In some business scenarios, a process
contains multiple transactions.
How to simulate (surge test) when there is a large traffic impact or a sudden drop in the number of requests;
system configuration: the system configuration in different environments is different, how to convert and compare the test results?
Test duration: during test execution, how long it takes to run, the time allocation of different transaction runs, etc.;
result display type: how to display the average, peak, and percentage values, and how to compare them?
Proportion of success/failure: Statistics on the proportion of successful and failed transactions during each test;
whether it can be reproduced: if an error or some abnormal situation occurs during the test, can it be reproduced?
Is it comparable: Are there other test tools or test results for comparison (execute the test as many times as possible to compare the test results: learn about standard deviation and normal distribution?)?

The Simple Way to Work
Gradually increasing the system load is an easy way to determine the maximum throughput a system can handle, and a viable strategy for finding inflection points in system performance (step stress testing).

Emphasis
The benchmark's work focuses on statistical analysis:

Statistics can be carried out from the following dimensions:
choose appropriate test tools, set reasonable test methods and system performance indicators that need to be confirmed;
choose different test tools, compare test results, and choose a stable and reflective system. Performance results;
execute tests multiple times, collect a large number of test data sets and indicators;
interpret and analyze data from different dimensions, and generate reports.

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled

1. From entry to mastery of Python programming

2. Interface automation project actual combat

3. Actual Combat of Web Automation Project

4. Actual Combat of App Automation Project

5. Resume of first-tier manufacturers

6. Test and develop DevOps system

7. Commonly used automated testing tools

Eight, JMeter performance test

9. Summary (little surprise at the end)

There will always be ups and downs in life, but as long as you have a firm goal and move forward bravely, even if the road is bumpy, you can walk out your own broad road. Let us bravely pursue our dreams, constantly surpass ourselves, and create our own wonderful life!

Life is like a marathon, not the fastest runner wins, but those who can persist to the finish line are the most worthy of respect. Keep working hard and being patient, don't forget your original intention, forge ahead, and run towards your goal!

There is no born strong in the world, only those who persevere. No matter how many challenges and difficulties there are ahead, we must face them bravely, maintain faith and perseverance, go on firmly, step by step, and fight to the end!