Summary from veteran, detailed performance testing process + difference between load/concurrency/stress testing...

---

Preface

Performance testing process

1. Access to performance test interface documents

1) System architecture design documents (system infrastructure diagram, business architecture diagram, data flow diagram).

2) Non-functional requirements document (performance test indicators such as: PV, number of users, TPS, concurrency, response time, system peak CPU, memory consumption, etc.).

3) Resource action decomposition table (application for stress testing environment, including hardware configuration and quantity information).

4) Test scenario (core business).

2. Prepare the system environment

1) Production environment

The measurement accuracy of the production environment is higher, but test data such as logs generated by the test need to be deleted, and the integrity of data deletion must be ensured. For the construction of basic data, please refer to the subsequent data volume section.

When performing stress testing on the production environment, try to select off-peak periods to avoid any impact on the production business.

2) Test environment

The risk is controllable, but the environment construction is more complicated, and the cost of consistent scale and production is the highest.

The ideal ratio is test:production=1:2 or test:production 1:4. (The ratio here is the number of overall architectures or hardware configurations, but the infrastructure cannot be changed)

However, when the actual conditions are not met, you can also use independent deployment of test clusters and shared databases in the production part for testing.

If you build a stress testing environment, after the environment is set up, you need to import basic data from production desensitization. Generally, the existing data is three years old, and at least 50,000 test case data query transactions must be prepared. Each data submission type should not be repeated, nas, etc. Three years of business volume are required to store data. (If the archiving time of business volume is less than three years, prepare according to the archiving time)

3. Test plan

The test plan should be as concise and clear as possible, and the specific person responsible for each task should be clearly explained. When dividing tasks, the organization should be clear and divided according to the entire performance testing process.

Generally, projects with continuous iterations use one week as the project cycle, and the end time is after one week, moving forward. The time for a new project is generally two to three weeks, and the specific time will be treated on a case-by-case basis.

4. Test plan

Use the information obtained from the early interface documents to write a test plan, which includes the background, purpose, indicators, test scope, system architecture... and other information of this test.

When developing a test plan, it is not enough to simply register information. You also need to understand the overall architecture of the system, core business processes, etc. Only in this way will you be able to be confident and fearless when facing possible performance problems or other problems.

5. Test execution

1) Test script

There are two types of test scripts. One is to record scripts, mainly for business systems. The other is an interface script, which is written according to the interface document provided by R&D.

There’s not much to say about the script. I’ll actively communicate with the R&D and adjust the script.

2) Scenario execution

scenes are divided into baseline, single scene, mixed scene, extreme scene and stability scene.
When executing, the business focuses on three points: transaction success rate, TPS, and response time.

The system focuses on five points: CPU consumption, memory consumption, network IO, disk IO, and Swap.
If a certain indicator does not meet the standard, use the link as a line and track it from front to back.

Take the common system architecture as an example: F5>Nginx Cluster>F5>Microservice Cluster>Redis>Mysql
First check whether F5 is a performance bottleneck, and then check the configuration of Nginx …Wait and check step by step to make sure nothing is missed.

6. Test report

After the test is completed, a corresponding test report will be compiled based on the test results.
Test scope, test indicators, actual indicators, resource consumption, performance bottlenecks, tuning records and other information.

The difference between load testing, concurrency testing, and stress testing

1. Load test

1) Definition
Load testing is to test the changes in system performance by gradually increasing the system load, and ultimately determine the maximum load that the system can withstand while meeting the performance indicators. Quantitative testing.

2) Purpose
A test that does not crash the system, so that the system can operate normally under maximum pressure. To obtain system indicators.

3) Method
Continue to increase the request pressure until a certain resource item of the server reaches saturation (for example, the CPU usage reaches 90%+) or a certain indicator reaches a safety threshold (such as Operation and maintenance monitoring alarm threshold or turning point).

System load pressure includes the number of concurrent users, continuous running time, data volume, etc. The number of concurrent users is an important indicator of load pressure.

2. Concurrency testing

1) Definition
Check whether the system has concurrency problems, such as memory leaks, thread locks, resource contention, etc.

2) Purpose
To determine the number of concurrent users, you must know the number of online users carried by the system. Then a certain amount of requests are initiated simultaneously within unit time (S).

3) Method to determine the number of concurrent users
For example:
There are 2,000 company OA system accounts or total users;
The peak number of people online is 500;

But these 500 people do not exist as concurrent users. That is to say, it does not represent the actual pressure carried by the server; it is possible that 40% of the attention is on the homepage news bulletin board and the like (note that reading news does not cause pressure on the server at this stage);

20% of users are querying information or operating tables; 20% of users are in a daze; 20% jump between pages; in this case, only 20% of users are actually having a substantial impact on the server.

We treat this query and operation form as a business category; directly call the concurrent users of this part of the business the number of concurrent users:
Calculate the average number of concurrent users: C=NL /T;
Peak number of concurrent users: C' ≈ C+3 root C;

In Formula 1, C is the average number of concurrent users; n is the number of login sessions; L is the average length of login sessions; T refers to the length of the time period under investigation.

Formula 2 gives the calculation method for the peak number of concurrent users, where C’ refers to the peak number of concurrent users, and C is the average number of concurrent users obtained in Formula 1. This formula is estimated assuming that the user's login session conforms to the Poisson distribution.

Suppose there is an OA system with 3,000 users. (You can see the registration information) On average, about 400 users access the system every day. (View the log file) For a typical user, within a day, the user goes from logging in to The average time to exit the system was 4 hours, and users only used the system for 8 hours over the course of a day.

According to formula 1 and formula 2, we can get:
C = 4004/8 = 200
C'≈200+3 square root 200 = 242

But the general practice is to take 10% of the number of users who access the system every day as the average number of concurrent users. The maximum number of concurrent users is multiplied by a value, 2 or 3.

Suppose the user requires that the system can handle a maximum of 100 login requests per second and 10/25/50/75/100 concurrent users to perform login operations, and then observe the system's response time and number of transactions per second under different loads. If the number of users is 100 and the response time is still within the allowable range, the number of users should be increased, such as 120, etc.

Personally, I understand that this number of users is set by what we often call the equivalence class and boundary value method.

3. Stress test

1) Definition
It is to continuously put pressure on the software, force it to run under extreme conditions, and observe to what extent it can run, thereby discovering performance defects.

2) Purpose
A test to crash the system.

3) Method
Based on load testing or concurrency testing, continuously put pressure on the software, force it to run under extreme conditions, and observe to what extent it can run. thereby discovering performance flaws.

The following is the most comprehensive software testing engineer learning knowledge architecture system diagram in 2023 that I compiled.

1. Python programming from entry to proficiency

2. Practical implementation of interface automation projects

3. Web automation project actual combat

4. Practical implementation of App automation project

5. Resumes of first-tier manufacturers

6. Test and develop DevOps system

7. Commonly used automated testing tools

8. JMeter performance test

9. Summary (little surprise at the end)

Chase your dreams, fear no wind and rain, embark on the journey and forge ahead. Only with courage and persistence can you create your own glory and let your life shine with splendid light. Believe in yourself, work hard, and success will come to you unexpectedly.

Every effort is an accumulation, and every struggle is a growth. Only by persevering and surpassing yourself can you write a brilliant chapter on the stage of life. Believe in your own strength, chase your dreams, and create your own brilliant life.

Burn the passion in your heart and embark on the journey. Even if the road is bumpy, you must persist in pursuing your dreams. Believe in your own abilities and move forward bravely. Only in this way can you create your own brilliant life.