Summary of senior testing, what is the purpose of performance testing? What indicators do you mainly look at?

---

Preface

1. What is performance testing?

Performance testing is to verify whether the software system can achieve the performance indicators proposed by the user, while discovering performance bottlenecks in the software system, optimizing the software, and finally optimizing the system.

Performance testing can be summarized into three aspects:
Testing of application performance on the client;
Testing of application performance on the network;
Application performance testing on the server side;

Usually, an effective and reasonable combination of the three aspects can achieve a comprehensive analysis of system performance and prediction of bottlenecks.

At present, most applications are distributed applications based on the network. We cannot know the number of users and the uncertainty of user scenarios. As a result, when testing the system, not only the functions, business logic, and interfaces are tested, but also the system performance is tested.

There is no problem with one user, but once there are too many users, various problems may occur, so system performance testing is required.

As the number of users increases, the system debt increases. Conduct a system performance test to know the number of concurrent users the system can bear, whether the bandwidth is sufficient, whether the CPU is sufficient, whether the memory is sufficient, and whether the hard disk speed can keep up.

From the server side, test whether the server can handle multiple concurrent users, whether the system is stable, and the response time and speed from the user's perspective.

Of course, performance testing not only refers to scenarios where the number of users increases, but also in all aspects.

Take a shopping website as an example. It can include the following performance testing content (not limited to this):
Sudden large number of users operating at the same time;
Long-term operation;
Operation under weak network (can continue to work normally under no matter how bad the network is);

2. Purpose of performance testing

1) Assess system capabilities

Load and response time data obtained from testing can be used to verify the capabilities of the planned model and aid decision-making. (Check whether it meets the indicators based on the test results. If it is the first version, it can be used as a reference for the indicators in the specification sheet).

2) Identify weaknesses in the system

Controlled load can be increased to an extreme level and breached, thereby repairing bottlenecks or weak points in the system. (Find weaknesses and continuously improve throughout the product life cycle. This is the so-called knowing yourself and the enemy).

3) System tuning

Improve performance by running tests repeatedly to verify that activities to tune the system have the expected results.

4) Detect problems in software

Long test execution can cause the program to fail due to memory leaks, revealing hidden problems or conflicts in the program. (The results can be predicted through the memory trend chart).

5) Verify stability (resilience) and reliability (reliability)

Performing tests under a production load for a certain period of time is the only way to evaluate whether system stability and reliability meet requirements.

3. What indicators do performance tests mainly look at?

Performance testing is different from functional testing. Functional testing only requires the realization of software functions, while performance testing tests whether the execution efficiency of software functions meets requirements.

For example, if a software has a query function, the functional test only tests whether the query function is implemented, while the performance test requires that the query function is accurate and fast enough.

However, for performance testing, it is difficult to define how fast the query speed is fast enough and what kind of query situation is accurate enough. Therefore, some indicators are needed to quantify these data.

Commonly used indicators for performance testing include response time, throughput, number of concurrent users, TPS, etc., which are introduced below.

1) Response time

Response time (ResponseTime) refers to the time required for the system to respond to user requests. This time refers to the time required for the entire process from when the user issues a request to the software client to when the user receives the returned data, including the processing time of various middleware (such as servers, databases, etc.).

The shorter the response time, the faster the software responds and the better the performance.

However, the response time needs to be combined with the specific needs of the user. For example, the response time of the train ticket inquiry function can generally be completed within 2 seconds, but when downloading movies from the website, the download speed is already very fast in a few minutes.

The response time of the system will become longer as the number of visits and business volume increases. Generally, during performance testing, in addition to testing whether the normal response time of the system meets the requirements, the response time of the system under certain pressure will also be tested. Variety.

2) Throughput

Throughput refers to the amount of work that the system can complete per unit time. It measures the processing power of the software system server.

The measurement unit of throughput can be the number of requests/second, the number of pages/second, the number of visitors/day, the number of business processes/hour, etc.

Throughput is a very important indicator for a software system to measure its own load capacity. The greater the throughput, the more data the system can process per unit time, and the stronger the load capacity of the system.

3) Number of concurrent users

The number of concurrent users refers to the number of users requesting and accessing at the same time. For example, for a certain software, if 100 users request to log in at the same time, the number of concurrent users will be 100.

The larger the number of concurrent users, the greater the impact on system performance. A large number of concurrent users may cause slower system response, system instability, etc. Concurrent access must be considered when designing software systems, and test engineers must also conduct concurrent access testing when performing performance testing.

4）TPS(TransactionperSecond)

TPS refers to the number of transactions and transactions that the system can process per second. It is an important indicator of the system's processing capabilities.

5) Click-through rate (HitsperSecond)

The click-through rate refers to the number of HP requests submitted by the user to the web server per second. This indicator is a unique performance indicator of web applications. The click-through rate can be used to evaluate the load generated by the user and determine whether the system is stable. The click-through rate is only a reference metric to help measure the performance of the web server.

6) Resource utilization

Resource utilization refers to the software's use of system resources, including CPU utilization, memory utilization, disk utilization, etc. Resource utilization is an important parameter for analyzing software performance bottlenecks.

For example, for a certain software, the expected maximum number of visits is 10,000, but when the number of visits reaches 6,000, the memory utilization has reached 80%, limiting the increase in the number of visits. At this time, it is necessary to consider whether the software has defects such as memory leaks, so as to proceed optimization.

The following is the most comprehensive software testing engineer learning knowledge architecture system diagram in 2023 that I compiled.

1. Python programming from entry to proficiency

2. Practical implementation of interface automation projects

3. Web automation project actual combat

4. Practical implementation of App automation project

5. Resumes of first-tier manufacturers

6. Test and develop DevOps system

7. Commonly used automated testing tools

8. JMeter performance test

9. Summary (little surprise at the end)

Every effort will be a catalyst for success. Don't give up on your expectations for the future, believe in your abilities and potential, strengthen your beliefs, meet challenges, and move forward courageously, and you will gain your own glory in life.

Find strength in adversity and wisdom in failure. Don't stop, move forward bravely, write your own glorious chapter with persistence and hard work, and achieve the true value of life.

Move forward bravely and keep pace. No matter how long and difficult the journey is, you must firmly believe in the power of struggle. Believe in your potential, embrace challenges, and work hard to pursue your dreams, and you will create a dazzling life of your own.