Test life, test old birds, organize performance test methods, you have everything you want...

Contents: Guide

foreword

The requirement background of performance testing generally comes from the following three situations:

The first is that there is a performance problem on the existing network, and the project team specially carried out performance improvement. For example, a modified interface is changed from the original synchronous call to asynchronous, or a new API is changed from the tcp protocol to the udp protocol. In order to ensure the reliability of the newly replaced API, performance testing is required.

The second type is a new system, and the operating personnel need to take a comprehensive pulse and understand the processing capabilities of the system.

The third is that with the rapid increase in the number of requests, but the system has never been tested for performance, the project team is worried that the system will not be able to handle it in the foreseeable short term, so the testers are required to conduct a comprehensive performance test for the system. a reference data

Depending on the background, we often have different preparation methods, but we can generally start from the following aspects.

1. Fully understand the overview of the system

Expected performance indicators of the system:
For the first and second two cases, there will be very clear live network performance indicators. For example, the previously tested acs is a new system, and it is clearly marked in the requirements specification that it needs to reach 1wtps. For the second In the three cases, we often need to simulate the live network as much as possible to obtain data for operation reference.

Networking and the form of communication between various systems on the network:
Sometimes our performance transformation is just a small system in the network, which requires us to figure out where this system is in the entire logical processing.

Understanding the position of the system under test in the entire transaction is crucial for the design of test cases and the construction of test environments

Secondly, it is also necessary to understand the communication mode between various modules in the network, tcporudp, synchronous call or asynchronous call, long connection or short connection.

Each logical branch of the system:
Understanding the logical branches of the system is mainly beneficial to the design of test cases.
In our actual work process, time is always limited, and a very important way for us to improve work efficiency is to pay attention to the design of use cases. After understanding the logical branches of the system, we can accurately prepare use cases and directly address the problems. In essence, reduce the time of fumbling.

Combination of internal modules of the system:
more complex systems will have their own module combination.
We need to understand that the system is composed of several modules, and what the coupling relationship of each module is, not only for the design of abnormal test cases in functional testing, but also for performance testing.

Give a relatively simple example:
aqc system, this system is for external query.
The internal modules are roughly divided into: network communication layer, request distribution layer, and function processing layer.
The network communication layer mainly uses a certain network communication component to process network communication, and requests the distribution layer to dispatch. It mainly distributes the packets of the network communication layer queue to the back-end function processing layer according to the cmdcode. The function processing layer is composed of small svr , each svr handles different query requests.

If there is a performance requirement, it is found that the performance of a query branch on the live network is not OK, then we need to quickly lock the key module, and the bottleneck is likely to exist on the svr that processes this branch.

Secondly, it is also important to understand the various modules of the system and the coupling relationship between the modules when understanding the performance curve and adjusting the test plan.

2. Step on the key points and dig deeper into the system

The performance indicators of the system, in addition to the most typical indicators, namely throughput or response time, there are also many indicators that we need to pay attention to, such as cpu, memory, io, database connection operations, etc., so we need to dig deeper before testing system.

Find out the following key points:
memory allocation and usage. Use of message queues, use of caches;
file and network IO operations: read large files into memory, or write memory to files, whether the operations are frequent;
cpu-consuming operations: such as some large memory sorting;
database operations: frequent Perform database read and write operations, frequently establish database connections, etc.;
network calls: network delay and concurrency of connections;
critical resources: whether there is inappropriate locking behavior in the multi-process processing mode;
...

3. Design test cases

After understanding the basic situation above, it is actually a preparation for this step. It is particularly labor-saving when solving the performance requirements of the first situation. After understanding the above situation, experienced performance testers can even determine where the problem lies without designing a test!

The test case of performance test is not compared with functional test, and many abnormal tests can be considered, because the cost is very small, and the execution of a performance test case often consumes a lot of resources and time, so accurate performance test cases are particularly important.

Select the most suitable logical branch for testing:
There are often many branches that can meet your testing requirements, when you choose.
You can start from the following two points:
A. It is least affected by the surrounding area, and B. It consumes the least resources.

Point A can help us quickly locate the problem. There may be many systems designed in the entire logic branch. We can choose the branch that involves the smallest surrounding system but can include the system under test. Of course, the easiest way is to You can directly pressure test the system under test, but some problems cannot be exposed by doing so.

Point B can help us save resources. For example, there is already a ready-made environment. In short, the things I need to prepare are reduced, and the speed and efficiency are naturally faster. For new systems or old systems that have never been tested for performance, in In the case of limited time, we also need to combine the actual situation and select the most important branch with the most user requests for testing.

According to the previous analysis, design the most targeted and effective test cases:
For example, I suspect that the reason for the decline in the performance of the aqc system is that this iteration has added a large-memory sort. For example, the aqc system has a branch that queries all the CDKeys of the user, sorts them by time, and returns them all to the user.

So how do we get this question verified?
When designing, you can choose two extreme situations and combinations for testing. One is that all users have no cdkeys, the other is that all users have 500 cdkeys, and the last one is a combination of the two.

By comparing, you can verify whether it is because occasionally some users have too many cdkeys, resulting in overall performance degradation.

Of course, in the process of testing, we also need to adjust the test cases according to the existing data to help us verify the conjecture and locate the problem more clearly.

4. Preparation of test environment

The construction of the test environment is often determined according to the design of the test cases. For the system that is undergoing performance testing for the first time, our goal is to be as consistent as possible with the live network. Can mainly start from the following aspects.

Data: The large amount of data and the diversity of data are often the difficulties of simulation. A large amount of data needs to write scripts to fill the database to a certain extent. If the requirements are high, you can even use the method of guiding data from the existing network.

Diversity is often difficult to achieve. It is necessary to understand the data diversity and proportion of the live network to achieve the effect of simulation

Network delay: This is very related to the company's IDC machine management. I always thought that all the test machines were in one IDC, but later found that it was not the case. Our test machines are distributed in different IDCs like the operation machines, and we When selecting machines for deployment, it is necessary to understand the network delay between machines in operation on the live network.

This is especially important when testing the performance of an entire branch of logic.

Configuration: log level configuration, the number of threads or processes, if conditions permit, the configuration can be upgraded to the hardware configuration of the machine, if it can be consistent, it is naturally the most ideal effect.

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled