Table of contents
2. What is performance testing?
3. What parameters need to be recorded for performance test results?
4. How to do performance testing (performance testing process)?
5. Common performance testing tools
1. What is performance?
Time complexity and space complexity : Performance not only includes processing efficiency, but also resource usage;
Under the same efficiency, the smaller the resources (CPU, memory, IO, etc.) occupied - the higher the performance;
Under the same resource usage (CPU, memory, IO, etc.), the higher the processing efficiency - the higher the performance;
Computer resources: CPU, GPU, memory, disk, network, etc.; performance problems are generally reflected in insufficient resources (mainly CPU, memory and IO);
Of course, there are also some performance problems that arise from the program. For example, if load balancing, single-core work, etc. are not done, the overall resources of the device will be mostly idle, but the processing performance will reach a bottleneck;
2. What is performance testing?
To put it simply: it is a test that tests whether the system and network equipment meet performance requirements under load conditions by simulating increased load, such as stress testing, load testing, capacity testing, etc.
| load test | It is a method of testing the performance bottlenecks or performance of systems and network equipment by continuously increasing the load. When the load is increased in a stepwise manner or reaches a certain load, find out the performance inflection point when the system CPU, memory, and network are insufficient. Example 1: Measure the maximum concurrency performance of the web system login interface; |
| pressure test | It is a method to test whether the system and network equipment are stable under extreme conditions and for a long time under extreme load. The main purpose is to test the fault tolerance and recoverability of the system under extremely abnormal high pressure. Stress testing is a long-term, high-load test. Example 1: Test whether the product display page of the web system is displayed normally under heavy pressure (exceeding the maximum load supported by the system) and whether the system is stable; |
| Capacity test | It is a method to test the maximum and optimal capacity of the system. For example: the maximum number of user authentications, the maximum number of module configurations, the maximum storage capacity, etc. Capacity will be covered in indicator testing, performance scenario testing, and stability testing. Example 1: Measure the maximum number of registered users supported by the web system and the maximum number of products supported by the shopping cart; |
Different companies usually have their own defined performance test items at work, which are essentially based on the above three basic concepts (load, pressure, capacity). Here is an example of performance test items commonly designed at work;
In testing activities, you will come into contact with indicator testing, reliability testing, and stability testing. The following focuses on them:
the difference:
1. Indicator testing : It prefers test modeling to verify various indicator data of the system under different loads. The test model, environment, methods, instruments, and reports are relatively standardized, and there is generally automated benchmark testing.
2. Reliability testing : It prefers to test simulated users’ real business and behavioral operations, as well as fault tolerance and recovery that may occur under high pressure. Generally, the test environment is temporarily built and reliability use cases need to be designed.
3. Stability test : It is also a type of reliability test, but the requirements are more stringent. Deployment, configuration, operation, pressure, monitoring, etc. must be strictly observed and recorded, and normal operation must be maintained for more than 2 weeks (recalculation if abnormality occurs). Until the project is released (some projects will continue to run for a few weeks after release)
Performance testing purpose:
- Evaluate system capabilities and compare data to help planners make next decisions
- Performance tuning to continuously improve product software performance
- Meet users' large-scale business needs and solution needs
- Provide product performance parameter table for customer selection or PK with friends.
- Assess system resources and promptly identify risks of insufficient resources
- Evaluate the impact that turning on a single module may have on overall performance
3. What parameters need to be recorded for performance test results?
As mentioned above, there are many performance tests, but the first test is usually the indicator test, which tests the maximum processing capacity of the system . The test results usually include the following.
Common performance parameters:
1. Number of new connections: Usually refers to the number of new connections per second for layer 4 TCP. For example: Nat module supports a maximum of 210,000 new creations per second.
2. Number of concurrent connections: Network equipment is usually tested to maintain the total number of layer 4 connections (tcp/udp/icmp), also called the number of sessions.
3. Throughput: The amount of data passing through the device per second (Bps/bps), which can be broken down into uplink throughput and downlink throughput.
4. Number of users: Usually when testing network equipment, it refers to the total number of active online users on the device.
5. Packet forwarding rate: data packets passing through the device per second (pps), which can be broken down into uplink pps and downlink pps.
6. The performance experience indicators of the page include: no-load condition, normal load condition and full load condition.
7. Resource usage: CPU usage, memory usage, and IO usage under various loads;
Different system test items are usually different. For example, the underlying network equipment usually tests the new and concurrent connection performance of layer 4; the server system usually tests the maximum concurrency (TPS) and response delay of layer 7, and records the performance under the maximum performance load. Resource usage (CPU, memory, IO).
4. How to do performance testing (performance testing process)?
The performance testing process usually includes five processes:
1. Indicator modeling
The biggest difficulty above lies in indicator modeling. The test model must consider: what kind of test environment configuration (how many cores and how many G), configuration switches and ratios, traffic services and ratios, etc. Finally, start designing performance scenario use cases based on the determined test model;
Taking a new functional module as an example, what are the performance scenario use cases that usually include it?
- Module specification baseline: Test the performance indicators of the maximum capacity specification (such as: resource usage under a full configuration of 10,000 online users);
- Module management surface performance: page response indicators, configuration delivery indicators (such as: response time for accessing the shopping cart page when the CPU load is 80%);
- Module data plane performance : the performance indicators of the module itself in user scenarios (such as: the maximum concurrent access supported by the product display page; resource usage after the module is enabled, etc.)
2. Diagnosis and tuning
After testing the performance data, how to determine whether the performance meets the standards? Usually the performance requirements are defined in the requirements stage, similar to the following definition:
Supports 10,000 concurrent user access, 95% of request response delays are less than 1s, and the module CPU usage does not exceed 50% under maximum load, and the memory usage does not exceed 1G.
If the performance is not up to standard, resource usage data needs to be provided to assist R&D in analyzing:
5. Common performance testing tools
The performance of network gateway equipment usually uses Spirent (spt) instruments to test the layer 4 network throughput performance;
Most Internet companies have web systems, and testing mainly focuses on web interface performance. Jmeter, loadrunner, or python are usually used for testing. The commonly used python tool for testing interface concurrency performance is locust.
You can basically find tutorials on how to use tools online, so I won’t describe them here.
If you have read this carefully, then I believe you already have a basic understanding of performance testing. Let’s go, boy. Performance testing is a systemic function that is not easily replaced by AI. The current salary for related positions is also good.
If you find it useful, give it a like or add it to your favorites!