Performance testing purpose
Performance tuning
After developers tune the system, they need the cooperation of testers to do performance testing to verify whether the optimization is effective. If the performance indicators are better than the previous performance indicators, it means that the system optimization is effective. On the contrary, it means that the tuning is not ideal.
New services and new interfaces are launched
The system is put online from scratch to verify whether the new system's capabilities can meet the system usage requirements for a certain period of time. Otherwise, when the peak period comes, the system may collapse.
Verify system stability
For performance testing, running for an hour or two or dozens of minutes is enough, but the stability of the system and whether the system can work stably for a long time are not enough. Stability problems that occur in the system generally include memory leaks, connection number leaks, deadlocks, insufficient cursors, etc. These problems may not be exposed in a short period of time. In order to verify the system stability, the peak performance concurrency number (that is, the maximum number of concurrency supported by the system) * 7 days * 24 hours is generally used. If the system throughput, average response time and other performance indicators are normal, the system stability is considered to be OK. of. According to the importance of the system, the running time can be adjusted appropriately, but at least one night of stress testing is required.
Verify system architecture for bottlenecks
For the same system, architects will provide different architectural solutions. So which of the different design solutions is better? We can verify the performance of different solutions through performance testing. Secondly, we can understand what level of concurrency the system will reach before the system will have problems, and understand which part of the system architecture will have bottlenecks when it reaches the peak performance. , to help developers perform targeted system tuning work.
Performance test scope definition
The following points are generally considered:
- Frequently used functions and called interfaces in the system, etc.
- The system involves a large number of database reading and writing functions
- The function of reading and writing a large number of system cache parts to verify whether the cache is effective
Generally speaking, places with a large number of user visits, frequent database operations, and core system functions should be considered within the scope of performance testing.
Performance testing principles
3+1 principle (referring to quantity, fullness, depth + speed)
Mainly focused on performance test design, test execution and data analysis.
Volume: including business volume (business type), load volume (traffic processed by the system), configuration volume (software configuration and hardware configuration), user volume (static users and dynamic users), and time volume (testing time).
Full: Mainly for test cases. The test case management includes three parts: preset conditions, test steps and expected results. This "full" focus is on the test result observations and data of the preset conditions and test steps.
Deep: First, the understanding of the system must be deep, and second, the analysis of defects must be deep.
Fast: First, the solidification of test experience to avoid omissions in test design and observation. Solidifying experience into templates or tools facilitates the inheritance of experience and reduces test duplication and omissions; the second is the automation of performance testing, including the automation of performance testing environment construction, test execution, and test analysis. Automation has greater value in improving performance testing efficiency. .
2-5-10 principle
Mainly for response time. To put it simply, when users can get a response within 2 seconds, they will feel that the system's response is fast; when users get a response between 2-5 seconds, they will feel that the system's response speed is okay; when users get a response between 5-5 seconds, they will feel that the system's response speed is okay; When a response is received within 10 seconds, the system's response speed will feel slow, but acceptable; and when the user still cannot get a response after more than 10 seconds, the system will feel bad, or the system has become unresponsive. Choose to leave this Web site, or initiate a second request.
80/20 principle
Used to reduce risks and focus on more testing: the 80/20 principle is the Pareto Principle. Users use 20% of the functions in a software product 80% of the time. "Key test" is to test these 20% of the functions, while the other 80% of the functions belong to the low-priority test range, accounting for 20% of the test resources.
Example: Estimating Test Intensity
Basic concept: 80% of business is done in 20% of each workday.
For example: if you work 8 hours a day, then 80% of your daily business will be completed within 8*20%=1.6 hours.
For example: About 1 million businesses were processed last year. Among them, 15% of the business processing required 7 requests to the application server for each business; 70% of the business processing required 5 requests to the application server for each business. request; in the remaining 15% of business processing, each business submits 3 requests to the application server. According to past statistical results, the annual business increment is 15%. Considering the needs of business development in the next three years, the test needs to be conducted at twice the existing business volume.
The intensity is estimated as follows:
The total number of requests per year is:
(100*15%*7+100*70%*5+100*15%*3)*2＝10 million times/year
The number of requests per day is:
1000/160=62,500/day <Note: Each month has 20 working days, so 1 year is 160 days>
Number of requests per second: (62500×80%)/(8*20%*3600)=8.68 times/second
Performance test environment setup
Ensure the consistency between performance testing and the real production environment, specifically from the following three aspects:
Hardware environment
For example, the model of the server, whether the server is shared with other applications, whether it is in a cluster environment, whether load balancing is performed through BIGIP , the hardware configuration used by the customer, the switch model used, and the network transmission rate.
Software Environment
version consistency
Including the versions of the operating system, database, middleware , and the version of the system under test.
Configuration consistency
The system (operating system/database/middleware/system under test) parameters are configured consistently. The configuration of these system parameters may have a huge impact on the system. Therefore, in addition to ensuring that the software version used in the test environment is consistent with that used in the real environment, attention should also be paid to whether the configuration of its parameters is consistent.
Consistency of usage scenarios
Basic data consistency
Including the predicted amount of business data and the allocation of data types. A very simple example, a system database has only 10 pieces of data and a database has tens of millions of pieces of data. When we perform performance testing on it, the performance indicators obtained may be very different.
In order to ensure a more consistent test environment each time, disk usage and disk fragmentation will also affect the performance to a greater or lesser extent.
Consistency of usage patterns
Try our best to simulate the user's usage in real scenarios. In fact, we are doing demand analysis in the early stage of performance testing, and its main purpose is to simulate the user's usage in a more realistic way.
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can help you too!