Performance testing Requirements analysis is different from traditional functional testing requirements. Functional testing requirements analysis focuses on analyzing the functionality, ease of use and other qualities of the object under test from the user level. Features and performance testing require analysis of services that may have performance bottlenecks in the system from multiple dimensions such as end-user applications, system architecture design, and hardware configuration.
Performance testing necessity assessment
Any project needs to conduct a necessity assessment before carrying out performance testing activities. Through necessity assessment activities, confirm whether the object under test is necessary to implement performance testing activities. Never perform performance for the sake of performance.
Under normal circumstances, necessity assessment can be analyzed by setting different conditions and different weights, and the assessment items are divided into key assessment items and general assessment items. As long as one of the key evaluation items is met, a performance test must be carried out, while the general evaluation items can be calculated through weighting. If the score exceeds 60, a performance test must be carried out.
Software testing activities can be divided into functional testing and non-functional testing according to testing requirements. Non-functional testing usually refers to performance testing. Of course, the specific situation needs to be analyzed on a case-by-case basis.
Common key evaluation items for performance testing are as follows:
1. The tested object must be reviewed and approved by the competent department or regulatory unit, and a performance test report must be provided. At present, when many companies' software products are officially put on the market for external sales and applications, government agencies, competent departments or regulatory agencies may require them to issue functional test reports, performance test reports, or even third-party test reports. In this case, it is necessary to Conduct performance testing;
2. Systems involving property life safety. Usually, e-commerce systems, financial business systems, medical and health assessments, those involving user or bank financial transactions, and life safety require performance testing;
3. A large-scale system put into production for the first time, with core business used by a large number of users;
4. System core database, business logic, software and hardware upgrades. Compared with the historical system, the system's core database, business logic adjustment, software and hardware equipment upgrades also need to be implemented for performance testing;
5. Historical versions have major non-functional defects or unevaluated items with high risks;
6. Business volume, user volume, and nodes increased by more than 30%. After the system upgrade, if the business volume, user volume, and application nodes increase by more than 30%, the specific values can be adjusted according to the actual situation. The growth of application nodes generally means that banks increase application nodes due to business needs, and banks expand branches, sub-centers, branches, business outlets, etc.;
7. Major changes in system architecture. Different system architectures may have large performance differences, so after the system architecture changes, performance testing must be implemented, and during this process, the system performance after the architecture change cannot be inferred by analogy;
8. After repairing serious non-functional defects in the production environment. After the major non-functional defects that occur during use of the production environment are successfully repaired, performance testing activities need to be re-carried out to verify whether the repair activities have adverse effects on the production environment.
The above only lists the key evaluation items that the author refers to in daily performance testing activities. For different key evaluation items that may exist for different industries and different test objects, readers can add or delete them by themselves.
Common general evaluation items of performance testing are mainly considered from a single version. If it is a platform, it is a key evaluation item. If it is a single version, a single component or business, the weight is evaluated from the following general evaluation items:
1. Whether it is in a core position in the platform (15 points);
2. Is there an upgrade, and the upgrade content includes external system docking interfaces, payment interfaces, Web Service calling interfaces and other interfaces associated with other systems (20 points);
3. Is there any adjustment or optimization of the deployment method (15 points);
4. Whether adjustments with higher performance risks have been added (20 points);
5. Are there components or business processes that customers require to be tested (20 points);
6. Whether it involves the repair of multiple functional defects and the process has undergone major changes (10 points).
If the total score of the above general evaluation items exceeds 60 points, a performance test is required.
Taking this ECShop platform as an example, through the evaluation of the above key evaluation items and general evaluation items, the third of the key evaluation items is satisfied: a large-scale system put into production for the first time, with core business used by a large number of users." Therefore, the ECShop platform Performance testing activities must be carried out.
Performance testing tool selection
After assessing the necessity of testing and determining the need to perform performance testing on the object under test, you need to consider which performance testing method to use. According to the business characteristics and architectural design of the object under test, the following two methods can be used to carry out effective performance testing activities.
If the object under test is implemented in batch processing and the start and end identification fields are set up in the database, you can use stored procedures or initiate batch processing. Resource monitoring can use monitoring scripts such as python scripts, shell scripts or other monitoring Tool, in the final statistics, the transaction time can be obtained by subtracting the start time from the end time, and the average transaction time can be obtained based on each transaction, which is relatively convenient.
If the object under test is not in batch mode and there may be a large amount of data interaction, you may need to use professional performance testing tools to implement it. Generally speaking, the commonly used performance testing tools in the industry are mainly the open source Jmeter and the commercial HP's LoadRunner.
Jmeter is an open source performance testing tool that is currently very popular in the market. It does not depend on the interface. Functional test scripts can also be run as performance test scripts. It does not require high technical skills of test engineers and provides parameterization and functions. , association and other functions facilitate script optimization and expansion.
LoadRunner stands out in the commercial field and has maintained a leading market share for many years. Compared with Jmeter, LoadRunner has powerful script development functions, complete function libraries and result analysis functions. The technical requirements for test engineers are relatively high, but because it has been popular in the industry for many years, LoadRunner has more information than Jmeter, making it easy to learn and apply.
When enterprises choose performance testing tools, they can customize and develop testing tools based on actual testing needs if possible, or they can choose testing tools commonly used in the market. Usually, the following issues need to be considered when choosing:
1. Can it be customized and developed to better meet actual testing needs?
2. Can enterprises afford the cost of commercial testing tools?
3. Whether the purchased test tools provide complete services and detailed training;
4. Whether the team members can master the tool skills required for testing activities.
Open source is an industry trend. This case project uses the open source performance tool Jmeter to implement performance testing.
Performance testing requirements analysis
Like functional testing requirements analysis, performance testing also requires requirements analysis for the object being tested. Generally speaking, when users or product teams set performance test requirements, they will only state the requirements in a literal sense, such as "the system TPS needs to reach more than 300, and the single transaction time should not exceed 3 seconds" etc. Performance testing engineers are required to decompose and extract explicit and implicit performance testing requirements based on user needs and the needs of the performance testing activities themselves.
With the rapid development of Internet technology, Internet application architecture is becoming more and more complex, and more and more stakeholders are involved in operating systems. Therefore, during the implementation of performance testing, it is necessary to analyze the needs to be tested from different user levels.
After determining the necessity of performance testing, the performance testing engineer mainly determines the performance testing requirements from the following two users:
business user
1. Business processes that are frequently used by users and used by a large number of users;
2. Business processes with a relatively high proportion of transactions and a daily proportion of more than 80% or even higher;
3. Business processes where special trading days or peak transactions account for more than 80% or even higher;
4. Poor performance and adjusted business processes;
5. Special business scenarios;
6. Business processes that undergo major process adjustments in core business.
From the business user level, the above considerations may require performance testing. During the actual implementation process, if possible, conduct surveys with end users.
project team
1. A business that has adjusted its architectural design after measuring performance;
2. Logically complex and critical business;
3. Businesses that may consume a lot of resources;
4. Businesses that have interface calls with external systems and interact with a large amount of data;
5. Call third-party business components and services with complex logic.
The above considerations from the perspective of project development may require performance testing business processes. Performance testing engineers need to have an in-depth understanding of the objects being tested and require the cooperation of the R&D team.
In addition to the above two types of users, it may also include the operations team to investigate future business development plans and the possibility that the system needs to meet future business needs.
Performance testing requirements review
After the performance testing requirements are determined, some level of testing requirements review activities need to be conducted, if necessary. Performance testing requirements review is similar to functional testing requirements review, both need to focus on the testability, consistency and correctness of the requirements themselves.
Measurable
Software testability is usually understood as whether the software itself has the conditions for testing and whether it is easy to detect and locate defects.
Within a certain time and cost range, by building a test environment, designing and executing test cases, test engineers can relatively easily discover and locate defects, thereby assisting R&D personnel in solving corresponding defects, whether it is functional testing or performance testing. The object under test has the above-mentioned testable characteristics.
A significant feature of performance testing activities and functional testing activities is that the running environment requirements of the objects under test are different. When implementing functional testing, as long as the object under test can run normally in a reasonable operating environment, even if the test environment and the production environment may be significantly different, performance testing is different and must simulate as real an operating environment as possible. When the test environment is significantly different from the actual production environment, the performance test results are often not accepted. If a relatively real test environment cannot be built during the performance test implementation process, it can be considered that the object being tested does not have performance testability.
consistency
Performance testing requires consistency, mainly focusing on user needs, production needs, and operational needs. Through the analysis of performance test requirements, determine whether the test requirements meet the performance requirements clearly listed in the user requirements specification. The production requirement is to pay attention to the authenticity of the operation of the object under test, so that relatively accurate data basis can be provided after the test is completed.
Operational requirements need to be based on historical data or current operational data to plan the possibility of future business development, so that the performance indicators of the measured objects have a certain degree of redundancy.
Through performance testing requirements review activities, it is determined that this performance requirement is consistent with the above concerns.
correctness
On the basis of ensuring testability and consistency, it is necessary to verify the performance test indicators to ensure the correctness of various project requirements, scenarios and indicators of concern in subsequent implementation activities, thereby minimizing the risk of rework and redesign. .
Through the evaluation of testability, consistency and correctness, the requirements for this round of performance testing are finally determined and used as input for subsequent test implementation activities.
Thank you to everyone who reads my article carefully. There is always a courtesy. Although it is not a very valuable thing, if you can use it, you can take it directly:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends, and this warehouse also accompanies Thousands of test engineers have gone through the most difficult journey, and I hope it can help you!Friends in need can click on the small card below to receive it
