foreword
1. The significance of standardizing the implementation process of performance testing
A standardized performance test implementation process can strengthen the control of the test workflow, clarify the work that should be completed at each stage of the performance test, guide the testers to carry out the performance test work correctly and orderly, and improve the work efficiency of each role in the performance test. The performance testing implementation process shared this time is a "guideline" for performance testing, and I hope to help you become a "master" of performance testing as soon as possible.
2. Performance testing implementation process
The performance testing process is divided into five stages, which are [requirement research stage] → [test preparation stage] → [test execution stage] → [test report stage] → [test summary stage].
What does each stage do? What to focus on?
1. Demand research stage
text
1.1. Phase overview
The main tasks in the research stage are: setting up a working group, project creation, demand analysis, model building, and customizing a detailed implementation plan for performance testing.
Focus on: demand research, needs analysis, model building
1.2. Description of key points
Demand research is divided into two steps: demand research and demand analysis.
This work is a necessary part of performance testing. The work output document is "XX Project Performance Test Requirement Form", such as: "Cloud Wisdom_XXX System_XXX Module Performance Test Requirement Form".
Model building at this stage is mainly business model building.
1.2.1 Demand research
Ø The requirement research work is led by the performance test implementer, and the product manager, development engineer, and operation and maintenance engineer cooperate to complete it;
Ø The main contents of demand research are:
n The performance requirements of the system online environment, such as performance requirements, reliability requirements, maintainability requirements, etc.;
n Other information related to system performance requirements, including system information (such as online environment hardware, parameter configuration, system architecture and deployment method, associated system deployment, etc.), business information (key business logic and processing flow, transaction list, transaction volume Information, business distribution rules, etc.), production problems, documentation, etc., and summarize the collected information to achieve an overall understanding of the business and technology of the system under test;
Ø The test task proposers such as the development project team, demand department, and operation and maintenance department should fill in the information such as "task information" and "test background" in the "Cloud Wisdom_XXX System_XXX Module Performance Test Requirement Form", and put forward the test requirements , if it cannot be explained in simple words, a file can be attached;
Ø The implementers of the performance testing team will fill in other contents obtained from the investigation into the "Cloud Wisdom_XXX System_XXX Module Performance Test Requirements Form";
Ø For a newly approved system or a new development version of the system, the "Cloud Wisdom_XXX System_XXX Module Performance Test Requirements Form" should be consistent with the performance requirements in the "Requirements Specification".
1.2.2 Demand Analysis
Ø The basic process of demand analysis is:
n First, the performance test engineer analyzes the information obtained from the demand survey, and converts the performance requirements in the "Cloud Wisdom_XXX System_XXX Module Performance Test Requirements Form" into specific performance requirement index values;
n Secondly, according to the difference analysis between the test environment and the online environment, the performance test engineer converts the performance requirement index value under the online environment condition into the performance requirement index value under the test environment condition;
For example: TPS (Transaction per Second): the number of transactions processed by the system per second, the derivation process is as follows,
The current online APP1.0 trial system is mainly for query transactions, accounting for 40% of transactions. The statistics of system production transaction volume are about 200,000 transactions per month. Assume that after the APP2.0 system is launched, the business volume will surge to 200,000 queries per day. Then the total daily trading volume T reaches:
T = 20W/40%=500000 pens/day
System processing capacity TPS derivation: After APP2.0 goes online, the maximum transaction volume is 500,000 transactions/day, and there is almost no transaction volume in the system at night. According to the 2:8 principle, then (500,000 80%)/(8 20% 3600)= 69.4 transactions / Seconds, rounded to 70 transactions/second, calculated on the basis of 50% increase in business volume per year, then the system processing capacity index after one year is approximately equal to 70+70 50%=105 transactions/second.
Derivation of stable trading volume: Take 60% of the system processing capacity_duration = 105 transactions/second * 60%*8_3600=1814400 transactions.
After analysis, it is summarized into test index values
Ø The main content and normative requirements of demand analysis are as follows:
n Performance test requirements: The performance test index items and demand index values should be accurately described.
n System scope: It is necessary to accurately describe the test scope information on which the performance test demand index value is based, such as the logical schematic diagram of the related system of the test scope and the information of each related system; Test scope, detailing the relevant subsystems of the system under test.
n Environmental difference analysis: It is necessary to accurately describe the test environment information on which the performance test demand index value is based, such as the overall network topology diagram of the test environment, the test environment machine configuration table (number, model, resource, operating system), and the corresponding software configuration, important parameter configuration, etc. At the same time, the above information of the online environment should be accurately described, and a detailed analysis of environmental differences should be carried out.
The above analysis will be an important part of the performance testing program.
1.2.3 Model building such as: business model
According to the business volume statistics of XX, XX, XX, 200X, the peak day of business between XX, XX, 200X, and XX, XX, 200X, the following business model is obtained after slight adjustments, requiring business model transactions to account for at least 90% of the online transaction volume % above:
2. Test preparation stage
2.1 Phase overview
The test preparation stage is an important stage in the performance testing work. In the preparation stage, it is necessary to complete the construction of the business model to the test model, the preparation of the performance test implementation plan, the preparation of the test environment, the design of the performance test case, the design of the performance test monitoring plan, the performance test script, and the preparation of related test data, and in the above After the completion of relevant preparatory activities, the admission inspection shall be carried out according to the test plan.
Focus on: test model construction, scheme design, case design, data preparation, etc.
2.2 Description of key points
2.2.1 Test model construction
The test model construction work is completed by the performance test implementers;
On the basis of demand analysis, analyze and sort out the relevant materials and information collected in the survey, focusing on the analysis of cross-system transaction paths, transaction correlations, data processing and circulation, business volume, transaction ratios, typical transactions, and system requirements. For performance test points such as processing capacity, multiple business scenarios are determined in a targeted manner, and a set of specific business transaction sets are selected for each scenario, and corresponding test models are built according to the proportion of business volume.
The output of this stage is each test scenario, as well as the typical transactions and their proportions in the scenarios.
Example: Derivation from Business Model to Test Model
According to the business model, through communication with the project team and product manager, it is determined that the test model needs to focus on the following:
(1) Considering that the future securities system database upgrade may affect the historical query, so this test adds a separate scene: 50% of historical entrustment and historical transaction query (ie 0456 and 0457). At the same time, considering that most of the transactions in the online environment are initiated by the main center, the transactions of "history entrustment and historical transaction query" in this test are all initiated from the main center;
(2) Increase the transaction scenario of treasury bond issuance. The subscription day of treasury bond issuance is generally carried out before the counter opens. In this scenario, only one transaction of treasury bond issuance and subscription is selected;
(3) At the same time, during the peak trading hours of the securities system, teller sign-in and teller sign-out transactions accounted for a relatively small proportion.
Through the above analysis, it is concluded that there are three test models: the general trading day day model, the national bond issuance day model, and the historical query transaction model.
General trading day intraday model:
Trading model of savings treasury bonds:
Historical query transaction model:
2.2.2 Scheme design
Compiling the performance test implementation plan is a necessary part of the performance test work, and its output is "Performance Test Plan", such as: "Cloud Wisdom_XXX Project_XXX Function Module_Performance Test Plan V1.0.xlsb".
It needs to be described in the plan: test requirements, start-stop criteria, test model design, test strategy, test content, test environment and tool requirements, and output documents of each stage. In the plan, it is also necessary to explain the time plan of the performance testing work, the expected risks and risk avoidance methods, etc. The content of the test model design comes from the test scenarios formed in the test model design at this stage, as well as the typical transactions and their proportions in the scenarios.
2.2.3 Case Design
In case design, it includes case description, test environment description (hardware, software, application version, test data), delay settings, stress scenarios, execution description, expected results, and monitoring points.
Case design is an essential part of performance testing, and the output document of case design is "Performance Test Cases".
2.2.4 Data preparation
Environment preparation involves the preparation of basic data. The quantity and logical relationship of test data are very strict, and the preparation of test basic data generally adopts self-made simulated data or desensitized online data.
2.2.5 Test script development
The test script development work is the time to play LR.
A test script is a programmatic representation of business operations, and a script is generally a description of a business process. This activity is mainly for the recording (writing), modification and debugging of scripts, so as to ensure that the scripts of each test case can be executed correctly under the conditions of a single transaction and a small number of iterations before the test is implemented. The general steps for test script development are as follows:
Ø Complete script code generation by recording or writing. When the code is generated, the transaction is mainly inserted according to the requirements, and it is used as the unit for counting the transaction response time during the test;
Ø According to the test requirements, carry out parameterized settings;
Ø Set the checkpoint, and judge whether the transaction is executed correctly according to the message content field, that is, the checkpoint is set at the application level;
Ø Determine whether to set up a collection point according to the test requirements;
3. Test Execution Phase
3.1 Phase overview
The test execution stage is the stage of executing test cases, obtaining system processing capability index data, and discovering performance test defects. During test execution, use test tools to execute test scenarios or test scripts, and cooperate with various monitoring tools. After the execution is completed, various result data are collected uniformly for analysis. According to the needs, system tuning and regression testing can be carried out in the execution phase.
Focus on: result recording, test monitoring, result analysis
3.2 Description of key points
3.2.1 Test Execution and Results Recording
There is a corresponding priority policy in the test execution process. According to the priority level of the test cases, the test cases with higher levels are given priority. During the test, it is determined whether to repeat the current case or execute a new test case through the analysis of each test result; usually if the bottleneck problem is found, it will be adjusted immediately and the test case will be re-executed until the current case passes.
In the execution phase, the execution of the test, analysis and optimization, and regression testing are repeated, and the entire execution process and execution results must be carefully recorded. The execution result data is the main basis for analyzing bottlenecks.
3.2.2 Test Monitoring
The monitoring work of the test is carried out synchronously with the execution work. When the scene or script starts to execute, the monitoring program (you can use nmon or system commands top/vmstat/iostat, etc.) , monitoring treasure can monitor the performance indicators of IT infrastructure such as website/webpage performance/Ping/DNS/FTP/UDP/TCP/SMTP, and perspective treasure can find performance bottlenecks of host resources, web applications, browsers, APP and other applications, as follows As shown in the figure:
Monitoring treasure monitoring page
Perspective treasure host resource monitoring page
After the execution ends, stop the test monitoring and extract the monitoring result data.
3.2.3 Analysis of test results
During the test process, comprehensively analyze the test problems that appear based on the results displayed by the front-end performance test tools and the monitoring results.
For example:
When the test group performed the "general day-to-day transaction model" load test with a pressure of 570TPS, the database monitoring found that there was a deadlock imagination, as follows:
**Problem analysis:** After analyzing with the development, the reason is as follows: when the flow control information collection program (pltflowGthDaemon) conducts transactions concurrently at the same teller at the millisecond level, a deadlock occurs in the plt_flowgather table. Online transactions in the test environment are initiated using the same teller number, so the probability of occurrence is relatively high.
4. Test report stage
4.1 Phase overview
After the test execution work is over, start writing the performance test report. Performance test reports need to be reviewed before release.
4.2 Description of key points
4.2.1 Report writing
The main contents of the performance test report include: test purpose, scope and method, environment description, test result description, result analysis, conclusion and suggestion, etc.
4.2.2 Description of test results
The description of test results should reflect the execution process of the performance test. For example, in the display of capacity test results in mixed scenarios, it is necessary to describe the test results and monitoring results under each concurrent gradient; in the result records in digital form, it is required to be accurate to 3 digits after the decimal point effective number.
4.2.3 Test defects and problems
In the performance test analysis report, the defects and problems found in the test process must be described, and the risk assessment should be carried out for the items confirmed to be test defects, and risk warnings should be given.
4.2.4 Final result analysis
The analysis of the final results of the test, this part should be comprehensive, thorough, easy to understand and more intuitive expressed through diagrams.
For example:
4.2.5 Test conclusion
The test conclusion is the content that must be included in the performance test analysis report. The conclusion of the test must clearly and accurately answer the various indicators described in the performance test requirements, and must fully cover the test requirements.
5. Test summary stage
5.1 Phase overview
The summary work of the performance test mainly summarizes the test process and test technology of the task. The performance testing work has entered the summary stage, which also means that the performance testing work is coming to an end. At this stage, all critical test assets should be archived as time permits.
Finally: The complete software testing video tutorial below has been organized and uploaded, and friends who need it can get it by themselves [Guaranteed 100% free]
Software Testing Interview Documentation
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Ali, Tencent, and Byte, and some Byte bosses have given authoritative answers. Finish this set The interview materials believe that everyone can find a satisfactory job.
