Table of contents
foreword
Recently, some students asked me about performance testing plans. The reason was that their company was going to do performance testing. The leader asked to write a performance testing plan. He had no way to start because he had not done related work before.
This blog, combined with some of my personal experience and summary, talks about how to formulate a more comprehensive performance testing plan.
1. Test background
First of all, the background of this performance test should be explained, that is, the type of the system under test, which users it is aimed at, what characteristics it has, why the performance test is carried out, some expected indicators, and so on.
For example: In order to ensure the stable operation of the system and high availability of the business during the "Double Eleven" promotion period, perform performance tests.
Core: Evaluate system performance, analyze performance change trends, locate system bottleneck risks, and assist in planning system capacity.
2. Purpose of test
The purpose of the test should be analyzed and set according to the test background, such as:
1. Some applications of online services are suspended due to high traffic, so the purpose of the test is to locate bottlenecks, analyze, optimize and verify;
2. The operation has done pull-in and new channel expansion, so the purpose of the test is to evaluate whether the system performance meets the new online business;
3. The system architecture is changed from cluster technology to microservices. The purpose of the test is to verify stability, availability, and single instance capacity, and provide capacity planning data for online service expansion;
A must-see video tutorial for performance test engineers: 2023's latest detailed explanation of the entire process of real enterprise performance test projects, the kind that can be written into resume interviews_哔哩哔哩_bilibili 
https://www.bilibili.com/ video/BV1PW4y1R7ye/?spm_id_from=333.999.0.0
3. Test scope
Through demand research, analyze user usage scenarios, quantitatively analyze business data volume growth trends and peak active users, and determine the application scope of the system under test, such as order + shopping cart + product + payment service.
| business attribution module | business related scene |
| Order | Create Order |
| cancel order | |
| shopping cart | add to cart |
| delete cart | |
| merchandise | product list |
| product details | |
| to pay | balance payment |
| Pay with Ali-Pay | |
| WeChat payment |
4. Terminology Conventions
The terminology here refers to: some technical terminology related to this performance test, the purpose is to unify the caliber, explain and explain, and facilitate the understanding of relevant personnel participating in this performance test. Common terms are as follows:
| term name | Glossary |
| concurrency | The number of requests initiated by the simulated client per unit time (S) |
| stability | Verify the performance of the system under long-term (24h/48h) load conditions |
| high availability | Verify whether the system can provide services normally and the service recovery rate after a part of the service is down |
| TPS | The number of transactions per second, that is, the request processing capacity of the system per unit time (S) |
| RT | Response time, and the time it takes for the system to process a request |
| request success rate | During the test, the system successfully processed the request as a percentage of the total number of requests |
PS : The term agreement is subject to the actual situation, and the audience of the performance test plan must also consider the degree of understanding of performance testing. This agreement aims to unify the description and reduce communication costs.
5. Environmental Description
Generally speaking, the environment for performance testing is in UAT or an independent performance testing environment. However, in order to accurately describe the type and configuration of the environment, as well as the difference between the test environment and the production environment, it is recommended to compare the production environment and the test environment.
1. Production environment
①, system architecture
PS : The picture above is just a simple microservice type system architecture, just for illustration and understanding.
②, service configuration
| service name | quantity | configuration | Remark |
| gateway server | 10 | 2C3G | Gateway service, authentication and request forwarding |
| web server | 3 | 2C3G | |
| app server | 5 | 2C3G | |
| Redis | 3 | 4G | Sentry mode, one master and two slaves |
| DB | 2 | 8C16G | one master one slave |
2. Test environment
①, system architecture
②, service configuration
| service name | quantity | configuration | Remark |
| gateway server | 5 | 2C3G | Gateway service, authentication and request forwarding |
| web server | 2 | 2C3G | |
| app server | 2 | 2C3G | |
| Redis | 2 | 4G | Sentinel mode, one master and one slave |
| DB | 2 | 8C16G | one master one slave |
3. Load machine configuration
The load machine (machine) is the machine that simulates the request sent by the client. In general, it is enough to explain the hardware configuration, quantity, IP, and pressure sending strategy of the load machine.
4. Network
That is, the network status of the load machine and the performance test environment, such as the domestic public network/intranet of the same VPC, firewall policies, etc.
6. Demand analysis
Demand analysis generally has these stages: demand access → demand collection → demand analysis, the main contents are as follows:
1. Business model
The following is a typical e-commerce platform core business link diagram, involving modules such as login, home page, product, shopping cart, payment, and sharing.
2. Performance indicators
The performance indicators here include the following two items:
①. Business performance indicators
That is, the expected TPS, RT, and request success rate (generally, the default request success rate is ≥99.99%).
②, hardware performance indicators
That is, the server-side resource consumption indicators. Conventional resource monitoring indicators include: CPU usage, Memory usage, system IO, network IO, etc.
A must-see video tutorial for performance test engineers: 2023's latest detailed explanation of the entire process of real enterprise performance test projects, the kind that can be written into resume interviews_哔哩哔哩_bilibili https://www.bilibili.com/ video/BV1PW4y1R7ye/?spm_id_from=333.999.0.0
7. Testing strategy
The test strategy used in this performance test, such as:
To detect the inflection point of system performance, a stepped pressure test is required;
To detect the maximum processing capacity of the system under acceptable performance indicators, load and capacity testing strategies need to be adopted;
To verify the stability and high availability of the system, it is necessary to adopt a stability and high availability test strategy;
To verify the performance of the system under different configurations, a configuration test strategy is generally adopted;
1. Test strategies and scenarios
①, capacity test
| scene name |
01_login 02_Home |
| execution time |
10min |
| Business ratio |
100% |
| testing strategy |
capacity test |
| Testing purposes |
Continuously increase the load to verify the maximum performance of a single node of the system |
② Stability test
| scene name |
mixed scene |
| execution time |
24 hours |
| Business ratio |
According to the ratio of production and business, proportional scaling |
| testing strategy |
stability test |
| Testing purposes |
Verify the long-term stability of the system and whether there are problems such as OOM |
PS : The above table description is still used as an example to illustrate, the main content includes: scenario number, test type, involved business scenarios, business ratio, execution time, and test purpose.
2. Test monitoring strategy
| monitoring object | Index range | monitoring tool |
| CPU | ≤90% | nmon、zabbix |
| Memory | ≤70% | |
| Network I/O | No obvious IO bottleneck | |
| JVM | No frequent FGC cases | jmap、jstat |
8. Preparation
The preparatory work mainly includes the following items:
| Preparations | preparation content | Responsible | estimated finish time |
| Tool preparation | Load tools, monitoring tools, analysis tools | Test/O&M | 0.5 working days |
| script preparation | test script | test | 0.5 working days |
| Environmental preparation | Machine configuration, service deployment joint debugging, script debugging | O&M/Development | 1 working day |
| data preparation | Basic data, test data, parameterized data, cached data | DBA/Development/Test | 1 working day |
9. Organizational structure
The organizational structure refers to the members of each role of the team involved in this performance test, mainly including these: PM role, testing, development, operation and maintenance, DBA, network, and infrastructure. Example:
10. Risk Analysis
List the risk items and countermeasures that will affect the performance testing work before the implementation, such as:
| risk type | risk description | risk level | Solution |
| delivery risk | Serious functional defects found in the UAT stage | high | The test time is postponed, or the corresponding personnel are added |
| change risk | Temporary demand change, new demand | high | Whether the requirements review is included in the scope of this test, and delivered in stages |
| data risk | Data desensitization takes a long time and test data is unavailable | middle | Re-draft data preparation plan, small-scale data verification |
| environmental risk | Deployment problems, joint debugging progress is slow, network bandwidth bottleneck | middle | Change environment, increase resource configuration, expand bandwidth |
11. Delivery list
In the performance test plan, it is necessary to explain the deliverables of each stage of this performance test, mainly including these items: performance test plan & scheme, test script, performance defect statistics, round subsections, and performance test report.
12. Phase progress
This mainly refers to the work progress and resource arrangement of each stage from the demand stage to the end. It is recommended to use the Kanban method to update the progress in time to facilitate the development of the work. Examples are as follows:
| stage |
matter |
Starting time |
End Time |
state |
Responsible |
| demand stage |
needs review |
Finish |
Multi-party participation |
||
| System Architecture Diagram |
Finish |
to develop |
|||
| requesting research |
Finish |
performance tester |
|||
| Preparation Phase |
environment delivery |
Finish |
Operation and maintenance, development |
||
| application deployment |
Finish |
Operation and maintenance, development |
|||
| data preparation |
Finish |
Development, DBA, Testing |
|||
| script development |
Finish |
performance tester |
|||
| Implementation phase |
Execute pressure test |
undone |
performance tester |
||
| service monitoring |
undone |
Operation and maintenance, testing |
|||
| data collection |
undone |
performance tester |
|||
| Finish |
report review |
undone |
Multiple review |
The above is a relatively complete performance test plan. Of course, after completing the performance test plan and passing the review, you can enter the test execution stage.
A must-see video tutorial for performance test engineers: 2023's latest detailed explanation of the entire process of real enterprise performance test projects, the kind that can be written into resume interviews_哔哩哔哩_bilibili https://www.bilibili.com/ video/BV1PW4y1R7ye/?spm_id_from=333.999.0.0
