Table of contents
4. Pressure testing machine management
Nine, configuration management
Organization
Here, I will introduce the idea of the performance test management platform according to the matrix organization structure in which the project team belonging to each different system is horizontal, and the performance test team is a vertical matrix organizational structure as a functional department.
mind Mapping
A must-see video tutorial for performance testing: 2023's latest detailed explanation of the entire process of real enterprise performance testing, the kind that can be included in resume interviews_哔哩哔哩_bilibili 
https://www.bilibili.com/video /BV1PW4y1R7ye/?spm_id_from=333.999.0.0
1. Task management
1. Task application
Generally speaking, there are two sources of performance testing requirements:
①. The project team raises the demand
The project team actively raises performance testing requirements and requires a unified performance testing task management module, which includes the project line to which the system under test belongs, system name, system architecture diagram, network topology diagram, related design documents and related environment configuration information ,
And the contact information of the project manager, development, operation and maintenance, DB, etc., as well as the delivery test time of the system under test, deadline time and other information.
This situation can be divided into three types:
New system release : When a new system is released and launched, it is necessary to conduct a complete test on functions, performance, security and other aspects to evaluate whether it meets the established online requirements of business and products.
Iteration of the old system : Some optimization of the existing system, the addition of new functions or the introduction of new business channels may bring higher traffic impact. At this time, the project manager or development manager will put forward relevant performance requirements, hoping to verify the existing Whether the system meets the online requirements.
Production accident repair verification : The system encountered performance problems in the production environment and caused some losses. After tuning or repairing, a round of comprehensive performance testing is required to evaluate whether it meets the existing actual business needs.
②, performance group requirements
Proactively propose the possible performance bottlenecks brought about by the iteration of the project and the introduction of new requirements, and then after evaluation, decide whether to conduct tests to evaluate the stability and usability of the system.
2. Task approval
After the performance test task application is submitted, the project team, performance team, and even other relevant personnel need to conduct a comprehensive evaluation based on the existing situation, work arrangement, and construction period to decide whether to conduct performance testing, when to start, and resource allocation.
It needs to involve the cooperation and participation of multiple teams and multiple personnel, as well as the risk estimation caused by the failure to deliver on time. Regarding the performance test requirements review, I will write a blog post to analyze some of the details.
3. Task scheduling
After the performance test task is evaluated, it is decided to proceed, and the next step is to carry out further work such as work arrangement, resource allocation, and work scheduling.
2. Use case management
The use case here refers to the abstract management of the business model established based on the task type, resources and other aspects in the performance test. Specifically, it can be divided into the following three business models:
1. Regular tasks
Routine tasks refer to the performance requirements proposed by system iterations or new system releases, including specific information such as project lines, system names, architectures, topology diagrams, relevant personnel information, and business models.
Carry out modeling and analysis of the specific system under test scenario according to the above situation, and then formulate specific test cases.
2. Daily polling
This type can refer to the automatic execution and conditional triggering in continuous integration, and set up scheduled tasks to perform performance tests on the systems within the scope. The test types mainly include concurrent, multi-node and other test types.
3. Full link pressure test
Full-link stress testing is mainly the performance testing of the overall business line in the production environment.
3. Environmental management
The prerequisite for performance testing is to have a stable environment that can satisfy performance testing. Generally speaking, it is carried out in the following two environments:
1、UAT
The UAT environment is what we commonly call the user acceptance test environment. The environment is relatively stable, and the configuration is the same as that of production or can be replaced by the same amount, which can meet the needs of conventional performance testing.
2、FAT
The FAT environment can be understood as a production verification test environment. The system version, configuration, data volume, etc. are consistent with production, which is more in line with the actual production situation in terms of testability and authenticity.
PS: There is also a full-link stress test that is carried out in a real production environment, but the risk of performance testing in the production environment is too high, and there are many transformations to the existing system. Whether to carry out a detailed evaluation can only be determined.
The challenges of full-link stress testing lie in the following points:
①. Business model sorting
②. Data issues, including the true availability of data, data isolation and data desensitization;
③. The environment is isolated and cannot affect the actual production business;
④. Service cluster load balancing, test strategy and communication between services;
⑤. Disaster recovery issues, including when certain services are down or unavailable, the actual production business operation cannot be affected, system availability, etc.;
4. Pressure testing machine management
1. Pressure testing machine scheduling
When the actual flow impact is high, a single press may not be able to support it. In this case, multiple pressure actuators are needed for distributed pressure measurement.
In the actual production environment, traffic changes are likely to be random. How to increase and decrease the stress test execution machine and make reasonable use of resources is a problem that needs to be considered and solved.
2. State management
This mainly includes the status changes of the stress testing machine, including idleness, use (even predicting when it can be released for other stress testing tasks, etc.), unavailability (damage or other reasons), etc.
3. Exception management
When the performance test is in progress, if the service is unavailable due to some reasons, it is necessary to stop the performance pressure test in time. Generally speaking, the following methods are mainly used:
①. Manual stop: stop the pressure test from the function button on the management interface;
②, fuse measures: that is, through monitoring and alarm measures, when the system is unavailable or exceeds a certain threshold, the pressure test will be automatically stopped;
③. The bottom line method: When manual stop or automatic fuse measures are not available, some external means are used to stop the pressure test execution, which is also a disaster recovery measure;
A must-see video tutorial for performance testing: 2023's latest detailed explanation of the entire process of real enterprise performance testing, the kind that can be included in resume interviews_哔哩哔哩_bilibili https://www.bilibili.com/video /BV1PW4y1R7ye/?spm_id_from=333.999.0.0
5. Data Management
Tests are driven by data, so what data do you need to prepare for performance testing?
1. Basic data
The basic data includes the data necessary for the normal operation of the system business, such as: SKU information, inventory data, etc. of the e-commerce platform, as well as user information of the bank, relevant data of certain businesses, etc., which can be solved by means of backup from production.
2. Embedded data
Embedded data mainly refers to the DB level, that is, performance testing needs to simulate the actual environment, including data volume, etc. From the DB level, the same database table has different data volumes under the same business model, and its performance is also different of.
The preparation of embedded data can be done by backing up from production, or customizing and preparing some usable data through scripts and SQL statements.
3. Test data
The data necessary for the test script to run can usually be resolved through parameterization.
Of course, from the perspective of the test platform, the solution to this problem can be managed through a unified data pool, and the interface uses different options to call APIs to generate data available for testing.
6. Monitoring and management
In performance testing, monitoring is an essential and important part. Generally speaking, the following aspects need to be monitored:
1. Front-end performance
The time spent on front-end display and resource rendering, and which resources consume the most time and resources, all need to be monitored to obtain relevant information.
2. Middleware
①、Redis
Some system architectures use Redis as a persistence layer or buffer, so information such as cache availability, cache failure, and cache penetration must also be monitored.
②、MQ
MQ is an asynchronous communication framework, and there are similar frameworks such as Kafka. It is also essential to monitor the production and consumption rates, resource occupation, and possible congestion of message queues.
③, other
3. Container
Now more and more enterprises are beginning to containerize services (especially in the microservice architecture, containerization is a necessary means), including environments, scripts, and services are placed in containers.
Then the monitoring of container resources is also very necessary.
4. Server
Server monitoring is mainly about memory, CPU, IO, etc., including proportion, usage, threshold, and reminders.
5、DB
At the database level, it mainly monitors information such as memory and CPU, and of course, it also includes time-consuming SQL statement execution and locks.
6. Network
The network is an essential part. An unstable network environment has a great impact on the test results and may cause some hidden problems; network monitoring generally focuses on these aspects: stability, gateway, CDN, firewall, etc. .
PS: In fact, judging from the above monitoring, they are all monitored layer by layer from the user layer to the final service processing layer, so that everything can be spoken with data.
Seven, log management
During the test process, sometimes hidden problems cannot be seen intuitively from the test data, so it is necessary to check the corresponding logs, analyze the nodes where the problem broke out from the detailed logs, and then conduct targeted analysis. Log monitoring can be roughly divided into the following levels:
1、web
This can be understood as user-level logs, including information such as local loading and caching of resources (in some cases, the detailed log package is large, and the log is written to the user-level operation).
2、app
This refers to the application layer. Sometimes the cause of performance problems occurs at the application layer, so the monitoring and log analysis of the application layer are also very important.
3、service
The background service layer, the log includes which operations are processed, which request is even called, etc.
4、DB
For database logs, similarly, which operations take a long time and have a lot of resources, etc., all require corresponding monitoring and necessary log analysis to identify problems.
PS: From the perspective of the performance testing platform, the above-mentioned log management methods all hope to display and filter the logs more intuitively. Otherwise, we need to use commands or other tools to find and analyze logs at specific levels. This will undoubtedly waste some time.
8. Report management
Report management mainly includes the following aspects:
1. Real-time results
The real-time monitoring results of the test are stored in the database, and then displayed on the interface through grafana and other tools to manage the test results more intuitively.
2. Test report
At the end of each round of testing, it is necessary to conduct statistical analysis on the test results, so as to provide a reference and evaluation basis for the next test; the test report interface can be displayed in a custom style.
3. Performance baseline
The performance baseline here refers to: use the final result of each performance test as a performance reference baseline, and for each subsequent iteration, take the last performance test result as the evaluation point, and then continuously update the performance baseline as the basis for the next evaluation .
It can be displayed in various ways such as data line, tree diagram, etc. The purpose is to monitor the long-term system stability and availability, and provide multiple dimensions of reference for system tuning or reconstruction.
Nine, configuration management
1. Baffle
Generally speaking, performance testing is to simulate concurrency and test by calling different service interfaces, but sometimes due to some reasons, some services are temporarily unavailable, or the number of times is limited. In this case, mock services are urgently needed .
①、HttpMock
HTTP is the most common protocol, and with the popularity of microservices, the application of restful architecture design, HttpMock's baffle service has become a common need.
②、SocketMock
The TCP connection is also a way for some business system services to communicate. In order to facilitate performance testing and reduce the dependence on certain other services, the SocketMock baffle service is also essential.
③, other
Other types of interface calls, including dubbo interface, can be designed in a targeted manner to turn mock into a service.
PS: From the perspective of the test management platform, the Mock object programming service is managed through the interface for adding, deleting, modifying and checking, which is more convenient for management and intuitive understanding.
2. Container
①、docker
Containerization is a trend that has become more and more popular in recent years, and docker container management for different test environments is also essential.
②, virtual machine
Limited to the situation of different technical architectures of different projects, the item of virtual machine management can be designed according to specific needs. If all containers are containerized, then there is no need for virtual machine management.
10. System Management
1. User management
Including user registration, addition, deletion, status management and other functions of using this system, as well as possible task assignment and other operations.
2. Authority management
Different user roles are assigned different system access rights, and this module can be designed using the principle of single sign-on.
3. Group management
The group management here, I personally understand it as functional groups divided based on different identities and roles, and manage functions such as adding new functions, assigning rights and functions, and changing status.
A must-see video tutorial for performance testing: 2023's latest detailed explanation of the entire process of real enterprise performance testing, the kind that can be included in resume interviews_哔哩哔哩_bilibili https://www.bilibili.com/video /BV1PW4y1R7ye/?spm_id_from=333.999.0.0
