Performance testing pain points
In the past, performance testing was usually developed through self-testing or implemented in a project demand-driven manner, that is, verifying the corresponding performance goals in the test environment based on requirements, and issuing a performance acceptance report. However, as the iterative speed of business systems continues to accelerate, this approach will also have many shortcomings:
First of all, the test results obtained in the test environment can verify program-level problems, but due to differences in environment and data, it is impossible to verify or obtain the performance indicators of the business system in the production environment.
Second, as the frequency of business requirements changes continues to increase, the release cycle is shortened, and many emergency projects skip performance testing and go to production. It also created a misunderstanding in the industry: function must be tested, performance can be untested. For example, changing the number of database connections is a counterexample.
Third, according to most companies I have learned, for the safety of production systems, the daily water level is very low. For example, the CPU utilization rate is less than 10%, and the peak period may reach 20%. The low resource utilization rate is also caused by the uncertainty of production capacity.
In addition, with the practice of project requirements-driven testing, the test results will be data islands, it is difficult to achieve sustainable performance baseline tracking and risk identification, and it is easy to cause cumulative avalanche problems.
Performance testing mature stage
In the past, the industry did not have an evaluation model for the maturity of performance evaluation. Based on past industry practical experience, it can be roughly positioned into five stages.
The first stage focuses on development and self-testing. Most start-up companies do not have an independent performance testing team. After the system is developed, developers use open source tools to perform stress testing on core interfaces, and lack professional testing methods.
The second stage is based on project requirements. The test team designs test scenarios based on project requirements to evaluate the system under test. Most of the test requirements and test goals rely on manual reviews.
The third stage is based on the waterfall model, with an independent performance testing team, and formulating unified performance access/access standards and specifications, promoting the entire IT department through an organizational form, and implementing a standardized performance testing and acceptance process.
In the fourth stage, turn passive to active, use a platform-based approach to form a complete performance regression system for key business changes, open up continuous integration, establish a sustainable performance baseline tracking mechanism based on iterative performance evaluation data, and identify frequent iterative changes. Hidden performance hazards.
The fifth stage is to test left and right empowerment, left empowerment for research and development, identify potential performance risks in advance, right empowerment for operation and maintenance, and provide effective guarantees for production capacity and stability.
Misunderstanding of pressure test
It is worth mentioning that: With the production pressure test, it does not mean that the test environment pressure test can not be done.
