A classmate sent me a private message and chatted with him about some topics about performance testing, and found that his understanding of performance testing had gone into some misunderstandings.
In some technical exchange groups, I also encountered various problems caused by many students due to misunderstandings about performance testing, such as:
The number of registered users = the number of concurrency, and then the service is directly broken;
· Pressure test directly in the production environment: production services are down, and customers complain;
Of course, these are relatively basic questions, and students who are just getting started may make such mistakes. If you have some practical project experience, you will understand that performance testing is much more complicated than we thought. In addition to certain requirements for the breadth and depth of technology, there are also certain requirements for familiarity with business, ability to analyze and understand requirements and scenarios, and even communication and coordination ability during stress testing implementation.
In this article, I will talk about topics related to performance testing, its implementation purpose, the focus in different scenarios, and some neglected points.
purpose of performance testing
First of all, we must realize that, regardless of the technology stack involved in performance testing, the essence of performance testing is no different from functional testing . Requirements analysis , scenario design, preparation of test cases and test data are also required . Functional testing is to manually execute use cases and observe the results, while performance testing is mostly to use tools or scripts to execute test cases and observe the results.
The purpose of functional testing is to verify the correctness of the product design and find bugs that are inconsistent with the design; performance testing is to find bottlenecks in the processing capabilities of application services, and then perform targeted optimization to ensure online capacity planning and service stability. sex provides support. So here comes the question: how to define the so-called processing power bottleneck? This requires anchoring business value.
The requirements for performance testing basically come from the business. For example, users report that the APP response is too slow, the financial or cost department reports that the hardware cost of IT is too high, or the operating activities fail to achieve business goals due to system failure. In terms of classification, these problems are the pain point appeals of users and businesses:
· APP response is too slow: increase processing speed - reduce response time (RT);
· Hardware cost is too high: reduce hardware cost——improve the processing power per unit resource (TPS);
· Business goals not achieved: Improve system stability - improve business success rate (99%-99.99%);
To sum it up: reduce costs, improve user experience, and ensure the achievement of business goals. This is the so-called business value!
The ultimate purpose of performance testing is essentially the same as that of functional testing, which is to provide users with correct and stable services and a good user experience to ensure the achievement of business goals. A series of technical solutions adopted to meet the needs of users and businesses are just means to achieve this goal.
Focus of different projects
After talking about the purpose of performance testing, then return to the specific project practice. The most common types of projects in daily work can be roughly divided into the following categories:
version iteration
Version iteration is the daily work of software engineers. In this type of project, the main focus of performance testing is on the processing capability of the system. That is to verify whether the system's processing capability has declined due to requirements iteration & new code introduction. The main indicators of concern are TPS&99RT&request success rate and other aspects. From the perspective of system construction, the long-term performance quality of the system can be evaluated by establishing a performance baseline.
configuration changes
We all know that many online faults are caused by changes, but in fact, many times, performance changes may be caused by a small parameter change. In terms of subdivision, one of the performance test scenarios is called configuration test, which is to verify the performance changes caused by changes in system parameters or service configurations. There are two common ones:
Changes in software parameters: such as the number of thread pool connections, timeout, etc.;
· Hardware configuration changes: such as the comparison of performance changes caused by server upgrades and downgrades;
The performance changes brought about by such configuration changes are more concerned with the middleware and basic service levels, because such changes are often easily overlooked, but such changes will have a great impact on the performance and stability of online services. Influence.
New service launched
In addition to the daily version iterations, it is relatively common for new services to be launched. For example, technical transformation, service splitting, introduction of new service providers, etc., generally require performance testing to verify whether it will affect the existing system.
The main purpose of performance testing for new services is to verify the robustness of the system, that is, to find some obvious performance problems, such as: memory leaks, business oversold, deadlock, slow SQL , etc.
Stability Guarantee
Most of the performance tests are carried out in the offline environment, but the results of the performance tests must provide support for the online capacity planning and stability guarantee, otherwise the performance tests are of little value.
Although it is already 2023, and it has been almost 11 years since the production full-link stress test was proposed, so far most companies still do not have the ability to perform performance tests in the production environment, and there are many reasons for this. For example, the cost of stress testing in the production environment is high and the risk is high. For example, most companies do not have a high number of concurrent visits. For example, the technical construction and reserves are not deep enough. The root cause is actually the balance between input and output.
Of course, how technology creates business value is a very complicated issue, but there is a misunderstanding about full-link stress testing, which is also overlooked by many people.
Production full-link stress testing is only suitable for certain companies with specific business needs, and whether it can be implemented depends on whether there are suitable organizational management capabilities and corresponding technical architectures. Production full-link stress testing is not a silver bullet, nor is it just a technical means of testing. If production full-link stress testing is regarded as a technical practice to promote the stability of production services, then it has a lot to explore value point. But in the actual implementation process, it can only be said that everyone seems to have fallen into a misunderstanding about the understanding of technology and the perception of business value.
Misunderstandings in scene modeling
Students often ask: Can I use a user's data to repeat the pressure test, anyway, it is also a concurrent request.
In functional testing, we will prepare test data corresponding to the corresponding scenarios according to the scenarios and test cases to be tested. Why is it ignored during performance testing? This is actually a cognitive misunderstanding: performance testing is to simulate high concurrency and send requests to the system.
The correct approach is similar to functional testing, establish the mapping relationship between business model & traffic model & data model, prepare corresponding test data that conforms to the test scenario, and ensure that the amount of data is sufficient for testing.
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey, and I hope it can help you! Partners can click the small card below to receive