The whole discussion will be very long in general, from automated thinking, models, tools, to automated testing technologies, testing frameworks, and testing platforms at all levels, including future-oriented automation technologies. part to write. In addition, due to the wide scope involved, some of the content is still somewhat in-depth. I try to describe the context completely so that everyone can understand it. If there is any unclear or wrong (knowledge is always limited), please correct me.
01. Automated testing thinking
Readers who are familiar with me should know that I am used to talking about thinking before writing specific applications. Because I think the first thing to grasp about anything is its essence. After understanding the essence, it will be much easier to look at the phenomenon. It can be said this way: first-rate tests are used as thinking, second-rate tests are used as models, and third-rate tests are used as tools (the "test" here is a verb, not a person, just pat).
When it comes to the design of automated testing system, many people's reaction is the pyramid model. However, with the development of testing theory to this day, the applicability of the pyramid model is no longer high (we will talk about it in detail later), we cannot just hold on to the old model and old ideas. What models need to be adopted in the business, or what new models can be born, are all guided by automated testing thinking. Many teams attribute the failure of automation application to the error of the model, which is biased. The problem is not the model, but the lack of thinking behind the model.
What is automated testing thinking ? The literal meaning is to make the testing work into a form that is automatically executed by the machine. Everyone should be able to understand this point, but the problem lies in the connection between ideas and practice. Understanding does not mean that you will use it. For example, do you think machine monitoring in the production environment is automated testing? It must be, it also solves the problem of error verification in the form of a system, no matter in terms of means or purpose, it is a kind of automated testing. So why do many people still think it is part of operation and maintenance?
Therefore, when we talk about automated testing, we should not be limited to unit testing, interface automation, and UI automation. Instead, we must consider whether automated testing capabilities can be formed in all aspects, and the input and output after adopting this capability Compared, making a comprehensive judgment , this is the thinking of automated testing. So far, the most successful application of automated testing thinking, I think there are two, one is the flow playback technology, and the other is the automation technology combining image recognition and machine learning. Don't worry, I will talk about it in detail later.
02. Automated test model
When it comes to automated testing models, the pyramid model must be an unavoidable threshold, and the ratio of 10->20->70 is regarded as the golden rule of automation practice. But now more than ten years have passed since the pyramid model was proposed, and the current business form, testing theory, and automation technology have undergone tremendous changes. To be practical, how many of your teams are strictly using this model? Is it just because domestic ideas have not kept up?
First, let's look at what the biggest challenge of automation is. We all know that the purpose of automation is to save labor costs, so if the cost of automation itself is high, do you still think it makes sense? of course not. Therefore, from the past to the present, the technological development of automation has been struggling with the issue of cost . Recall carefully, recording playback, traffic playback, screen comparison, image recognition, etc., which one was not born to reduce automation costs. Therefore, the basis of the pyramid model is based on the premise that the cost of using this model is optimal due to the technical constraints at the time.
现在我也找了很多测试的朋友，做了一个分享技术的交流群，共享了很多我们收集的技术文档和视频教程。
如果你不想再体验自学时找不到资源，没人解答问题，坚持几天便放弃的感受
可以加入我们一起交流。而且还有很多在自动化，性能，安全，测试开发等等方面有一定建树的技术大牛
分享他们的经验，还会分享很多直播讲座和技术沙龙
可以免费学习！划重点！开源的！！！
qq群号：110685036
But now it is 2023, the traffic replay technology makes the API (in fact, the traffic replay can be used for multiple levels, and I will elaborate on it later) the cost of automatic regression is close to 0, and the new machine learning technology based on image recognition also makes the UI automation The cost of regression is close to 0. The adoption of these two technologies makes the model directly evolve into a spindle shape or even an inverted triangle. Can we say it is unreasonable? Obviously not. Therefore, the cost is the core element that determines the "shape" of the model. Whether it is a pyramid, a spinner, or an inverted triangle, as long as the ROI is high, it is a good model .
In addition, even if it is a pyramid, the existing graphics are drawn in various ways. Integration testing is confused with interface testing, and interface testing is confused with end-to-end testing. For example, end-to-end, in fact, some expansions have been made to the original "system testing" concept. For Paas/Iaas services, the endpoint is often not an interface but an interface, or simply a command line. So we can't think that end-to-end is the interface, it's not the same thing.
Strictly speaking, there are two types of pyramid structures: one is based on granularity, such as unit test -> integration test -> system test (end-to-end); the other is based on hierarchical depth, such as device layer -> Coding layer -> interface layer -> interface layer . This does not mean that every layer must exist, and needs to be analyzed according to its own business.
03. Automated test coverage
After we have the model, do we just need to determine their coverage according to the "fat and thin" of each layer of the model? Or is it possible that the higher the coverage rate, the better? We are used to treating each level of automation separately and formulating their coverage indicators separately. Few people think about the relationship between levels and levels. Consider the following example, where different product prices give different discounts. Assume that the business code is as follows and an access interface is packaged:
public float getDiscount(float commodityPrice) {
float discount;
if (commodityPrice >= 500.0) {
discount = 0.3;
} else if (commodityPrice >= 300.0) {
discount = 0.2;
} else if (commodityPrice >= 100.0) {
discount = 0.1;
} else {
discount = 0.0;
}
return discount;
}Whether this logic is code (unit) testing or interface testing, it is easy to implement. Four use cases (without considering boundaries and exceptions for the time being) can be done, so most people will pursue the automation of code testing and interface testing 100% coverage (four codes + four interfaces), after all, the cost is not high. But in terms of effectiveness, the redundancy is as high as 75%. Because the business logic has been done at the code layer, for the interface test, we only need to ensure that the interface link is connected, and there is no need to repeat the verification of each logic branch.
For example, the water purifiers we use at home generally have 3 to 5 layers of filter screens, starting from larger objects such as sediment, to tiny objects such as bacteria, each layer has its purpose and function . If we use the finest filter material from the first layer, it will inevitably lead to material waste and cost increase.
The same is true for automation layering. We should start with the layer with the lowest implementation cost (again, not necessarily the code layer) to cover as many use cases as possible, and then sort the parts that were not covered by the previous layer according to the order of cost. Combined with the characteristics of the level itself to supplement. Therefore, the idea of automatic layering is actually a complementary idea, and should not be viewed independently .
One thing that needs to be reminded is that don't put too much faith in high coverage. For example the following code:
public float test(float divisor) {
return 100.0/divisor;
}We only need test(10.0) to achieve 100% coverage, but obviously the code doesn't handle the divisor-by-zero case, so we can't say that 100% (code) coverage is bug-free. Therefore, the coverage rate is divided into business coverage rate and code coverage rate, and there are many methods that can be used to test the effectiveness, such as mutation testing, chaos engineering, and so on. Since there is no rigid relationship between test effectiveness and automation, this article may briefly introduce the relevant content, and the rest will not be explained too much. Today's content about automation will be covered here for the time being, we will continue next week :)
The following are supporting learning materials. For friends who do [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse also accompanied me through the most difficult journey. I hope it can help you too!
Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
Information acquisition method:
