Thinking about automated testing -- getting started (1)
Start of automated testing
Automated testing, from a concept unfamiliar to everyone, now more and more people are paying attention to using automated testing. It seems that automated testing has become an "advanced". But in fact, many people have a lot of misunderstandings about automated testing itself. Automated testing is not a silver bullet, not a Swiss army knife. Automated testing is not a substitute for testing itself. Many leaders or customers learn about automated testing, and they will have the urge to cover tests with automated testing. Under this impulse, a lot of human and financial resources are invested, often without success. What the hell happened to automated testing?
Automated testing is not as perfect as you think
What are the problems with automated testing? Why do we often feel like we can't do it anymore? Why invest so much, the effect is always a little bit.
Automated testing itself has its own problems with automated testing. With a lot of investment in automated testing, these problems will become more and more obvious. For example:
- Writing and maintaining automated test scripts is too expensive
- I run automated tests every day, but the number of defects found is so small
- Why does automated testing not save labor costs?
- Why can only a few of you use automated testing, and so many manual testers can't use it?
These problems, in fact, are not problems with automated testing themselves, but misplaced automated testing itself. As I said at the beginning, automated testing is not really like this.
What do we do with automated testing
Automated testing generally requires scripting, and the purpose of the test is achieved through the execution of the script. Generally, if the system under test is stable, the script does not need to be maintained. Otherwise, it is terrifying. Therefore, to perform automated testing, you need to work hard to develop and maintain scripts, and the workload is not small. Because of this feature, automated testing, especially in business-oriented functional testing, is best not to desperately cover the entire business, cover all cases, and focus on key core businesses for regression testing. In this way, the workload of development and maintenance can be reduced, and the test quality of key businesses can be guaranteed as much as possible, and the test cost performance is the highest. 原则1: 针对重点业务，进行回归的自动化测试
In addition, try to conduct automated testing for relatively stable business or relatively stable testing methods, so that the human input is mainly concentrated in the writing of the initial script, and then because of the relatively stable, the maintenance workload of the script will not be too large. Of course, this kind of investment is not the sooner the better. Many project managers feel that automated testing is invested as early as possible, so that we can use it early in the project development and enjoy the convenience brought by automated testing as soon as possible. In fact, automated tests generally need to be developed when the environment is relatively stable, which can reduce maintenance costs. In addition, for unstable environments, executing automated tests is not very effective. There are a lot of problems in scripts that often run out. In fact, it can't be used for a long time, so the significance of the test is not great. 原则2：针对稳定的业务（或接口），在环境比较稳定的情况，前期投入脚本开发，有利于减少后期维护成本
There are also many automated tests, which are run almost every day, and the script errors are always found. How come there are no real defects? The probability of defects is so high, and the checkpoints are added blindly, and the result is still the same. In fact, there is no need to worry about not being able to find defects. Automation generally ensures that the main functions are completely available. These are its core values, not how many defects are found. 原则3：自动化测试主要是为了保证主要功能完整可用，而不是为了多发现缺陷
Of course, there are still some people who do automation to save the labor cost of manual testers. They are doing it, and they find out why the labor cost has not been reduced after doing automated testing for so long. Why don't I do automated testing? This is also a misunderstanding. Automated testing does not reduce the labor cost of testing, but to speed up test feedback and improve test quality. I suggest that automated testing can be bound with automated deployment tools. During daily builds, automated execution can detect product problems earlier and automatically feedback developers, thereby improving the efficiency of development and repairing defects.原则4：自动化测试并不能减少测试的人力成本，而是为了加快测试反馈，提升测试质量
There is also a problem, this problem is that people who start automated testing in the early stage will be more prone to misunderstanding-recording and playback is the best. In fact, recording and playback is not good. The recorded scripts cannot be used directly, and if the business or system changes, it is very likely that it is difficult to modify the script and need to re-record and restore. This workload is not small. And there are many people. The automated test script should be linked with the business, so that people can see at a glance what my automated test script has done. The script should be "visualized" . In fact, these seemingly conveniences bring a lot of difficulties to script maintenance. The business and system are changing, and the script needs to be maintained continuously. In fact, visualization and recording and playback do not improve efficiency, but increase the workload for script maintenance. It is recommended that if you have the ability, you can manage the script itself, write scripts without recording and playback, use some auxiliary tools, or design some frameworks, it will be better to write scripts. 原则5：不要对录制回放抱有幻想了，可视化也不是一个好的想法
The last question is that there are many people tasked with executing automated tests so easily. Can manual testers perform automated tests? May I? Many people hold different views. Some people’s task testing should actually become a development test, and they have the development ability to write automated test scripts; some people think that testing and automated testing should be separated and managed by two teams; some people’s tasks , test execution, test data writing, etc. can involve manual personnel. Actually, I don't think this is the best idea. What should really be involved in automated testing is development.原则6:开发参与自动化测试，让开发和测试融合在一起
Best Practices
Having said so many sesame and mung beans, whether there is a principle or an idea, is there a set of methods to support the entire automated test? I want to put it simply, I support the more popular saying of layered testing. Use a suitable testing framework, according to the scope theory of layered testing, and combine with better CI tools to integrate automated testing. The specific method I want to explain one by one in the following chapters.