We are full of hope for automated testing, however, automated testing often brings us frustration and disappointment. Although, automated testing can free us from a difficult environment, while implementing automated testing to solve the problem, it also brings the same many problems. In the work of automated testing, the key issue is to follow the basic rules of software development. This article introduces the seven steps of automated testing: improving the automated testing process, defining requirements, verifying concepts, supporting product testability, design for sustainability, planned deployment, and facing successful challenges. Follow the above 7 steps, arrange your personnel, tools, and develop your automated test project plan, and you will lead to a path to success.
a story:
I have worked in many software companies, large and small, and communicate with people from other companies, so I have experienced or heard of various problems that affect the effectiveness of automated testing. This article will provide several methods to circumvent problems that may arise in automated testing. Let me tell you a story first so that you can understand what problems will arise in automated testing.
Before, we had a software project, and everyone in the development team thought that automated testing should be used in the project. The manager of the software project is Anita Delegate. She evaluated all possible automated testing tools and finally chose one and purchased several copies. She appointed an employee, Jerry Overworked, to be responsible for automated testing. In addition to being responsible for automated testing, Jerry has many other tasks. He tried to use the automated testing tool he had just purchased. When applying testing tools to software product testing, problems were encountered. This test tool is too complicated and difficult to configure. He had to make a few calls to the customer support hotline of the test tool. Finally, Jerry realized that he needed the technical support staff of the test tool to help install the test tool and find out the problem. After making a few calls, the test tool manufacturer sent a technical expert. After the technical experts arrived, they found out the problem, and the test tool can work normally. This is going well. However, after a few months, they still didn't realize the test automation. Jerry refused to continue working on this project. He was afraid that automated testing would do nothing and just waste time.
The project manager Anita reassigned the project to Kevin Shorttimer, a person who had just been hired to do software testing. Kevin has just received a degree in computer science and hopes to use this job to move towards a more challenging and worthwhile job. Anita sent Kevin to participate in tool training, to avoid Kevin from following Jerry's footsteps-he became frustrated because of the difficulties in using the test tool, which led to abandon the project he was responsible for. Kevin is very excited. The testing of this project requires repeated testing, which is a bit annoying, so he is very willing to adopt automated testing. After a major version is released, Kevin is ready to start automated testing throughout the day. He is very eager to get an opportunity to prove that he can write very complex and difficult code. He built a test library and used some tricks to support most of the tests. This took a lot of time than the original plan. However, Kevin made the whole test work smoothly. He used the existing test suite to test the new product version, and indeed found a bug. Next, Kevin got an opportunity to work in a software development position and left the automation position.
Ahmed Hardluck takes over Kevin's work and is engaged in automated test execution. He found that Kevin left not only few documents, but also not much value. Ahmed spent a lot of time figuring out the existing test design and researching how to carry out test execution. During this process, Ahmed experienced many failures and was not sure whether the test execution method was correct. In the test execution, the error message after the execution failed did not have much reference value, and he had to delve deeper. Some test executions seem to never end. In addition, some test execution requires some specific test environment construction requirements. He updated the test environment construction documents and worked tirelessly. Later, during the automated test execution, it found several execution failure results. After analysis, it was found that there were bugs in the software version of the regression test, which caused the test execution to fail. After discovering the product bugs, everyone was happy. Next, he carefully analyzed the contents of the test suite, hoping to optimize the test suite to make the test more reliable, but this work has not been completed, and the expected optimization results have not been achieved. According to the plan, the next release version of the product had several major changes. Ahmed immediately realized that the product changes would destroy the existing automated test design. Next, in the new version of the test product, most of the test cases failed to execute. Ahmed studied the failed tests for a long time, and then sought help from others. After discussion, the automated test should be modified according to the new interface of the product so that the automated test can run. Finally, everyone modified the automated tests based on the new interface, and the tests passed. The product was released to the market. Next, the user immediately called to complain and complained that the software could not work. Everyone found out that they had rewritten some automated test scripts, causing some error messages to be ignored. Although the test execution actually failed, the result of the failed test execution was ignored due to a programming error when rewriting the script. This product finally failed.
This is my story. Perhaps you have personally experienced some plots in the story. However, I hope you do not have such a similar ending. This article will give some suggestions to avoid such an outcome.
problem
This story sheds light on several reasons why automated testing projects are in trouble:
Insufficient time for automated testing: According to the arrangement of the project plan, testers are often arranged to use their own personal time or intervene in automated testing later in the project. This makes it impossible to get enough time for automated testing and to get real attention.
Lack of clear goals: There are many good reasons to carry out automated testing. For example, automated testing can save time, make testing easier, improve test coverage, and allow testers to maintain better testing initiative. However, automated testing cannot simultaneously meet the above goals. Different people have different hopes for automated testing. These hopes should be raised, otherwise they are likely to face disappointment.
Lack of experience: Junior programmers who try to test their own programs often use automated testing. Due to lack of experience, it is difficult to ensure the smooth development of automated testing.
High turnover: Test automation often takes a lot of time to learn. When automated tests are frequently updated, you lose the automated testing experience you just learned.
The desperate response: The problem is already lurking in the software before the test has even started. Software testing just found these latent problems. As far as testing is concerned, testing is a very difficult task. When testing the modified software over and over again, the testers become tired. When will the test end? When the software should be delivered as planned, the despair of the testers becomes particularly intense. If you don't need testing, how great it would be! In this environment, automated testing may be an alternative solution. However, automated testing may not be the best choice. It is not a realistic solution, but more like a hope.
Reluctant to think about software testing: Many people find that implementing automated testing of products is more interesting than the testing itself. In many software projects, automation engineers do not participate in the specific activities of software testing. Due to the separation of test automation and human testing, many automations are not very helpful to software testing.
Focus on technology: How to automate software testing is a very attractive technical problem. However, too much attention has been paid to how to implement automated testing, which has led to neglect whether the automated testing program meets the testing needs.
Follow the rules of software development
You may know the CMM (Capability Maturity Model) proposed by SEI (Software Engineering Institute). CMM is divided into 5 sectors. This model is used to classify software development organizations. Jerry Weinberg (a well-known American software engineering expert) also created his own set of software organization models, adding an extra level to his organization model, which he called zero level. Obviously, a zero-mode organization actually develops software; in a zero-mode organization, there is no difference between developers and users [Weinberg 1992]. It is precisely in this type of organizational environment that automated testing methods are often used. Therefore, use resources for automated testing, and treat automated testing as a software development activity, and upgrade software testing automation to a level. This is the solution to the core of test automation. We should operate software automation testing projects like other development projects.
Like other software development projects, we need software developers to focus on the development tasks of test automation; like other software development projects, automated testing can automatically complete specific test tasks. For specific test tasks, automation developers may not Experts in this area, therefore, software testing experts should provide consulting related to specific test tasks and provide test automation requirements; like other software development projects, if the overall design of test automation is helpful to test before development and coding The smooth development of automation. Like other software development projects, automated test code needs to be tracked and maintained, so source code management is required. Like other software development projects, BUG will also appear in test automation. Therefore, it is necessary to track the BUG in a planned way and test the modified BUG. Like other software development projects, users need to know how to use the software, so a user manual is required.
This article does not cover software development too much. I assume that you belong to a software organization that already knows what reasonable and effective methods to use to develop software. I just push you to abide by the established software development rules during the automated test development process. This article is organized in accordance with the standard procedures adopted in software development projects, focusing on issues and challenges related to test automation.
● Improve the software testing process
● Define requirements
● Verify the concept
● Support product testability
● Design for sustainability (design for sustainability)
● Planned deployment
● Face the challenge of success
Step 1: Improve the software testing process
If you are responsible for improving the efficiency of a commercial transaction operation, first of all, you should confirm that the specific process of this operation has been well defined. Then, before you invest time and money to adopt a computer to provide an automated business transaction operating system, you want to know if you can use a simpler and lower cost method. Similarly, the above process is also used for automated testing. I prefer to interpret the term "test automation" to make the testing process simple and efficient, making the testing process faster and without delay. The automated test script running on the computer is just one aspect of the automated test.
For example, many test teams are beginning to adopt test automation methods in the regression testing process. Regression testing needs to be executed frequently and then executed to check that the effective test cases that have been executed have not failed because of software changes. Regression testing needs to be performed repeatedly and is tedious. How can we do a good job of documenting regression testing? The usual practice is to use a list of product characteristics, and then check against the list. This is a good start, and the regression test checklist can tell you what to test. However, the regression test checklist is only for those who understand the product and know which test method needs to be used.
Before you start test automation, you need to complete the regression test checklist mentioned above, and make sure that you have adopted a certain test method, indicate what kind of data is needed in the test, and give a complete method of designing the data. It will be better if the test has mastered basic product knowledge. After confirming that the above-mentioned documents can be provided, the detailed description of the test design needs to be clarified, and the expected results of the test should also be described. These are usually ignored and the tester is recommended to know. Too many testers don't realize what they are lacking, and are afraid to ask for help because they are afraid of embarrassment. Such a detailed document brings immediate results to the test team, because now anyone with basic product knowledge can carry out test execution based on the document. Before starting a more complete sense of test automation, the test design document must have been completed. Test design is the most important test requirement specification for test automation. However, at this time, do not go to extremes to explain each step of the test execution in detail. Just make sure that those who have the basic software operation knowledge can complete the test execution work according to the document. However, don't assume that they understand the software test execution ideas that remain in your mind, just describe these test design ideas clearly.
I used to be responsible for the automated testing of a software module. Some features of this module make it very difficult to automate. When I learned that this work does not need to be completed in a short period of time, I decided to develop a detailed regression test design plan. I carefully checked each closed defect related to the module in the defect tracking library, and for each defect, I wrote a test execution operation that can find the problem. I plan to use this method to provide a detailed list of automation requirements, which can tell me which part of the module is most suitable for automated testing. After completing the above work, I have no chance to complete the implementation of test automation. However, when we need to do a complete regression test on this module, I will provide the above-mentioned documents to a number of testers who only understand the tested product but have no testing experience. According to the guidance of the document, with almost no guidance, they completed the regression test and found the BUG. From a certain perspective, this is actually a very successful automated test. In this project, instead of developing automated test scripts, we might as well document the test execution steps. Later, in other projects, we developed automated test scripts and found that relevant personnel can only understand and execute automated test scripts if they receive relevant training. If the test automation is well designed, it may be better. However, after practice, we concluded that completing a well-designed test document is much simpler than completing a well-designed test script.
Another easy way to improve test efficiency is to use more computers. It's obvious that many testers use several computers at every turn. The reason why I emphasize the use of more computers is because I have seen some testers being misled to complete some large-volume automated test execution work on a single machine. In this case, the wrong use of test equipment and testing Environment, the test is not effective. Therefore, automated testing needs to focus on the required supporting equipment.
With regard to improving the software testing process, my last suggestion is to improve the tested product to make it easier to test. There are many improvement measures that can help users use the product better, but also help testers test the product better. Later, I will discuss the testable requirements of automated testing. Here, I just suggest to give the improvement points of the product, which is very helpful for manual testing.
Some products are very difficult to install, and testers spend a lot of time on installing and uninstalling software. In this case, instead of realizing automated testing of product installation, it is better to improve product installation functions. With this solution, the end user will benefit. Another approach is to consider developing an automatic installation program that can be released with the product. In fact, there are many commercial tools that specialize in making installers.
Other product improvements require the use of tools to find errors in the test execution log. Using manual methods, it is easy to be tedious to query error information page by page in the log. So, hurry up and adopt an automated method. If you understand the error message format recorded in the log, it is easy to write an error scanner. If you are not sure about the format of the error message in the log and start the handwriting error scanning program, you may face a disaster. Don't forget the example that the test suite mentioned in the story at the beginning of this article cannot determine whether the test case execution fails. End users are also unwilling to find error messages by searching logs. Modify the format of the error message to make it suitable for the log scanning program, so that the scanning tool can accurately scan all the error messages. In this way, scanning tools can be used in the test.
Improving product performance is also very helpful for testing. Obviously, if the performance of the product affects the test speed, identify the product function with poor performance, measure the performance of the product function, treat it as a defect that affects the test progress, and submit a defect report.
The above-mentioned aspects can greatly improve test efficiency without building an automated test system. Improving the software testing process will cost you the time to build an automated testing system, but improving the testing process will undoubtedly make your automated testing project more smoothly developed.
The above are some video resources I collected, which helped me a lot in this process. If you don't want to experience the feeling that you can't find the information during self-study, no one answers your questions, and insists on giving up after a few days, you can join our deduction group [313782132], which has various software testing resources and technical discussions.
Of course, there are interviews. Interviews are generally divided into technical and hr interviews. In terms of format, there are few group faces. A small number of companies may have an intersection, but in general, the technical aspect is basically an inspection of your professional and technical level. , then hr surface mainly to see the person's overall quality and family situation character does not meet company requirements, in general, technology, then as long as through the technical hr surface is basically no problem (there are a few companies hr face will brush a lot of people)
we Mainly speaking on the technical side, the technical side is mainly to examine the professional technical knowledge and level. The above are also selected interview questions I have compiled.
Come on, test man! If you need to improve your plan, do it. It's better to be on the road than to wait and see at the beginning. There must be a law, and then there will be success.
If the resources are good, give a recommendation~