Recently, I am preparing for an automated testing training at the newly hired company. This is when I learned that I will do automated testing training, I drew a picture at random, and I was shocked:
These are some points that I can think of about automated testing. Then I updated it based on an essay about automated testing that I wrote three years ago. Of course, unfortunately, the successful agile development projects I have come into contact with so far are still very good. Less, although Agile has been very popular in recent years. There is only one unsuccessful experience about agile automated testing, so I avoided this one in this article:
1. What is automated testing
Use programs to test programs, replace thinking with codes, and replace manual testing with scripts. Automated testing covers: functional (black box) automated testing, functional (white box) automated testing, performance testing, stress testing, GUI (Graphical User Interface) testing, security testing, etc.
Regarding what is automation, I have consulted some materials, and there is no authoritative and standardized explanation. The following is an excerpt from Wikipedia:
In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes.Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or add additional testing that would be difficult to perform manually.
First of all, there is a difference between test automation and automation test. Test automation covers a wider range. This article is about automated testing, so let's confuse these two concepts for now.
This paragraph is not difficult in English, translate it by yourself. Several points I saw in my eyes: 1. Need tools; 2. Tool control process, compare expected output and actual output; 3. High repeatability and necessary test process can be automated; 4. It is difficult to achieve manual testing field of
Talk about my own understanding:
Automated testing is an extension of testing thinking, providing test engineers with a "tentacles" whose behavior can be seen as a tool, but in essence, automated testing is still a kind of thinking.
By the way, automated testing in a narrow sense refers to automated testing based on GUI, while unit testing and API testing, have you ever thought about how to do it manually without any tools? So they are born in the category of test automation.
2. Advantages of automated testing
Regression testing is more convenient and reliable; it can run more and more cumbersome tests, and it is fast and efficient; it can perform some manual tests that are difficult or impossible to perform, such as a large number of concurrent users; make better use of resources and have consistency With the characteristics of repeatability, the automated test scripts are completely reusable; the credibility of the software is improved; testing under multiple environments, etc.
The most tangible advantage of automation is that it is easy to find a job: a test engineer (not herself) found an interesting phenomenon. Almost all test positions she has applied for require automated test experience when recruiting. But when she started working, she found that these companies were trying to do automated testing, but most of the results were not very good. However, although she is involved in some cup projects, she can always pack these cups into washware for the next interview.
So, since automated testing has so many advantages, why do so many projects fail?
I personally have an inference: the advantages of automated testing are the conclusions obtained from the successful completion of automated testing, and the disadvantages of automated testing are the basis for the establishment of automation projects.
There is another advantage: automated testing can solidify product knowledge into scripts to reduce the impact of the flow of testers on the project. But the premise of this advantage is that these scripts are easy to maintain, which requires some necessary documentation, which is another issue.
3. Things that cannot be done by automated testing and disadvantages
It is never possible to completely replace manual testing. Automated testing cannot achieve the coverage of manual testing. Not every test case is suitable for automation, such as suggesting whether the layout of a page is correct, installation testing, document testing, compatibility testing, recovery Sex test.
There are far more defects found in manual testing than in automation. Automated testing is almost impossible to find new defects, the biggest use is to regression, to ensure that the previous bug does not reappear in the new version.
Automated testing tools are dead, they don't have any imagination. The quality of automated testing depends entirely on the test engineer.
The cost is high and the risk is high. The technical requirements for testers are high, and there are also requirements for test tools.
Regarding cost, it includes capital budget, human resources, personnel training, hardware resources, etc. The following figure shows the relationship between the investment cost and time of automated testing. Obviously, most of the previous time, the cost is very high.
Based on the above disadvantages, although "you are" an automated test engineer, I spend more than half of my time advising the boss, "Dear, can you not do automation". This is really a sad story.
4. Appropriate introduction of automation
The project cycle is long, the system version is continuous, and the requirements are not frequently changed. At this time, it is suitable to introduce automated testing.
The test object of the system can basically be recognized normally, and whether it can provide a solution for the unrecognized control.
There are not a large number of unrecognizable third-party controls in the system.
It requires repeated testing, such as reliability testing, regression testing, etc., which require thousands of system tests.
5. Not suitable for automation
The project cycle is short and the requirements change frequently. Even for a long-period project, if the requirements change frequently, it is not suitable for automation.
If the software version is not stable yet, the main function or a large number of functions may be changed again, which is not suitable for automation.
There is no clear project test automation plan, measure and management.
Most objects are unrecognizable, and script maintenance is frequent and difficult. There is one of the two, and automation must fail.
6. The process of automated testing
Reasonable entry point for automation: Usually, the project only has the basic conditions for introducing test automation after undergoing a complete system test.
Personal opinion: Regardless of the test, the sooner it is involved, the better it will be to reduce costs and risks. With the emergence of new development models, automated testing also has the conditions for early intervention. For example, in agile development, after the core function of a core module is completed, automated testing can be started for that function of the module.
Test automation analysis:
(1) Feasibility analysis
(2) Sampling demo analysis, the demo generally selects smoke test cases to check whether the script can be successfully run and passed, and whether all the designed test points are executed
(3) Test requirement analysis, analyze which function points are ready to be automated
(1) Feasibility analysis is one of the most important parts of automated testing. Feasibility analysis is one of the most important parts of automated testing. Feasibility analysis is one of the most important parts of automated testing. The important words should be said three times.
Regarding the feasibility analysis, please refer to points 2, 3, 4, and 5; one of your wrong decisions (initiating an automated test project) is likely to bring full-time job opportunities to several people. From this perspective, it can also promote chicken Fart==
(2) Smoking Demo is mainly used to verify whether your tools can be used
(3) Automated testing is not 100% testing, and it is impossible to reach the coverage of manual testing. It is necessary to screen function points for automated testing
Test plan customization: The more comprehensive the automated test plan, the more routinely implemented later, the higher the success rate of automated testing
The plan cannot keep up with the changes, and sometimes it may not be a good thing to be too comprehensive.
Automated test design phase: mainly divided into automated test framework and automated test cases.
(1) Design, development and construction of automated test framework: it should be able to ensure the distributed execution of the test, modularization of scripts, data-driven, log analysis, error screenshots, report recovery, shared object library, common function library, environment configuration, unified design Mode, exception handling, scene recovery, an unattended test framework for each independent project
Regarding why an automated testing framework is needed, I have another article that explains it in detail, so I won’t repeat it here.
Then, by the way, I will talk about finding the object, it is the automated testing framework to find the object, not me looking for the object:)
Generally, each framework should support two ways to find objects, dynamic and static. Static search involves object libraries, including a series of issues such as reading, writing, merging, and maintaining object libraries. These can be done by the framework;
Regarding dynamic search, let me give you an example of RFT. What do you think:
Two types of find API in Rational Functional Tester:
find(Subitem Properties).
find(Subitem Properties, Boolean mappableOnly).
Subitems can be either atChild() or atDescendant() or atList().
atChild: One or more properties that must be matched against the direct child of the starting TestObject.
atDescendant: One or more properties that can be matched against any child of the starting TestObject
atList: A sequential list of properties to match against. Valid subitems for atList are atChild, atDescendant, and atProperty.
mappableOnly: Arguments that limit the search. If it is set to true, the search for children will be limited to those test objects that are mappable, otherwise non-mappable test objects are also searched.
First, the test tool will provide interfaces or methods for dynamic search. The RFT provides the find method. You can call these interfaces or methods to achieve dynamic search.
The advantage of dynamic search is that the "relative path" can be used to locate the object, while relative, the object library uses the "absolute path". If some properties of the object are changed, the static search method may not find the object. Of course, the automated testing is becoming more and more intelligent, and it is possible to select the object with the highest matching degree and return. Another advantage of dynamic search is that the objects it finds are "code", you can further process these objects in the framework, and each object in the object library is an independent object, you can use them, but It is difficult to change them.
Usually, the current automated testing framework adopts a combination of dynamic and static, that is, both methods of finding objects will be taken into account, because in general, the static search method is faster and more efficient. However, the problems caused by static search are also significant, mainly focusing on the maintenance, management and merging of objects, how to share objects, and avoid adding objects repeatedly. At this point, it is very important to standardize the object naming. In the automated test projects I have done in the past, these are all pits.
(2) Automated test case design trilogy: manual test cases are created from scratch, and then automated test cases are written based on manual test cases. First, screen manual test cases. Then convert manual test cases, and finally add & supplement automated test cases.
Why can't manual test cases completely replace automated test cases?
The scope of automated test cases is often the core business process or the repetitive execution rate is high, and the coverage rate of automated testing cannot reach the coverage rate of manual testing. The use case selection of automated testing is generally based on the forward direction, while there are many reverse situations. However, not all reverse situations are covered by automated testing, but there are selected parts of the filter. Not all manual test cases can be used for automation, such as page layout checks. Manual testing may not need to go back to the origin, but automated testing is often necessary. Automated test cases are different from manual test cases, and there is no need to write expected results in every step.
Usually when doing automated testing, I will write a test case called shake-down test. This use case will go through all the completed forms in the system, just do a Navigate operation to ensure that a certain page is available.
Before each regression test, you can run the shake-down test first, and you can quickly determine which functions are accessible, which is equivalent to doing a smoke test of the entire system.
Test script design and development:
The test script can be roughly divided into:
(1) Linear script: linear executable script generated directly by recording
(2) Structured scripts: scripts with sequence, loop, branch and other structures
(3) Shareable scripts: scripts that can be used by multiple test cases and called by other scripts (ie modular scripts)
(4) Data-driven script: a script that separates test data from business process control, and a script that drives the process by reading in data files
(5) Keyword-driven script: script, data, business separation, data and keywords are in different data tables, and keywords are used to drive test business logic. The characteristic of keyword-driven is that it is more like describing what a test case is doing, rather than how it is done.
(6) Mixed script: any two or more of the above
(7) Agile automated test script/framework: I will add this one after I have successful experience =. =
Automated test execution:
(1) Unattended testing: environment construction, deployment and configuration; automated test cases and test scripts are bound to each other; automated test case execution sequence and combination
(2) Exception handling and scene recovery
Submit automated test products: roughly need to submit implementation status, test results, analysis reports, test reports, quality conditions, etc.
Test script maintenance: Strictly speaking, test script maintenance is being done at each stage. An automated test project that is not worth maintaining is not worth setting up.
If you
① Engaged in functional testing and want to advance automated testing
②I have been in the testing industry for one or two years, but still can’t type code
③ Interviews with big companies but repeatedly bumped into walls
I recommend a group! Come on~~ Tester, 313782132 (There are technical experts in the Q group to communicate and share, the value of learning resources depends on your actions, don’t be a "collector") to get more technology and interview materials from big factories