What is automated testing?
I have been doing testing for several years, and I have really learned and practiced automated testing for a year. I feel that I have gained a lot during this year. I have always wanted to write an article to share some experience in automated testing practice. Finally decided to take some time to do this.
First, clarify the concept of automated testing. Broadly speaking, automation includes all activities that replace or assist manual testing through tools (programs), which can be regarded as automation, including performance testing tools (loadrunner, jmeter), or written by yourself A section of program used to generate 1 to 100 test data. In a narrow sense, the process of manual testing is simulated by means of tool recording or scripting, and test cases are executed by replaying or running scripts, thereby replacing manual verification of system functions.
Of course, we are more generally aware of "automated testing" as "automated testing based on the UI layer of the product or project."
Layered automated testing
This concept has been exposed recently. Traditional automated testing pays more attention to automated testing at the UI layer of the product, while hierarchical automated testing advocates automated testing at different stages (levels) of products.
I believe that the test students are not unfamiliar with the above pyramid. Isn't this the test corresponding to the different stages of product development? We need standardized unit testing and also need corresponding unit testing frameworks, such as Java’s Junit, testNG, C#’s NUnit, python’s unittest, pytest, etc. Almost all mainstream languages will have their corresponding unit testing frameworks.
Integration and interface testing are not easy to understand for many novice testers. Unit testing focuses on the implementation logic of the code, such as the implementation of an if branch or a for loop; then integration and interface testing focus on a function, class (method) Whether the provided interface is reliable. For example, if I define an add() function to calculate the result of two parameters and return it, then I need to call add() and pass the parameters, and compare the return value whether the two parameters are added. Of course, the interface test can also be passed in the form of url. For example, if we send a request to the server through the get method, the content we send is passed to the server as part of the URL. But for example, a public interface provided by the Web service technology needs to be tested through tools such as soapUI.
You should be familiar with the automated testing of the UI layer. Most of the work of most testers is to test the functions of the UI layer. For example, we repeatedly test a form submission, result query and other functions, and we can simulate these operations through the corresponding automated testing tools, thereby freeing up repetitive labor. There are many automated testing tools for the UI layer, and the mainstream ones are QTP, Robot Framework, watir, selenium, etc.
Why draw a pyramid shape instead of a rectangle or an inverted triangle? This is to indicate the proportion of automated testing invested in different stages. If a product has never done unit testing and interface testing, it is unscientific to only do automated testing at the UI layer, making it difficult to essentially guarantee the quality of the product. If you try to achieve a comprehensive UI layer automated testing, it is a move that will hurt the people and money, and invest a lot of manpower and time, and the final benefit may be far lower than the cost paid. Because the higher the level, the higher the maintenance cost. Especially the elements of the UI layer will change from time to time. Therefore, we should put more automated testing in the unit testing and interface testing phases.
Since the automated testing of the UI layer is so laborious, we can only do unit testing and interface testing. NO! Because no matter what kind of product, the UI layer is finally presented to the user. Therefore, testers should focus more on the UI layer. So it is precisely because the testers put a lot of effort into the UI layer, so it is necessary for us to help us "partially liberate" the repetitive work through automation.
The most feared thing in automated testing is change, because the direct result of the change is the failure of the test case, so the automation script needs to be maintained; how to control the failure and reduce the maintenance cost is crucial to the success or failure of the automation. Conversely, an automated test case that always runs successfully is worthless.
As for the ratio of the three tests in the pyramid, it should be divided according to actual project requirements. In the book "Google Test Method", for Google products, 70% of the investment is unit testing, 20% is integration and interface testing, and 10% is UI layer automation testing.
Why should I do automated testing?
According to the "Survey Report of Chinese Software Testing Practitioners" by 51testing, manual testing accounts for 89% of the total. Relative to development, the threshold for testing is low and the salary is generally low. Although the required knowledge has a certain breadth, it lacks depth. This is the general status of testing.
It is precisely because of the low threshold of manual skill testing that a large number of graduates, even non-professionals, have poured into this industry. Thereby increasing the fierce competition in this industry. For those who have worked for several years and are in manual testing, there will be a strong sense of crisis. As the technical content of the work is not high, the salary increase has encountered a bottleneck. On the other hand, it is threatened by new entrants. The same job company can do it with 5K recruits, so it will not spend 8K.
Well, this question shouldn't appear in the topic of discussing technology, but it is indeed a problem that most testers have to face. Therefore, in terms of the development of testers themselves, I actually need to use automation technology to increase my competitiveness. Of course, testers who have reached a certain number of years will choose to transfer to management or other positions, which is another topic.
From the perspective of the development of the testing industry, due to the product characteristics of domestic products, there are not many world-class products, relatively low technical content, relatively low quality requirements, outsourcing foreign projects, and low testing labor costs, so a large number of manual testers are required .
Therefore, in the not-too-distant future, I think the demand for pure manual testers is diminishing, and the company needs more testing with higher technical capabilities. Quality needs testing. Testing behavior will never disappear, but it is possible whether pure manual testers disappear.
Well, you can say that testing the industry with a lot of sunshine, I am purely alarmist. Regardless of the future, we all need to improve our skills, right!
What project is suitable for automated testing?
If you have decided to learn automated testing, how to learn is the next problem to face? This problem is analyzed based on the tested product. If the technology you have learned cannot be applied (verified), it will make your learning process difficult.
First consider whether the product is suitable for automated testing. The general consensus on this method is to weigh from three aspects.
Software requirements change infrequently
The stability of the test script determines the maintenance cost of the automated test. If the software requirements change too frequently, testers need to update the test cases and related test scripts according to the changed requirements, and the maintenance of the script itself is a code development process that needs to be modified, debugged, and if necessary, the automated test must be modified. Framework, if the cost is not less than the test cost saved by using it, then the automated test will fail.
Some modules in the project are relatively stable, while the requirements of some modules are highly variable. We can perform automated testing on relatively stable modules, while manual testing is still used for larger changes.
Long project cycle
Because the determination of automated testing requirements, the design of automated testing frameworks, and the writing and debugging of test scripts all take a long time to complete. Such a process itself is a development process of testing software, and it takes a long time to complete. If the project cycle is relatively short and there is not enough time to support such a process, then automated testing becomes a joke.
Automated test scripts can be reused
The reuse of automated test scripts should be considered from three aspects. On the one hand, whether the tested items are very different (such as the difference between the C/S system and the B/S system); whether the selected test tool is suitable This difference; finally, whether the tester has the ability to develop an automated test framework that adapts to this difference.
What tool to choose for automated testing
If you have confirmed that the XX project is suitable for automated testing, then the next thing you have to do is to choose a testing tool.
First of all, confirm whether the product you are testing is a desktop program (C/S) or a web application (B/S).
Tools for desktop programs are: QTP, AutoRunner
Tools for web applications: QTP, AutoRunner, Robot Framework, watir, selenium
Due to the many advantages of the B/S architecture, a large number of C/S architecture applications were converted to the B/S architecture a few years ago. This also promotes the development of web development and testing technology. If the product being tested is of C/S architecture, then QTP is recommended. QTP accounts for half of the trial rate in the field of UI automation testing. Therefore, it is sufficient to show that QTP is powerful and easy to use in the field of automation. Learning mainstream tools can also give you more opportunities. Books on QTP on the market are also very rich. Of course, if you want to learn QTP well, you must master the VBS scripting language.
If the tested product is a B/S structure, then selenium is recommended, why not QTP or other tools? Because selenium supports B/S applications very well, and more importantly, it supports multi-language development. To really try selenium, you have to master more than just a tool, you also need to learn a language. Why should I choose selenium? I also need to learn a language, which undoubtedly increased my learning costs. While increasing the cost, it also increases your competitiveness. In the process, you not only learn an automation tool, you can use the language you learn to do more things.
Ok! If you decide to try selenium, you are faced with a new problem, choose a language. Selenium supports java, python, ruby, php, C#, JavaScript.
In terms of language ease of learning, ruby and python are preferred
In terms of language application breadth, java, C#, php,
In terms of language-related testing technology (and information): ruby, python, java
Or you can consider the mainstream language used by the entire technical team, and then choose the corresponding language.
Before using selenium
OK! After the above process, I believe you must make the corresponding choice. If you choose the selenium tool, then read on.
First, you need to spend one to two months to learn a language before you start selenium. This is based on students who have no language foundation. I recommend ruby, python, java any language for learning.
Of course, if you have a good language foundation to skip this link, or your rich java programming skills, then learning python may only take a few days or less.
If you have already gotten the foundation of a language, then you need to understand selenium first. Selenium is not just a tool. It is a collection of tools. Moreover, it is divided into 1.0 and 2.0. Of course, 3.0 is also Has arrived.
Selenium is not a simple tool, but consists of several tools, each of which has its own characteristics and application scenarios.
selenium IDE
Selenium IDE is a plug-in embedded in the Firefox browser to realize simple browser operation recording and playback functions. So when is it used?
Quickly create bug reproduction scripts. During the testing process of testers, after finding bugs, they can record the reproduction steps through the IDE to help developers reproduce the bugs more easily.
The script recorded by the IDE can be converted into multiple languages to help us develop the script quickly. We will introduce this function in detail later when we use it.
selenium Grid
Selenium Grid is an automated testing aid. Grid can speed up the functional testing of Web-app by using the existing computer infrastructure. Using Grid, you can conveniently run multiple test cases in parallel on multiple machines and in a heterogeneous environment. Its characteristics are:
- Parallel execution
- Use a host to uniformly control the use cases to run in different environments and different browsers.
- Flexible addition of change test machines
selenium RC
Selenium RC is the core tool of the selenium family. Selenium RC supports a variety of different languages to write automated test scripts. The selenium RC server serves as a proxy server to access applications to achieve the purpose of testing.
Selenium RC uses Client Libraries and selenium Server. The Client Libraries library is mainly used to write test scripts to control the selenium Server library.
Selenium Server is responsible for controlling browser behavior. In general, Selenium Server mainly includes three parts: Launcher, Http Proxy, and Core. Among them, Selenium Core is embedded in the browser page by Selenium Server. In fact, Selenium Core is a collection of JS functions. It is through these JS functions that we can use programs to operate the browser. Launcher is used to start the browser, load Selnium Core into the browser page, and set the browser's proxy to Selenium Server's Http Proxy.
selenium 2.0
I understand the family relationship of selenium 1.0, selenium 2.0 adds WebDriver to this family; simply expressed as a formula:
selenium 2.0 = selenium 1.0 + WebDriver
What needs to be emphasized is that in selenium 2.0, the main recommendation is WebDriver. WebDriver is a replacement for selenium RC. Because selenium is backward compatible, selenium RC has not been completely abandoned. If you use selenium to develop a new automated test project, it is strongly recommended It is recommended to use WebDriver. So what is the main difference between selenium RC and webdriver?
Selenium RC runs JavaScript applications in the browser, and uses the built-in JavaScript translator of the browser to translate and execute selenese commands (selenese is a collection of selenium commands).
WebDriver directly controls the browser through native browser support or browser extension. WebDriver is developed for each browser and replaces JavaScript embedded in the tested web application. The tight integration with the browser supports the creation of more advanced tests, avoiding the restrictions caused by the JavaScript security model. In addition to support from browser vendors, WebDriver also uses operating system-level calls to simulate user input.
If it is a new project, it is OK to directly learn webdriver, RC is an outdated technology.
Selenium learning route
Configure your test environment to configure your corresponding selenium test environment for the language you are learning. Selenium is like the definition of semantics—"Hello". If you are using Chinese, to express hello, your writing is "Hello". If you are using English, your writing is "hello". Therefore, the same semantics will have different writing (grammar) in different languages.
Then you need to be familiar with the webdriver API, which is a method defined by selenium for locating and operating various elements on the page.
First learn the positioning of elements. Selenium provides positioning methods such as id, name, class name, tag name, link text, partial link text, xpath, css, etc. The powerful syntax of xpath and css is slightly complicated, and you may need to know more front-end knowledge in the meantime. xml, javascript etc.
The purpose of locating elements is to operate the elements. Then you have to learn about various elements including operations, input boxes, drop-down boxes, button clicks, file uploads, downloads, paging, dialog boxes, warning boxes... etc.
After a period of study, you can easily simulate manual testing to manipulate various elements on the page. Then all you need to do is to organize these "use cases" and run them together.
Then all you need to do is to learn and use the unit testing framework. The unit testing framework itself solves the organization and operation of use cases.
After you write some "test cases", you will find that there are a lot of repeated operations in the use cases. Can you write them in a separate file and call these operations when needed? Of course, it will be very simple to use your programming skills to achieve this. Then, you find that there are some data in each use case. These data are also the same, but if it changes and it is very troublesome to modify, you can also write it to a separate file for reading.
Then you encounter new questions. The scripts (use cases) I wrote are all streamlined. How do I know if the use case fails or succeeds? Then you need to add some verification and assertion to the script.
Then you have more ideas. The log of the unit test framework is too crude. Can you generate a beautiful test report? Can I run this script regularly? Can you send the test results of each script run directly to my mailbox? Can you...
To solve these problems, you have to learn more programming techniques, and then your "test structure" will become more powerful and more flexible. Produced a certain degree of versatility and portability. A realistic automated testing framework was born.
If one day you no longer do automated UI testing, you will find that there is basically no difficulty in doing unit testing or interface testing. It's not a problem to develop a test tool, thank you Selenium! By the way, thank me too!
Recommend good articles:
A pack of sad spicy strips to share-top ten best automated testing tools
About software testing! Everything you want to know is here, Xiaobai must see!
10 years of software testing engineers' perceptions-to friends who are still confused
What kind of person is suitable for software testing?
Knowledge to understand python automated testing(3)
Which is more suitable for automated testing, Python or Java?
The daily work of software testers
Play with Python+Selenium automated testing in 10 minutes, and teach you a quick start!
Finally: Welcome to follow the editor to receive a summary of the core knowledge of Python automated test engineers with a 300-page pdf document! Software testing technology exchange group: (313782132) The content of these materials are all the knowledge points that the interviewer must ask during the interview. The chapter includes many knowledge points, including basic knowledge, Linux essentials, Shell, Internet program principles, Mysql Database, package capture tool topics, interface testing tools, advanced testing-Python programming, Web automation testing, APP automation testing, interface automation testing, advanced continuous integration testing, testing architecture development testing framework, performance testing, security testing, etc.